Google Protocol Buffers
in [TCL]
|
Prev: Tk8.6, tile, png images and transparency
Next: Using Expect to automate file transers through Telnet with YMODEM...
From: nedbrek on 28 May 2010 09:29 Hello all, I am trying to decode Google Protocol Buffers from inside Tcl. (The data format spec is http://code.google.com/apis/protocolbuffers/docs/encoding.html) First, I am looking at decoding the varint (which is the key to most stuff). So far, I have: # find the var int starting at offset i # return tuple with new i and value proc getVarInt {i tl} { # split the data into a list of bytes set tl [split $tl {}] # return value set v 0 # place multiplier set mul 1 # pull the first byte (lsb) binary scan [lindex $tl $i] c b # check for multi-byte while {$b < 0} { set v [expr $v + ($b & 0x7f) * $mul] set mul [expr $mul * 128] incr i binary scan [lindex $tl $i] c b } set v [expr $v + $b * $mul] incr i return [list $i $v] } (For most uses, the split would be moved up to the caller) This form allows: getVarInt 0 [binary format cc 128 1] Which should return {2 128} (Next index = 2/end of list, value = 128) Any improvements to this code? I have it posted on the wiki at http://wiki.tcl.tk/21674 Feel free to change it there, or copy it off to another page. Thanks, Ned
From: hae on 28 May 2010 09:00 On 28 Mai, 15:29, "nedbrek" <nedb...(a)yahoo.com> wrote: > Hello all, > I am trying to decode Google Protocol Buffers from inside Tcl. (The data > format spec ishttp://code.google.com/apis/protocolbuffers/docs/encoding.html) > > First, I am looking at decoding the varint (which is the key to most stuff). > > So far, I have: > # find the var int starting at offset i > # return tuple with new i and value > proc getVarInt {i tl} { > # split the data into a list of bytes > set tl [split $tl {}] > > # return value > set v 0 > > # place multiplier > set mul 1 > > # pull the first byte (lsb) > binary scan [lindex $tl $i] c b > > # check for multi-byte > while {$b < 0} { > set v [expr $v + ($b & 0x7f) * $mul] > set mul [expr $mul * 128] > > incr i > binary scan [lindex $tl $i] c b > } > > set v [expr $v + $b * $mul] > incr i > > return [list $i $v] > > } > > (For most uses, the split would be moved up to the caller) > > This form allows: > getVarInt 0 [binary format cc 128 1] > > Which should return {2 128} (Next index = 2/end of list, value = 128) > > Any improvements to this code? I have it posted on the wiki athttp://wiki.tcl.tk/21674 > > Feel free to change it there, or copy it off to another page. > > Thanks, > Ned There was a post in January 20th about the Google Protocol Buffers. See http://groups.google.com/group/comp.lang.tcl/tree/browse_frm/thread/500dba19fc8a4f46/786d19fb4f075be4?rnum=1&q=Google+Protocol+Buffers&_done=%2Fgroup%2Fcomp.lang.tcl%2Fbrowse_frm%2Fthread%2F500dba19fc8a4f46%2F811ea06184bc9761%3Flnk%3Dgst%26q%3DGoogle%2BProtocol%2BBuffers%26#doc_786d19fb4f075be4 Maybe there is some code that can be shared. Ruediger
From: nedbrek on 28 May 2010 11:41 "hae" <r_haertel(a)gmx.de> wrote in message news:31c092a3-ae16-4440-996c-9b8eb4ae2a83(a)o39g2000vbd.googlegroups.com... > On 28 Mai, 15:29, "nedbrek" <nedb...(a)yahoo.com> wrote: >> Hello all, >> I am trying to decode Google Protocol Buffers from inside Tcl. (The data >> format spec >> ishttp://code.google.com/apis/protocolbuffers/docs/encoding.html) > > There was a post in January 20th about the Google Protocol Buffers. > > Maybe there is some code that can be shared. Weird, I did a Google search for "protocol buffer" and didn't find anything... Ned
From: Zarquon on 28 May 2010 11:40 nedbrek wrote: > "hae" <r_haertel(a)gmx.de> wrote in message > news:31c092a3-ae16-4440-996c-9b8eb4ae2a83(a)o39g2000vbd.googlegroups.com... >> On 28 Mai, 15:29, "nedbrek" <nedb...(a)yahoo.com> wrote: >>> Hello all, >>> I am trying to decode Google Protocol Buffers from inside Tcl. (The data >>> format spec >>> ishttp://code.google.com/apis/protocolbuffers/docs/encoding.html) >> There was a post in January 20th about the Google Protocol Buffers. >> >> Maybe there is some code that can be shared. > > Weird, I did a Google search for "protocol buffer" and didn't find > anything... > > Ned > The thread has the subject "protobuf", and was started by Message-Id: <b906cb96-5ce0-4b7a-a4a5-c7fe3c5b3685(a)e37g2000yqn.googlegroups.com>. Using http://al.howardknight.net/ to search with the "Google Search Only" box checked, the search returns "could not find", but if I uncheck the "Google Search Only" box, it finds the article in question. Odd. The post in question was posted through Google Groups... Ah, well. Here's the Message-Ids of all the posts in that thread. <b906cb96-5ce0-4b7a-a4a5-c7fe3c5b3685(a)e37g2000yqn.googlegroups.com> -> <0404d6f8-503f-4c1e-8de7-5e3f87576611(a)c34g2000yqn.googlegroups.com> -> <4b56fdc0$1(a)news.x-privat.org> -> <hj6uog$1nhk$1(a)saria.nerim.net> --> <b906cb96-5ce0-4b7a-a4a5-c7fe3c5b3685(a)e37g2000yqn.googlegroups.com> ---> <uPJ5n.104264$o06.78184(a)en-nntp-08.dc1.easynews.com> ---> <938fb759-8cdc-4c81-9a01-68253fa83b66(a)l19g2000yqb.googlegroups.com> -> <fd657dea-b468-4582-b881-5b12997426f2(a)b2g2000yqi.googlegroups.com> --> <72deac0e-03db-4f8a-b5a4-04aa148735f8(a)a15g2000yqm.googlegroups.com> -> <58ij27-kvk.ln1(a)klein-habertwedt.de> I don't know if that thread will help any, though. -- I think someone needs to graph these results so we can find out if connecting the dots makes a kitty or a turtle. -- Kibo
From: tom.rmadilo on 28 May 2010 12:37
On May 28, 6:29 am, "nedbrek" <nedb...(a)yahoo.com> wrote: > Hello all, > I am trying to decode Google Protocol Buffers from inside Tcl. (The data > format spec ishttp://code.google.com/apis/protocolbuffers/docs/encoding.html) > > First, I am looking at decoding the varint (which is the key to most stuff). Is this really the extent of your interest? This protocolbuffers thing seems totally useless in Tcl since you would have to compile the .proto file for a service and provide, or create an interface to the library code provided by Google. Assuming you use their library, you should not need to encode/decode anything, which is the whole point of their scheme: create a compressed wire format for structured data. Given the short history of this project, I would suggest creating a Tcl interface to the C++ library so you can more easily pick up changes...maybe look into using incrTcl. The project concept is against the current trend to hide internal program data structures, types and API, but is obviously optimized for bandwidth minimization and parsing speed reduction. I don't think that they will ultimately achieve protocol and type safety since it is allowed to include additional structure members and the order isn't defined...it could be different on different platforms. The data is also not self describing, so it doesn't solve the ultimate problem of persistent storage. |