S-expression I/O in Ada
in [ADA]
|
Prev: GPRbuild compatibility
Next: Irony?
From: Natacha Porté on 12 Aug 2010 04:53 On Aug 12, 1:39 am, "Randy Brukardt" <ra...(a)rrsoftware.com> wrote: > But there is no *requirement* to take those concepts into account in order > to write Ada code. There is no requirement that you have to separate the > creation and writing of atoms. That's exactly to address these point that I (tried to) imagined Sexp_Stream; that was a while back and I haven't felt any progress between before and after this. It's very depressing to feel there is something here without being able to grasp it. > Your code would be better in *any* language > if you did, but it isn't always obvious how to accomplish that, and > depending on the situation, it might not be worth the mental effort to do > it. I have to admit I was quite surprised to find my old way of using S- expressions to be so extremely more difficult in Ada compared to C (whaddoyoumean I can't just feed to strcmp() raw data I've just read from a file!?). But on other hand it might be a good thing to be shaken into realizing the deep implication from the difference in typing strength. > In any case, you are spending way too much effort trying to figure out how > to fit your work into what Dmitry is suggesting. That's because I feel there is something there that might help a lot in my personal improvement. Even I end up not using his point of view, for any rational or non-rational reason, I'm sure there is something to get from understanding it. > Bob and I have both tried > to point out that there are lots of other ways to structure programs over > what Dmitry is suggesting. Thanks a lot to both of your for that. > Don't let one person's opinions push you away from Ada!! Indeed, I have to admit I overreacted. It looks like I'm still vulnerable to unrelated personal problems overflow, I'll have to find a way to patch that. Thanks for your help, Natacha
From: Georg Bauhaus on 12 Aug 2010 05:22 On 8/12/10 10:53 AM, Natacha Port� wrote: > I have to admit I was quite surprised to find my old way of using S- > expressions to be so extremely more difficult in Ada compared to C > (whaddoyoumean I can't just feed to strcmp() raw data I've just read > from a file!?). You can "=" just those bytes: you'll just have to drop type checking by using an instance of Unchecked_Conversion of the-bytes-you-read to String. > But on other hand it might be a good thing to be > shaken into realizing the deep implication from the difference in > typing strength. The 12K SLOC Python programs on my screen surely would be a little easier to rewrite if Python had explicit typing. assert isinstance(this, that) seems a helpful workaround, but tedious in comparison to strong typing during translation...
From: Natacha Kerensikova on 12 Aug 2010 05:26 On Aug 10, 8:13 pm, Jeffrey Carter <spam.jrcarter....(a)spam.not.acm.org> wrote: > On 08/09/2010 11:47 PM, Natacha Kerensikova wrote: > I was commenting on your ideas about writing an S-expression to a stream, which > included operations to open and close a list, and put an atom. I thought this > was how you expected a client to output an S-expression. You thought right, except that due to my discussion with Dmitry I ended up considering the possibility of splitting my first idea into two different packages. Sexp_Stream is supposed to perform S-expression I/O over streams, without ever constructing a S-expression into memory. It is supposed to do only the encoding and decoding of S-expression format, to expose its clients only atoms and relations. A further unnamed yet package would handle the memory representation of S-expressions, which involve object creation, memory management and things like that. It would be a client of Sexp_Stream for I/O, so the parsing would only be done at one place (namely Sexp_Stream). As I said Ludovic Brenta's code might take this place. There are situations when a memory representation of S-expressions is not needed, and the tcp-connect example here seems to be such a case. That's why I imagined TCP_Info as a client of Sexp_Stream instead of a client of the second package. The stuff with operations to put an atom and open and close a list is indeed how I think a client of Sexp_Stream should behave. And if I want Sexp_Stream to have no in-memory representation of a S- expression, I can't imagine another way of specifying it. However I'm still undecided whether it's a good thing to have a Sexp_Stream without memory representation, and whether packages like TCP_Info should be a client of Sexp_Strean or of the in-memory S- expression package. > I think the client should just do > > S : S_Expression := Some_Initialization; > > Put (File => F, Expression => S); To achieve such an interface, the client has to build an in-memory S- expression object. In the tcp-connect example, there are eight nodes (five atoms and three lists). They have to be somehow built, and it doesn't look like a simple initialization. The second interface I proposed, with a lot of nested calls, builds the S-expression object with functions looking like: function NewList(Contents, Next: S_node_access) return S_node_access; function AtomFromWhatever(Contents: whatever, Next: S_node_access) return S_node_access; However, unless I'm building other higher-level functions, to build 8 nodes I need 8 function calls, exactly like the second example. That 8-function-call example looks very C-ish to me, and honestly I don't like it, I prefer the sequence of procedure calls from Sexp_Stream example, but that's just a personal taste. The problem is that I don't see how to do it otherwise. Any idea would be welcome. For the TCP_info stuff particular case, the only simplification I can imagine is leveraging the fact that only 2 among the 8 nodes change. So one could construct a global S-expression skeleton with the 6 constant nodes, and when actually writing data append to the "host" and "port" atoms the relevant variable atoms. However that means having a read/write global variable, which is bad for tasking, so one might prefer a constant 6-node skeleton, duplicated in the declarative part of the writing procedure, and then appending the variable nodes. However that appending would probably end up with lines like: root_sexp.FirstChild.Next.FirstChild.Append(host_node); root_Sexp.FirstChild.Next.Next.FirstChild.Append(port_node); which looks very ugly to me. > Your TCP_Info-handling pkg would convert the record into an S-expression, and > call a single operation from your S-expression pkg to output the S-expression. That's the tricky part. At least so tricky that I can't imagine how to do it properly. > Your S-expression library would implement the writing of the expression as a > series of steps. Indeed, as a series of calls similar to my Sexp_Stream example. I'm glad to have at least that part right. Thanks for your comments, Natacha
From: Ludovic Brenta on 12 Aug 2010 06:55 Natacha Kerensikova wrote on comp.lang.ada: [...] > Sexp_Stream is supposed to perform S-expression I/O over streams, > without ever constructing a S-expression into memory. It is supposed > to do only the encoding and decoding of S-expression format, to expose > its clients only atoms and relations. But how can it expose atoms and relations without an in-memory tree representation? Honestly, I do not think that such a concept is viable. Hence, the "S-Expression stream" must actually depend on the in-memory tree representation, both for reading and for writing. That is whay my package does. > A further unnamed yet package would handle the memory representation > of S-expressions, which involve object creation, memory management and > things like that. It would be a client of Sexp_Stream for I/O, so the > parsing would only be done at one place (namely Sexp_Stream). As I > said Ludovic Brenta's code might take this place. No, it replaces both of your hypothetical packages and I don't think you can have the "stream" package without the "S-Expression" package. You could, however, have an S-Expression package without any I/O. > There are situations when a memory representation of S-expressions is > not needed, and the tcp-connect example here seems to be such a case. > That's why I imagined TCP_Info as a client of Sexp_Stream instead of a > client of the second package. If no S-Expressions are needed in memory, then your entire data is just one big unstructured String, and you cannot parse it into a TCP_Info record unless you re-introduce the S-Expression tree in memory. >> I think the client should just do >> >> S : S_Expression := Some_Initialization; >> >> Put (File => F, Expression => S); > > To achieve such an interface, the client has to build an in-memory S- > expression object. In the tcp-connect example, there are eight nodes > (five atoms and three lists). They have to be somehow built, and it > doesn't look like a simple initialization. > > The second interface I proposed, with a lot of nested calls, builds > the S-expression object with functions looking like: > function NewList(Contents, Next: S_node_access) return S_node_access; > function AtomFromWhatever(Contents: whatever, Next: S_node_access) > return S_node_access; That's what I do in the test.adb example (using Cons () and To_Atom () respectively). > However, unless I'm building other higher-level functions, to build 8 > nodes I need 8 function calls, exactly like the second example. > > That 8-function-call example looks very C-ish to me, and honestly I > don't like it, I prefer the sequence of procedure calls from > Sexp_Stream example, but that's just a personal taste. > > The problem is that I don't see how to do it otherwise. Any idea would > be welcome. Here is one idea: embed the S-Expression "language" in Ada: function To_Sexp (S : in String) return S_Expression.T is String_Stream : constant Ada.Streams.Stream_IO.Stream_Access := Stream_Reading_From (S); -- implementation left as an exercise :) begin return Result : S_Expression.T do S_Expression.T'Read (String_Stream, Result); end return; end To_Sexp; TCP_Info : constant String := "(tcp-connect (host foo.bar) (port 80))"; TCP_Info_Structured := constant To_TCP_Info (To_Sexp (TCP_Info)); (this is based on my implementation). >> Your TCP_Info-handling pkg would convert the record into an S-expression, and >> call a single operation from your S-expression pkg to output the S-expression. > > That's the tricky part. At least so tricky that I can't imagine how to > do it properly. Yes, the S_Expression.Read operation is quite involved. But it can be done, and has been done :) -- Ludovic Brenta.
From: Natacha Kerensikova on 12 Aug 2010 08:16
On Aug 12, 12:55 pm, Ludovic Brenta <ludo...(a)ludovic-brenta.org> wrote: > Natacha Kerensikova wrote on comp.lang.ada: > [...] > > > Sexp_Stream is supposed to perform S-expression I/O over streams, > > without ever constructing a S-expression into memory. It is supposed > > to do only the encoding and decoding of S-expression format, to expose > > its clients only atoms and relations. > > But how can it expose atoms and relations without an in-memory tree > representation? Honestly, I do not think that such a concept is > viable. I have already showing how to write S-expressions to a stream without in-memory representations of the S-expression (thought I still need an in-memory representation of atoms, and I believe this can't be worked around), using an interface like this: procedure OpenList(sstream: in out Sexp_Stream); -- more or less equivalent to output a '(' into the underlying stream procedure CloseList(sstream: in out Sexp_Stream); -- more or less equivalent to output a ')' into the underlying stream procedure PutAtom(sstream: in out Sexp_Stream, atom: in octet_array); -- encode the atom and send it into the underlying stream I guess it will also need some functions to convert usual types to and from Octet_Array. The reading part is a bit more tricky, and I admitted when I proposed Sexp_Stream I didn't know how to make it. Having thought (maybe too much) since then, here is what the interface might look like: type Node_Type is (S_None, S_List, S_Atom); function Current_Node_Type(sstream: in Sexp_Stream) return Atom_Type; procedure Get_Atom(sstream: in Sexp_Stream, contents: out octet_array); -- raises an exception when Current_Node_Type is not S_Atom -- not sure "out octet_array" works, but that's the idea -- maybe turn it into a function for easier atom-to-object conversion procedure Move_Next(sstream: in out Sexp_Stream); -- when the client is finished with the current node, it calls this -- procedure to update stream internal state to reflect the next node in -- list procedure Move_Lower(sstream: in out Sexp_Stream); -- raises an exception when Current_Node_Type is not S_List -- update the internal state to reflect the first child of the current list procedure Move_Upper(sstream: in out Sexp_Stream); -- update the internal state to reflect the node following the list -- containing the current node. sortof "uncle" node -- the implementation is roughly skipping whatever nodes exist in the -- current list until reading its ')' and reading the following node Such an interface support data streams, i.e. reading new data without seek or pushback. The data would probably be read octet-by-octet (unless reading a verbatim encoded atom), hoping for an efficient buffering in the underlying stream. If that's a problem I guess some buffering can be transparently implemented in the private part of Sexp_Stream. Of course atoms are to be kept in memory, which means Sexp_Stream will have to contain a dynamic array (Vector in Ada terminology, right?) populated from the underlying stream until the atom is completely read. Get_Atom would hand over a (static) array built from the private dynamic array. The implementation would for example rely on a procedure to advance the underlying to the next node (i.e. skipping white space), and from the first character know whether it's the beginning of a new list (when '(' is encountered) and update the state to S_List, and it's finished; whether it's the end of the list (when ')' is encountered) and update the state to S_None; or whether it's the beginning of an atom, so read it completely, updating the internal dynamic array and setting the state to S_Atom. Well actually, it's probably not the best idea, I'm not yet clear enough on the specifications and on stream I/O to clearly think about implementation, but that should be enough to make you understand the idea of S-expression reading without S-expression objects. Now regarding the actual use of this interface, I think my previous writing example is enough, so here is the reading part. My usual way of reading S-expression configuration file is to read sequentially, one at a time, a list whose first element is an atom. Atoms, empty lists and lists beginning with a list are considered as comments and silently dropped. "(tcp-connect )" is meant to be one of these, processed by TCP_Info's client, which will hand over only the " " part to TCP_Info.Read (or other initialization subprogram). So just like TCP_Info's client processes something like "(what-ever- config ) (tcp-connect ) (other-config )", TCP_Info.Read will process only "(host foo.example) (port 80)". So TCP_Info's client, after having read "tcp-connect" atom, will call Move_Next on the Sexp_Stream and pass it to TCP_Info.Read. Then TCP_Info.Read proceeds with: loop case Current_Atom_Type(sstream) is when S_None => return; -- TCP_Info configuration is over when S_Atom => null; -- silent atom drop when S_List => Move_Lower(sstream); Get_Atom(sstream, atom); -- make command of type String from atom -- if Get_Atom was successful Move_Next(sstream); if command = "host" then -- Get_Atom and turn it into host string elif command = "port" then -- Get_Atom and turn it into port number else -- complain about unrecognized command end if; Move_Upper(sstream); end case; end loop; TCP_Info's client S-expression parsing would be very similar, except if command = would be followed by a call to TCP_Info.Read rather than a Get_Atom. So where are the problems with my Sexp_Stream without in memory object? What am I missing? Or is it so ugly and insane that it should be kept in C's realm? > > A further unnamed yet package would handle the memory representation > > of S-expressions, which involve object creation, memory management and > > things like that. It would be a client of Sexp_Stream for I/O, so the > > parsing would only be done at one place (namely Sexp_Stream). As I > > said Ludovic Brenta's code might take this place. > > No, it replaces both of your hypothetical packages and I don't think > you can have the "stream" package without the "S-Expression" package. Yes, I indeed realized this mistake short after having send the message. My bad. > You could, however, have an S-Expression package without any I/O. Indeed, though I have to admit I have trouble imagining what use it can have. > TCP_Info : constant String := "(tcp-connect (host foo.bar) (port > 80))"; > TCP_Info_Structured := constant To_TCP_Info (To_Sexp (TCP_Info)); That's an interesting idea, which conceptually boils down to serialize by hand the S-expression into a String, in order to unserialize it as an in-memory object, in order to serialize back into a Stream. Proofreading my post, the above might sound sarcastic though actually it is not. It's a twist I haven't thought of, but it might indeed turn out to be the simplest practical way of doing it. Actually for a very long time I used to write S-expressions to file using only string facilities and a special sx_print_atom() function to handle escaping of unsafe data. By then I would have handled TCP_Info.Write with the following C fragment (sorry I don't know yet how to turn it into Ada, but I'm sure equivalent as simple exists): fprintf(outfile, "(tcp-connect\n\t(host "); sx_print_atom(outfile, host); fprintf(outfile, ")\n\t(port %d)\n)\n", port); > >> Your TCP_Info-handling pkg would convert the record into an S-expression, and > >> call a single operation from your S-expression pkg to output the S-expression. > > > That's the tricky part. At least so tricky that I can't imagine how to > > do it properly. > > Yes, the S_Expression.Read operation is quite involved. But it can be > done, and has been done :) Actually what I called "tricky" was the "single operation" part of TCP_Info's job. As I said, 8 nodes to build and send, it's not that easy to fit in a single operation, though your tick performs it quite well. By comparison the S-expression reading from a stream feels less tricky to me, though a bit more tiring, and has indeed been done, even by me (if you can read C, that the second half (lines 417 to the end) of http://git.instinctive.eu/cgit/libnathandbag/tree/csexp.c ). Thanks for your comments and your implementation, Natacha |