Saving C structures to file -- best practices
in [Embedded]
|
Prev: 50 ohm PCB antenna track impedance
Next: BT earpieces
From: Jon Kirwan on 16 Apr 2010 20:34 On Wed, 24 Mar 2010 11:16:13 -0400, Dave Boland <dboland9(a)fastmail.fm> wrote: >I'm doing some work on a 16-bit micro and one of the things the software >does is store data in a structure for easy access. Currently, I save >the entire structure to file and get it back from file when the program >is restarted. No problem so far. > >However, if I need to change the structure (add a member, remove one, >change one, etc.), the data, when read in from file, gets corrupted as >one may expect. In addition, this code may be ported to a 32-bit >controller, which means the alignment of the data in the structure will >be different. So even if the structure doesn't change, when reading >data created by a 16-bit processor I would expect the 32-bit processor >to corrupt the data. > >So here is my question -- what are the best practices for saving data >from a C structure so it is universally readable -- even if the >structure definition changes? I'm leaning toward a character array >(buffer) that uses name-value pairs, but that is a lot more work, and >may be over-kill. Any thoughts? I wasn't sure what to say to this, at first. There are so many perspectives and you didn't narrow this down then. However, you have clarified some of it with the mention of 2k memory available, for example. I've used several different methods for similar situations (some where I had to do something different than what I'll suggest here were limited to 256 bytes, so I used a different method for robust/small/flexible there.) I liked one of the suggestions I've seen here for using a text format with <name>=<value>. I've used that method successfully for systems where versions undergo (and are expected to undergo) many revisions. For example, I have one system that is now in its 57th incarnation in 3 years' time. And I used this method without any serious problems, at all, despite many new features being added and modified over that time. That system is where a user (or computer) may interact and change parameters that need to be saved by command to do so, but where the system needs to start up from such a saved state, too. One of the things I do is to require the program to read _all_ of the lines of that file. The parsing structure is consistent, so the <name> field can be easily adduced. If the name doesn't match something in the current incarnation of the program, it is ignored. However, anytime the current state is saved, the entire file is re-read, all known fields changed, and all unknown ones preserved in the new save file. Old systems that know nothing about new parameters will therefore preserve those values they don't know how to use. New systems that don't require older parameters also still preserve those older parameters. Only those parameters that mean the same thing between different incarnations are modified by "saving" the state. Finally, all parameters for any system (new or old) has a default state that takes place if the parameter isn't found, at all, when a save state is loaded. And saving a state writes out all internal state values even if they weren't in the earlier save state, so the text save state may grow when saving. It has saved me trouble. Old systems run as expected on new save states. New systems run as expected on old save states. Etc. Jon
From: ChrisQ on 17 Apr 2010 18:51 Hans-Bernhard Br�ker wrote: > ChrisQ wrote: >> May be missing something here, but how about this ?: >> >> typedef struct { >> ... >> ... >> ... >> } DATA; >> >> typedef union { >> struct DATA sData; >> U8 u8Data [sizeof (struct DATA)]; >> } uData; > > This achieves nothing useful at all. There's nothing to be gained by > having that union, compared to a plain and simple equivalent of a > memcpy() or fwrite()/fread() function, operating directly on a pointer > to the struct (cast to unsigned char*). > > In a nutshell this is the opposite of best practice. It's about the > worst possible way to express worst practice. That's very good. Two paragraphs having nothing constructive to say at all :-). We can agree to differ, but aliasing a structure to an array is a good technique to ensure that the correct number of bytes get written to the file, irrespective of the cpu architecture, compiler and future changes to the structure. If you don't understand the importance of data structures in programming, then I guess there's no point in continuing... Regards, Chris
From: ChrisQ on 17 Apr 2010 18:55 ChrisQ wrote: > Hans-Bernhard Br�ker wrote: >> ChrisQ wrote: >>> May be missing something here, but how about this ?: >>> >>> typedef struct { >>> ... >>> ... >>> ... >>> } DATA; >>> >>> typedef union { >>> struct DATA sData; >>> U8 u8Data [sizeof (struct DATA)]; >>> } uData; >> >> This achieves nothing useful at all. There's nothing to be gained by >> having that union, compared to a plain and simple equivalent of a >> memcpy() or fwrite()/fread() function, operating directly on a pointer >> to the struct (cast to unsigned char*). >> >> In a nutshell this is the opposite of best practice. It's about the >> worst possible way to express worst practice. > > That's very good. Two paragraphs having nothing constructive to say at > all :-). > > We can agree to differ, but aliasing a structure to an array is a good > technique to ensure that the correct number of bytes get written to the > file, irrespective of the cpu architecture, compiler and future changes > to the structure. > > If you don't understand the importance of data structures in > programming, then I guess there's no point in continuing... > > Regards, > > Chris > And yes, where do you get the count for your memcopy ?. The usual hack is to sizeof (struct), then use that count in memcopy with a pointer to the start of the struct. If this is the best you can do, it's no surprise that software has so many bugs :-)... Regards, Chris
From: John Devereux on 18 Apr 2010 06:32 ChrisQ <meru(a)devnull.com> writes: > Hans-Bernhard Bröker wrote: >> ChrisQ wrote: >>> May be missing something here, but how about this ?: >>> >>> typedef struct { >>> ... >>> ... >>> ... >>> } DATA; >>> >>> typedef union { >>> struct DATA sData; >>> U8 u8Data [sizeof (struct DATA)]; >>> } uData; >> >> This achieves nothing useful at all. There's nothing to be gained >> by having that union, compared to a plain and simple equivalent of a >> memcpy() or fwrite()/fread() function, operating directly on a >> pointer to the struct (cast to unsigned char*). >> >> In a nutshell this is the opposite of best practice. It's about the >> worst possible way to express worst practice. > > That's very good. Two paragraphs having nothing constructive to say at > all :-). But true. > We can agree to differ, but aliasing a structure to an array is a good > technique to ensure that the correct number of bytes get written to > the file, irrespective of the cpu architecture, compiler and future > changes to the structure. So what happens when the compiler changes the struct layout so the length changes? Exactly the same thing as if you had simply used the plain struct in the first place! > If you don't understand the importance of data structures in >programming, then I guess there's no point in continuing... It's not that simply writing a bit copy of the struct to a file is a great idea, But your wrapper does not appear to improve on it at all. It just obfuscates what is going on and introduces a gratuitous extra layer when accessing it. -- John Devereux
From: Hans-Bernhard Bröker on 18 Apr 2010 08:17
ChrisQ wrote: > Hans-Bernhard Br�ker wrote: >> In a nutshell this is the opposite of best practice. It's about the >> worst possible way to express worst practice. > That's very good. Two paragraphs having nothing constructive to say at > all :-). To which you reply with _three_ paragraphs of nothing constructive at all. Pott. Kettle. Black? And what else, exactly did you expect to get for barging in three weeks after this discussion ran out, offering as a "solution" essentially the very same approach the OP described as the _problem_ he wanted a solution for: > Currently, I save the entire structure to file and get it back from > file when the program is restarted. ? > We can agree to differ, but aliasing a structure to an array is a good > technique to ensure that the correct number of bytes get written to the > file, No, it's not. It's not a single bit better than a perfectly straightforward fwrite(&struct, sizeof(struct), 1, fp); > irrespective of the cpu architecture, compiler and future changes > to the structure. Actually, neither your union nor the direct binary fwrite achieves any of that. A file written like this by one platform will typically _not_ be readable by the same code running on another platform. Which makes it the worst practice, and the opposite of what the OP asked for. > If you don't understand the importance of data structures in > programming, Come to think of it: since you obviously didn't understand the OP's question, nor any of the replies he got, why did you answer? |