Prev: 50 ohm PCB antenna track impedance
Next: BT earpieces
From: Thad Smith on 27 Mar 2010 13:12
Dave Boland wrote:
> I'm doing some work on a 16-bit micro and one of the things the software
> does is store data in a structure for easy access. Currently, I save
> the entire structure to file and get it back from file when the program
> is restarted. No problem so far.
>
> However, if I need to change the structure (add a member, remove one,
> change one, etc.), the data, when read in from file, gets corrupted as
> one may expect. In addition, this code may be ported to a 32-bit
> controller, which means the alignment of the data in the structure will
> be different. So even if the structure doesn't change, when reading
> data created by a 16-bit processor I would expect the 32-bit processor
> to corrupt the data.
>
> So here is my question -- what are the best practices for saving data
> from a C structure so it is universally readable -- even if the
> structure definition changes? I'm leaning toward a character array
> (buffer) that uses name-value pairs, but that is a lot more work, and
> may be over-kill. Any thoughts?

There are many feasible ways to do this. The important aspect is to adequately
document your choice.

Consider ASN.1 BER encoding for a well-defined, fairly efficient mechanism. The
ASN.1 packed encoding, PER, is more storage-efficient but much more complex in
the worst case. ASN.1 provides a formal way to document the data structure.

--
Thad
From: Peter Dickerson on 28 Mar 2010 02:04
"Thad Smith" <ThadSmith(a)acm.org> wrote in message
news:4bae2e93$0$77545$892e0abb(a)auth.newsreader.octanews.com...
> Dave Boland wrote:
>> I'm doing some work on a 16-bit micro and one of the things the software
>> does is store data in a structure for easy access. Currently, I save the
>> entire structure to file and get it back from file when the program is
>> restarted. No problem so far.
>>
>> However, if I need to change the structure (add a member, remove one,
>> change one, etc.), the data, when read in from file, gets corrupted as
>> one may expect. In addition, this code may be ported to a 32-bit
>> controller, which means the alignment of the data in the structure will
>> be different. So even if the structure doesn't change, when reading data
>> created by a 16-bit processor I would expect the 32-bit processor to
>> corrupt the data.
>>
>> So here is my question -- what are the best practices for saving data
>> from a C structure so it is universally readable -- even if the structure
>> definition changes? I'm leaning toward a character array (buffer) that
>> uses name-value pairs, but that is a lot more work, and may be over-kill.
>> Any thoughts?
>
> There are many feasible ways to do this. The important aspect is to
> adequately document your choice.
>
> Consider ASN.1 BER encoding for a well-defined, fairly efficient
> mechanism. The ASN.1 packed encoding, PER, is more storage-efficient but
> much more complex in the worst case. ASN.1 provides a formal way to
> document the data structure.

I don't recall ASN.1 being anything but unfriendly. BER is easier than PER,
but as I recall some of the tag values in the encoding will change with
changes of the structure.

OTOH I too have been bitten by software that writes out images of structs,
so something needs to be done. In a recent case the from 'header' part of
the struct was read back in by the UI when constructing menus, so needed to
be timely. Even so, its hard to justify when there are twenty-odd versions
of that struct used in the one app (actually derived classes with no virtual
methods).

I think that I would use a text file with one line per datum or the
name=value style. Then in the code have a table of name, offsetof(field),
type-indicator. Fill the struct will 0; scan the table filling fields with
default value of type (if not zero); read the file line by line, match name,
parse according to type-indicator and save at offsetof in struct. If you
delete a field from the struct then the table won't compile. If you add a
field without updating the table it will at least be zeroed.

*.comp.lang.c people aren't interested in this sort of stuff so removed.

Peter


From: Thad Smith on 28 Mar 2010 14:09
Peter Dickerson wrote:
> "Thad Smith" <ThadSmith(a)acm.org> wrote in message
> news:4bae2e93$0$77545$892e0abb(a)auth.newsreader.octanews.com...
>> Dave Boland wrote:
>>> I'm doing some work on a 16-bit micro and one of the things the software
>>> does is store data in a structure for easy access. Currently, I save the
>>> entire structure to file and get it back from file when the program is
>>> restarted. No problem so far.
....
>> There are many feasible ways to do this. The important aspect is to
>> adequately document your choice.
>>
>> Consider ASN.1 BER encoding for a well-defined, fairly efficient
>> mechanism. The ASN.1 packed encoding, PER, is more storage-efficient but
>> much more complex in the worst case. ASN.1 provides a formal way to
>> document the data structure.
>
> I don't recall ASN.1 being anything but unfriendly. BER is easier than PER,
> but as I recall some of the tag values in the encoding will change with
> changes of the structure.

There are different ways to use ASN.1. A simple way is to use the BER low-level
encoding and make up your own tags -- done. Another way is to create a formal
ASN.1 description of the data. The language has provisions for extensibility --
in fact, I consider that one of its strengths. There are compatible ways within
the language to extend specifications, so that old data is represented the same,
yet new data can be encoded.

> I think that I would use a text file with one line per datum or the
> name=value style. Then in the code have a table of name, offsetof(field),
> type-indicator. Fill the struct will 0; scan the table filling fields with
> default value of type (if not zero); read the file line by line, match name,
> parse according to type-indicator and save at offsetof in struct. If you
> delete a field from the struct then the table won't compile. If you add a
> field without updating the table it will at least be zeroed.

That is fine, too.

--
Thad
From: ChrisQ on 16 Apr 2010 18:06
Dave Boland wrote:
> I'm doing some work on a 16-bit micro and one of the things the software
> does is store data in a structure for easy access. Currently, I save
> the entire structure to file and get it back from file when the program
> is restarted. No problem so far.
>
> However, if I need to change the structure (add a member, remove one,
> change one, etc.), the data, when read in from file, gets corrupted as
> one may expect. In addition, this code may be ported to a 32-bit
> controller, which means the alignment of the data in the structure will
> be different. So even if the structure doesn't change, when reading
> data created by a 16-bit processor I would expect the 32-bit processor
> to corrupt the data.
>
> So here is my question -- what are the best practices for saving data
> from a C structure so it is universally readable -- even if the
> structure definition changes? I'm leaning toward a character array
> (buffer) that uses name-value pairs, but that is a lot more work, and
> may be over-kill. Any thoughts?
>
> Dave,

Dave,

May be missing something here, but how about this ?:

typedef struct {
...
...
...
} DATA;

typedef union {
struct DATA sData;
U8 u8Data [sizeof (struct DATA)];
} uData;

You can add elements at will and the compiler takes care of all the
alignment and padding issues. You access structure elements via the
union.sData and access via union.byta array array to save to a file. It
should also be portable across any wordsize architecture.

In short, make the complier do the work for you :-)...

Regards,

Chris



From: Hans-Bernhard Bröker on 16 Apr 2010 19:42
ChrisQ wrote:
> May be missing something here, but how about this ?:
>
> typedef struct {
> ...
> ...
> ...
> } DATA;
>
> typedef union {
> struct DATA sData;
> U8 u8Data [sizeof (struct DATA)];
> } uData;

This achieves nothing useful at all. There's nothing to be gained by
having that union, compared to a plain and simple equivalent of a
memcpy() or fwrite()/fread() function, operating directly on a pointer
to the struct (cast to unsigned char*).

In a nutshell this is the opposite of best practice. It's about the
worst possible way to express worst practice.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5 6
Prev: 50 ohm PCB antenna track impedance
Next: BT earpieces