Prev: Generate Assembly File Via Borland 3.1
Next: linker script and compilers.....
From: Gary Lynch on 2 May 2010 13:58
I am looking for a tool to facilitate maintenance of a
firmware-based product, but I don't know whether it exists,
nor do I have a name for it. Ideally it would be vendor
agnostic as I never wish to be covenant-married to any
particular processor, language, or IDE.

I will illustrate a typical problem (greatly simplified)
from a real-life example.

Let's say my executable comes from 2 source files (in C),
holding 3 & 4 functions; respectively:

Rev 1.6:
+------ File1.c
| +---- File2.c
| |
- - --------
x . <decl>
x . func1a()
x . func1b()
x . func1c()
| |
. x <decl>
. x func2a()
. x func2b()
. x func2c()
. x func2d()

The <decl> lines refer to the declaration section at the top
of the file, which is outside all function definitions.

This is the way it looked when I released revision R1.6. My
manager wanted me to follow that up with inclusion of 2 new
features:

H New hardware: The new platform has more I/O ports,
more complex signal processing,

I Input Sampling: Due to a misunderstood requirement, I
had been sampling 2 input signals at the same
frequency. When one of them needed more noise
filtering while the other needed quicker response
time, I had to split them into separate processing
pipes, with new names.

In my revision control system, I give it a prototype
revision number: R1.6.HI.

The changes ripple through the design in a crazy-quilt kind
of pattern, Along the way, I have to add a new function to
File2.c and other functions therein grow until File2.c
becomes unmanageably large. I have to split it into 2 files
of 3 & 2 functions each, keeping the old function names:

R1.6.HI - R1.6:
New hardware -------+
Input Sampling ---+ |
| |
+------ File1.c | |
| +---- File20.c | |
| | +-- File21.c | |
| | | | |
- - - ---------- - -
x . . <decl> . .
x . . func1a() . .
x . . func1b() . .
x . . func1c() I .
| | | | |
. x . <decl> . .
. x . func2a() . .
. x . func2b() . H
. . x func2c() I H
| | | | |
. --> <decl> I H
. --> func2d() . H
. . + func21e() I .

Say after 3-4 weeks of effort I reach a point where this
builds without error and appears to run correctly, but I
have not completed regression testing and don't consider it
releaseable into the wild.

At this point the service manager returns from the field
saying one of the user inputs needs an abnormally long
debounce time to fit industry standards. My manager says
this request trumps what I am working on, so I must stop
what I am doing and get it out NOW.

I must make my changes to R1.6 (the lastest release).
This time, File1.c becomes a monster and I split it.

R1.6.D - R1.6:
Debounce ---------+
|
+------ File10.c |
| +---- File11.c |
| | +-- File2.c |
| | | |
- - - ---------- -
x . . <decl> .
x . . func1a() .
x . . func1b() D
| | | |
. --> <decl> D
. --> func1c() .
. . + func11d() D
| | | |
. . x <decl> .
. . x func2a() D
. . x func2b() .
. . x func2c() .
. . x func2d() D

I get that tested, release it as R1.7, and return to
my prior task, which now must be patched into R1.7.

At this point, what I really need is a utility that would
allow me to implement the top-level command:

R1.7.HI = R1.7 + (R1.6.HI - R1.6)

where each variable above is a rather complex, multi-level
structure of base type char, but having elements involving
files and functions and lines. Without it, I must either:

- Throw away the 3 man-weeks I spent creating Rev_1.6.HI
and repeat that task on a new base, or
- Attempt to execute the command above manually,
(Keeping track of all the levels can get tedious).

A couple of years ago, I thought a decent revision control
system ought to be able to handle this, since the users can
be forced to supply all the info necessary to drive it at
check-in time.

The most difficult sub-task is isolating changes to
individual functions, but most revision control systems come
with a file differencing utility that goes a long way toward
providing this feature. I have yet to find one, however,
that can handle functions migrating between files due to re-
factoring.

Although I made this example extremely simple, just
representing these relationships is difficult in the 2-D
system I have to work with.

Furthermore, writing this down makes me realize my problem
is in the tools, for whom the smallest unit of analysis is
the file; whereas in my source deck, the "atoms" are really
functions.

So I'll stop the example here and ask:
- Does such a tool exist?
- Is there a generic name for it?
- Where do I learn more about this topic?

Any ideas?
============================================================
Gary Lynch | To send mail, change no$pam
gary.lynch(a)no$pam.org | in my domain name to ieee
============================================================

From: Paul E. Bennett on 2 May 2010 14:46
Gary Lynch wrote:

> I am looking for a tool to facilitate maintenance of a
> firmware-based product, but I don't know whether it exists,
> nor do I have a name for it. Ideally it would be vendor
> agnostic as I never wish to be covenant-married to any
> particular processor, language, or IDE.

[%X]

> This is the way it looked when I released revision R1.6. My
> manager wanted me to follow that up with inclusion of 2 new
> features:
>
> H New hardware: The new platform has more I/O ports,
> more complex signal processing,
>
> I Input Sampling: Due to a misunderstood requirement, I
> had been sampling 2 input signals at the same
> frequency. When one of them needed more noise
> filtering while the other needed quicker response
> time, I had to split them into separate processing
> pipes, with new names.
>
> In my revision control system, I give it a prototype
> revision number: R1.6.HI.
>
> The changes ripple through the design in a crazy-quilt kind
> of pattern, Along the way, I have to add a new function to
> File2.c and other functions therein grow until File2.c
> becomes unmanageably large. I have to split it into 2 files
> of 3 & 2 functions each, keeping the old function names:

[%X]

> Say after 3-4 weeks of effort I reach a point where this
> builds without error and appears to run correctly, but I
> have not completed regression testing and don't consider it
> releaseable into the wild.
>
> At this point the service manager returns from the field
> saying one of the user inputs needs an abnormally long
> debounce time to fit industry standards. My manager says
> this request trumps what I am working on, so I must stop
> what I am doing and get it out NOW.
>
> I must make my changes to R1.6 (the lastest release).
> This time, File1.c becomes a monster and I split it.

[%X]

> I get that tested, release it as R1.7, and return to
> my prior task, which now must be patched into R1.7.
>
> At this point, what I really need is a utility that would
> allow me to implement the top-level command:
>
> R1.7.HI = R1.7 + (R1.6.HI - R1.6)
>
> where each variable above is a rather complex, multi-level
> structure of base type char, but having elements involving
> files and functions and lines. Without it, I must either:
>
> - Throw away the 3 man-weeks I spent creating Rev_1.6.HI
> and repeat that task on a new base, or
> - Attempt to execute the command above manually,
> (Keeping track of all the levels can get tedious).
>
> A couple of years ago, I thought a decent revision control
> system ought to be able to handle this, since the users can
> be forced to supply all the info necessary to drive it at
> check-in time.

It takes a bit more than that.

> The most difficult sub-task is isolating changes to
> individual functions, but most revision control systems come
> with a file differencing utility that goes a long way toward
> providing this feature. I have yet to find one, however,
> that can handle functions migrating between files due to re-
> factoring.
>
> Although I made this example extremely simple, just
> representing these relationships is difficult in the 2-D
> system I have to work with.
>
> Furthermore, writing this down makes me realize my problem
> is in the tools, for whom the smallest unit of analysis is
> the file; whereas in my source deck, the "atoms" are really
> functions.

This would be a good read to start with to see if any of the Configuration
Management tools might offer some sort of assistance.

<http://en.wikipedia.org/wiki/Configuration_management>

However, Configuration Management alone may not help you out with this sort
of problem. Certainly not as simply as you seem to imply you would like it.
Perhaps taking some time to think about the way you structure your systems
is called for. Would Object Oriented, Component Oriented or similar
techniques be helpful? Would a different architectural approach to your
systems be useful?


Based on just a glance through the problems you pose, then I would suggest
that the re-assessment of architecture of your systems, and employing
reasonable configuration management tools in your development process may be
the answers to your needs.

Dealing with the code in terms of atomic functions you should probably think
about ensuring that a collection of atomic functions in a file are closely
related such that they can be considered as aspects of the same idea.

Take some time to think about the five views of your system design:
* The Logical View
* The Process View
* The Development View
* The Physical View
* The Use Scenarios

These are explained in many System Development Texts (especially those
dealing with IEC61499).

--
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett(a)topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

From: Tim Wescott on 2 May 2010 14:53
Gary Lynch wrote:
> I am looking for a tool to facilitate maintenance of a
> firmware-based product, but I don't know whether it exists,
> nor do I have a name for it. Ideally it would be vendor
> agnostic as I never wish to be covenant-married to any
> particular processor, language, or IDE.
>
> I will illustrate a typical problem (greatly simplified)
> from a real-life example.
>
> Let's say my executable comes from 2 source files (in C),
> holding 3 & 4 functions; respectively:
>
> Rev 1.6:
> +------ File1.c
> | +---- File2.c
> | |
> - - --------
> x . <decl>
> x . func1a()
> x . func1b()
> x . func1c()
> | |
> . x <decl>
> . x func2a()
> . x func2b()
> . x func2c()
> . x func2d()
>
> The <decl> lines refer to the declaration section at the top
> of the file, which is outside all function definitions.
>
> This is the way it looked when I released revision R1.6. My
> manager wanted me to follow that up with inclusion of 2 new
> features:
>
> H New hardware: The new platform has more I/O ports,
> more complex signal processing,
>
> I Input Sampling: Due to a misunderstood requirement, I
> had been sampling 2 input signals at the same
> frequency. When one of them needed more noise
> filtering while the other needed quicker response
> time, I had to split them into separate processing
> pipes, with new names.
>
> In my revision control system, I give it a prototype
> revision number: R1.6.HI.
>
> The changes ripple through the design in a crazy-quilt kind
> of pattern, Along the way, I have to add a new function to
> File2.c and other functions therein grow until File2.c
> becomes unmanageably large. I have to split it into 2 files
> of 3 & 2 functions each, keeping the old function names:
>
> R1.6.HI - R1.6:
> New hardware -------+
> Input Sampling ---+ |
> | |
> +------ File1.c | |
> | +---- File20.c | |
> | | +-- File21.c | |
> | | | | |
> - - - ---------- - -
> x . . <decl> . .
> x . . func1a() . .
> x . . func1b() . .
> x . . func1c() I .
> | | | | |
> . x . <decl> . .
> . x . func2a() . .
> . x . func2b() . H
> . . x func2c() I H
> | | | | |
> . --> <decl> I H
> . --> func2d() . H
> . . + func21e() I .
>
> Say after 3-4 weeks of effort I reach a point where this
> builds without error and appears to run correctly, but I
> have not completed regression testing and don't consider it
> releaseable into the wild.
>
> At this point the service manager returns from the field
> saying one of the user inputs needs an abnormally long
> debounce time to fit industry standards. My manager says
> this request trumps what I am working on, so I must stop
> what I am doing and get it out NOW.
>
> I must make my changes to R1.6 (the lastest release).
> This time, File1.c becomes a monster and I split it.
>
> R1.6.D - R1.6:
> Debounce ---------+
> |
> +------ File10.c |
> | +---- File11.c |
> | | +-- File2.c |
> | | | |
> - - - ---------- -
> x . . <decl> .
> x . . func1a() .
> x . . func1b() D
> | | | |
> . --> <decl> D
> . --> func1c() .
> . . + func11d() D
> | | | |
> . . x <decl> .
> . . x func2a() D
> . . x func2b() .
> . . x func2c() .
> . . x func2d() D
>
> I get that tested, release it as R1.7, and return to
> my prior task, which now must be patched into R1.7.
>
> At this point, what I really need is a utility that would
> allow me to implement the top-level command:
>
> R1.7.HI = R1.7 + (R1.6.HI - R1.6)
>
> where each variable above is a rather complex, multi-level
> structure of base type char, but having elements involving
> files and functions and lines. Without it, I must either:
>
> - Throw away the 3 man-weeks I spent creating Rev_1.6.HI
> and repeat that task on a new base, or
> - Attempt to execute the command above manually,
> (Keeping track of all the levels can get tedious).
>
> A couple of years ago, I thought a decent revision control
> system ought to be able to handle this, since the users can
> be forced to supply all the info necessary to drive it at
> check-in time.
>
> The most difficult sub-task is isolating changes to
> individual functions, but most revision control systems come
> with a file differencing utility that goes a long way toward
> providing this feature. I have yet to find one, however,
> that can handle functions migrating between files due to re-
> factoring.
>
> Although I made this example extremely simple, just
> representing these relationships is difficult in the 2-D
> system I have to work with.
>
> Furthermore, writing this down makes me realize my problem
> is in the tools, for whom the smallest unit of analysis is
> the file; whereas in my source deck, the "atoms" are really
> functions.
>
> So I'll stop the example here and ask:
> - Does such a tool exist?
> - Is there a generic name for it?
> - Where do I learn more about this topic?
>
> Any ideas?

I have heard the Git fanatics claim that it is much better at this sort
of thing than SVN or any other traditional revision control system,
because "GIT works with patches".

It sounds like one of those wild-eyed "XYZ technology will save the
world -- and make your girlfriend smell nicer, too!", but it may be
worth looking into.

A more traditional solution would be to pay more attention to how you
are architecting your source files, to keep related stuff related and
unrelated stuff in separate files.

I suspect -- all GIT-fanatic claims taken with due consideration -- that
it's a tough nut to crack, no matter how you approach it.

--
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com
From: Dombo on 2 May 2010 15:47
Gary Lynch schreef:
> I am looking for a tool to facilitate maintenance of a
> firmware-based product, but I don't know whether it exists,
> nor do I have a name for it. Ideally it would be vendor
> agnostic as I never wish to be covenant-married to any
> particular processor, language, or IDE.
>
> I will illustrate a typical problem (greatly simplified)
> from a real-life example.
>
> Let's say my executable comes from 2 source files (in C),
> holding 3 & 4 functions; respectively:
>
> Rev 1.6:
> +------ File1.c
> | +---- File2.c
> | |
> - - --------
> x . <decl>
> x . func1a()
> x . func1b()
> x . func1c()
> | |
> . x <decl>
> . x func2a()
> . x func2b()
> . x func2c()
> . x func2d()
>
> The <decl> lines refer to the declaration section at the top
> of the file, which is outside all function definitions.
>
> This is the way it looked when I released revision R1.6. My
> manager wanted me to follow that up with inclusion of 2 new
> features:
>
> H New hardware: The new platform has more I/O ports,
> more complex signal processing,
>
> I Input Sampling: Due to a misunderstood requirement, I
> had been sampling 2 input signals at the same
> frequency. When one of them needed more noise
> filtering while the other needed quicker response
> time, I had to split them into separate processing
> pipes, with new names.
>
> In my revision control system, I give it a prototype
> revision number: R1.6.HI.
>
> The changes ripple through the design in a crazy-quilt kind
> of pattern, Along the way, I have to add a new function to
> File2.c and other functions therein grow until File2.c
> becomes unmanageably large. I have to split it into 2 files
> of 3 & 2 functions each, keeping the old function names:
>
> R1.6.HI - R1.6:
> New hardware -------+
> Input Sampling ---+ |
> | |
> +------ File1.c | |
> | +---- File20.c | |
> | | +-- File21.c | |
> | | | | |
> - - - ---------- - -
> x . . <decl> . .
> x . . func1a() . .
> x . . func1b() . .
> x . . func1c() I .
> | | | | |
> . x . <decl> . .
> . x . func2a() . .
> . x . func2b() . H
> . . x func2c() I H
> | | | | |
> . --> <decl> I H
> . --> func2d() . H
> . . + func21e() I .
>
> Say after 3-4 weeks of effort I reach a point where this
> builds without error and appears to run correctly, but I
> have not completed regression testing and don't consider it
> releaseable into the wild.
>
> At this point the service manager returns from the field
> saying one of the user inputs needs an abnormally long
> debounce time to fit industry standards. My manager says
> this request trumps what I am working on, so I must stop
> what I am doing and get it out NOW.
>
> I must make my changes to R1.6 (the lastest release).
> This time, File1.c becomes a monster and I split it.
>
> R1.6.D - R1.6:
> Debounce ---------+
> |
> +------ File10.c |
> | +---- File11.c |
> | | +-- File2.c |
> | | | |
> - - - ---------- -
> x . . <decl> .
> x . . func1a() .
> x . . func1b() D
> | | | |
> . --> <decl> D
> . --> func1c() .
> . . + func11d() D
> | | | |
> . . x <decl> .
> . . x func2a() D
> . . x func2b() .
> . . x func2c() .
> . . x func2d() D
>
> I get that tested, release it as R1.7, and return to
> my prior task, which now must be patched into R1.7.
>
> At this point, what I really need is a utility that would
> allow me to implement the top-level command:
>
> R1.7.HI = R1.7 + (R1.6.HI - R1.6)
>
> where each variable above is a rather complex, multi-level
> structure of base type char, but having elements involving
> files and functions and lines. Without it, I must either:
>
> - Throw away the 3 man-weeks I spent creating Rev_1.6.HI
> and repeat that task on a new base, or
> - Attempt to execute the command above manually,
> (Keeping track of all the levels can get tedious).
>
> A couple of years ago, I thought a decent revision control
> system ought to be able to handle this, since the users can
> be forced to supply all the info necessary to drive it at
> check-in time.
>
> The most difficult sub-task is isolating changes to
> individual functions, but most revision control systems come
> with a file differencing utility that goes a long way toward
> providing this feature. I have yet to find one, however,
> that can handle functions migrating between files due to re-
> factoring.
>
> Although I made this example extremely simple, just
> representing these relationships is difficult in the 2-D
> system I have to work with.
>
> Furthermore, writing this down makes me realize my problem
> is in the tools, for whom the smallest unit of analysis is
> the file; whereas in my source deck, the "atoms" are really
> functions.
>
> So I'll stop the example here and ask:
> - Does such a tool exist?

CM Synergy from Telelogic seems to fit the bill for the most part; since
changes are always done in the context of a task, it is relatively easy
to work on different tasks in parallel (by the same or different
persons) and include or exclude specific tasks for a specific release.
However merging two parallel tasks touching the same objects (=files)
with one task involving a complete restructuring of the code will be
anything but automatic; it will require quite a bit of effort and
attention of the one doing the merge.

Though CM Synergy is quite powerful, especially when used in combination
with ChangeSynergy, I'm reluctant to recommend it because its a rather
heavy/complex tool with a horrible user interface and it can get rather
slow when the project gets large. Though CM Synergy can handle complex
scenarios it will become a full time job to manage this tool real
quickly; on projects where I have seen this tool had at least on person
dedicated to configuration management. For small projects (<10 man) I
would stay away from it.

> - Is there a generic name for it?

Configuration Management.

> - Where do I learn more about this topic?

I guess googling for Configuration Management turns up more than you
could ever read. (sorry I don't any specific links at hand)
From: Bob on 2 May 2010 16:05
On 2 May, 18:58, Gary Lynch <bookw...(a)execpc.com> wrote:
> So I'll stop the example here and ask:
>   - Does such a tool exist?
>   - Is there a generic name for it?
>   - Where do I learn more about this topic?

Ah yes, the Holy Grail of software development...some call it the
Magic Bullet.

Many years ago I worked on a mainframe OS, which had a build control
system working at the function level. I have not seen anything like it
since. All VCS systems I know work at the file level, and since this
is how C works there is little incentive to invent something better.

There are plenty of VCS systems that allow all manner of branches and
sub-branches, but I have never found anything to help much with the
task of integrating them back together, apart from a good merge tool
and man hours. It is best to avoid the situation by efficient
planning of the development - this may equate to what I call "managing
the manager".
 |  Next  |  Last
Pages: 1 2
Prev: Generate Assembly File Via Borland 3.1
Next: linker script and compilers.....