Auto-update protocol
in [Embedded]
|
Prev: Microchip Introduces 16 New Enhanced Mid-Range 8-bit Core Microcontrollers
Next: New Rabbit 6000 Microcontroller has on-board WiFi
From: D Yuniskis on 8 Mar 2010 14:34 Hi, I'm looking for ideas for a lightweight (yet robust) autoupdate protocol for a family of network clients I'm designing. Some criteria: - Unattended, persistent operation. A device shouldn't require hand-holding from a user to get itself into a working configuration. It is acceptable (though not *desireable*) to withold its normal functionality from the user while updating *or* recovering from a botched upgrade. But, it must be able to eventually "right itself". - I don't want to have to *tell* the devices that they need to be updated. The user shouldn't even be aware that this is happening. I.e., the devices should be able to examine/query the "current images" to see if they coincide with *their* images. - I should be able to tailor an image to a *particular* device (i.e., an *instance* of a device; not just a "model number"). - Images need to be signed so they can't be forged. - The protocol can't be spoofed by "unfriendlies". - The protocol should be as light as possible -- but no lighter! ;-) - Minimize unnecessary network traffic as well as load on the server (the goal is for the user *not* to notice this activity -- though I am not trying to "keep it secret") (seems like I have forgotten something -- but I can't recall what! :< Too early in the day...) So, all I should have to do is put "current images" on the user's server and wait for each device to discover the image and update itself accordingly. For example, the devices could check the server at IPL (and periodically at run time -- though updates at run time can be more of a challenge as they will probably interfere with normal operation :< ) and "fingerprint" the current image located there to see if it looks like it differs from its own image. One way of doing this is to store the image for each device instance in a R/O file bearing the MAC of the device in question. But, this would require the device to perform a "trial download" of the image for the sole purpose of computing the fingerprint (why not just do a bytewise compare if you are going to this extreme?!). This will hammer the network pretty hard. Granted, the individual segments for each device will see modest traffic -- but the server's segment will quickly max out if multiple devices do this simultaneously (e.g., powering up together). This would necessitate a second layer of the protocol to randomize/defer such competition. :< Another approach is to *store* the fingerprint on the server in a uniquely accessible manner (e.g., use a file name like MAC.fingerprint). But, that represents a duplication of data (pet peeve of mine) which makes it at risk for getting out of sync. For example, updating the image and forgetting to update the fingerprint; or, a run-time race -- the device examines the fingerprint, sees that it differs, starts to download image but the image hasn't been updated yet. Or, the image is updated but the fingerprint is stale when the device examines it. As a result, the image is NOT seen as "new". [you could get around this if you implemented network file locking -- but that is more complexity and leaves open the possibility of stale locks, etc.] This could be worked around by forcing the server to recompute the fingerprint each time an image is added. But, that still leaves a race window *and* requires the server to be aware of the introduction of the new image file(s). The simplest compromise would seem to be having the device track the timestamp of the image file and check *that*. If changed, then *assume* the image actually has changed (of course, touch(1)-ing the image file would then force the device to consider the image file as changed -- this could be an advantage?). The device could then either fingerprint the image itself *or* naively assume it to be a new image and begin the (secure) update procedure. Security then is another facet to be addressed. E.g., given that the devices *don't* have enough resources to store an entire image before flashing, the protocol would have to be interruptible. E.g., encrypted packets so they can't be spoofed. But, accepting the possibility that the entire image might not become available "when needed" for the reflash. I.e., fall back to a secure boot loader that can do the update without the rest of the application being available. Note that this loader should *not* restart the upload but, instead, *continue* where it apparently left off, previously -- even in light of any intervening power cycles. This saves time, reduces flash wear and is just "smarter", in general. :> But, it seems like this *still* requires server side locking so the file isn't changed *while* it is being doled out. E.g., something like TFTP would be inappropriate as it doesn't reliably lock the file from one packet to the next. Shirley, this sort of thing has been done before (?). Pointers? Thanks! --don
From: Ignacio G. T. on 9 Mar 2010 09:11 El 08/03/2010 20:34, D Yuniskis escribi�: > Hi, > > I'm looking for ideas for a lightweight (yet robust) > autoupdate protocol for a family of network clients > I'm designing. [...] > Another approach is to *store* the fingerprint on the > server in a uniquely accessible manner (e.g., use > a file name like MAC.fingerprint). But, that represents > a duplication of data (pet peeve of mine) which makes > it at risk for getting out of sync. You could embed the fingerprint inside the image, at a reserved space in its beginning. -- Saludos. Ignacio G.T.
From: Boudewijn Dijkstra on 9 Mar 2010 09:52 Op Mon, 08 Mar 2010 20:34:53 +0100 schreef D Yuniskis <not.going.to.be(a)seen.com>: > I'm looking for ideas for a lightweight (yet robust) > autoupdate protocol for a family of network clients > I'm designing. > > Some criteria: > > - Unattended, persistent operation. A device shouldn't > require hand-holding from a user to get itself into a > working configuration. It is acceptable (though not > *desireable*) to withold its normal functionality > from the user while updating *or* recovering from a > botched upgrade. But, it must be able to eventually > "right itself". > > - I don't want to have to *tell* the devices that they > need to be updated. The user shouldn't even be aware > that this is happening. I.e., the devices should be > able to examine/query the "current images" to see if > they coincide with *their* images. > > - I should be able to tailor an image to a *particular* > device (i.e., an *instance* of a device; not just a > "model number"). > > - Images need to be signed so they can't be forged. > > - The protocol can't be spoofed by "unfriendlies". > > - The protocol should be as light as possible -- but > no lighter! ;-) > > - Minimize unnecessary network traffic as well as > load on the server (the goal is for the user *not* > to notice this activity -- though I am not trying > to "keep it secret") > > (seems like I have forgotten something -- but I > can't recall what! :< Too early in the day...) > > So, all I should have to do is put "current images" > on the user's server and wait for each device to > discover the image and update itself accordingly. > > For example, the devices could check the server at IPL > (and periodically at run time -- though updates at > run time can be more of a challenge as they will > probably interfere with normal operation :< ) > and "fingerprint" the current image located there > to see if it looks like it differs from its own > image. > > One way of doing this is to store the image for > each device instance in a R/O file bearing the MAC > of the device in question. But, this would > require the device to perform a "trial download" > of the image for the sole purpose of computing > the fingerprint (why not just do a bytewise compare > if you are going to this extreme?!). > > This will hammer the network pretty hard. Granted, > the individual segments for each device will see > modest traffic -- but the server's segment will > quickly max out if multiple devices do this > simultaneously (e.g., powering up together). > This would necessitate a second layer of the > protocol to randomize/defer such competition. :< > > Another approach is to *store* the fingerprint on the > server in a uniquely accessible manner (e.g., use > a file name like MAC.fingerprint). But, that represents > a duplication of data (pet peeve of mine) which makes > it at risk for getting out of sync. > > For example, updating the image and forgetting to update > the fingerprint; That's a bug. > or, a run-time race -- the device > examines the fingerprint, sees that it differs, starts > to download image but the image hasn't been updated > yet. How can the server know the fingerprint before it has been fully updated? > Or, the image is updated but the fingerprint > is stale when the device examines it. As a result, > the image is NOT seen as "new". No biggy. The device will check again sometime, right? > [you could get around this if you implemented network > file locking -- but that is more complexity and leaves > open the possibility of stale locks, etc.] Have you considered your own simple protocol? E.g. * client identifies to server and asks for update since $TIMESTAMP * server says yes or no * if yes, start streaming > This could be worked around by forcing the server to > recompute the fingerprint each time an image is added. > But, that still leaves a race window Only if the image is already accessible while the fingerprint isn't there yet. And why would you do that? > *and* requires > the server to be aware of the introduction of the > new image file(s). > > The simplest compromise would seem to be having the > device track the timestamp of the image file and > check *that*. If changed, then *assume* the image > actually has changed (of course, touch(1)-ing the > image file would then force the device to consider > the image file as changed -- this could be an > advantage?). The device could then either > fingerprint the image itself *or* naively assume > it to be a new image and begin the (secure) update > procedure. > > Security then is another facet to be addressed. E.g., > given that the devices *don't* have enough resources to > store an entire image before flashing, the protocol > would have to be interruptible. E.g., encrypted > packets so they can't be spoofed. But, accepting > the possibility that the entire image might not > become available "when needed" for the reflash. > I.e., fall back to a secure boot loader that > can do the update without the rest of the application > being available. > > Note that this loader should *not* restart the upload > but, instead, *continue* where it apparently left off, > previously -- even in light of any intervening power > cycles. This saves time, reduces flash wear and is > just "smarter", in general. :> > > But, it seems like this *still* requires server side > locking so the file isn't changed *while* it is being > doled out. Why would you do that? Each unique image should have a unique name. > E.g., something like TFTP would be > inappropriate as it doesn't reliably lock the file > from one packet to the next. > > Shirley, this sort of thing has been done before (?). > Pointers? > > Thanks! > --don -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/ (remove the obvious prefix to reply by mail)
From: D Yuniskis on 9 Mar 2010 13:09 Hi Ignacio, Ignacio G. T. wrote: > El 08/03/2010 20:34, D Yuniskis escribi�: >> Hi, >> >> I'm looking for ideas for a lightweight (yet robust) >> autoupdate protocol for a family of network clients >> I'm designing. > > [...] > >> Another approach is to *store* the fingerprint on the >> server in a uniquely accessible manner (e.g., use >> a file name like MAC.fingerprint). But, that represents >> a duplication of data (pet peeve of mine) which makes >> it at risk for getting out of sync. > > You could embed the fingerprint inside the image, at a reserved space in > its beginning. I *think* this is probably the best approach. It pushes all the "special processing" into the activity that "creates an image" instead of having to burden the system that *hosts* the image or the individual responsible for putting the image in place. I still have to consider how to ensure the file is locked during the update (so it can't be removed or replaced) but that will tend to be driven by the choice of transport protocol.
From: D Yuniskis on 9 Mar 2010 13:49
Hi Boudewijn, Boudewijn Dijkstra wrote: > Op Mon, 08 Mar 2010 20:34:53 +0100 schreef D Yuniskis > <not.going.to.be(a)seen.com>: [snips] >> - Unattended, persistent operation. A device shouldn't >> require hand-holding from a user to get itself into a >> working configuration. It is acceptable (though not >> *desireable*) to withold its normal functionality >> from the user while updating *or* recovering from a >> botched upgrade. But, it must be able to eventually >> "right itself". >> >> - I don't want to have to *tell* the devices that they >> need to be updated. The user shouldn't even be aware >> that this is happening. I.e., the devices should be >> able to examine/query the "current images" to see if >> they coincide with *their* images. >> >> - I should be able to tailor an image to a *particular* >> device (i.e., an *instance* of a device; not just a >> "model number"). >> >> - Images need to be signed so they can't be forged. >> >> - The protocol can't be spoofed by "unfriendlies". >> >> - The protocol should be as light as possible -- but >> no lighter! ;-) >> >> - Minimize unnecessary network traffic as well as >> load on the server (the goal is for the user *not* >> to notice this activity -- though I am not trying >> to "keep it secret") >> >> So, all I should have to do is put "current images" >> on the user's server and wait for each device to >> discover the image and update itself accordingly. >> >> Another approach is to *store* the fingerprint on the >> server in a uniquely accessible manner (e.g., use >> a file name like MAC.fingerprint). But, that represents >> a duplication of data (pet peeve of mine) which makes >> it at risk for getting out of sync. >> >> For example, updating the image and forgetting to update >> the fingerprint; > > That's a bug. That's "human nature" :> I.e., part of the problem is anticipating what sorts of things are going to "go wrong" and designing the protocol so they have minimal impact. >> or, a run-time race -- the device >> examines the fingerprint, sees that it differs, starts >> to download image but the image hasn't been updated >> yet. > > How can the server know the fingerprint before it has > been fully updated? I was assuming the fingerprint is computed elsewhere (i.e., part of building the image). So, you have xxxxx.image and xxxxx.fingerprint (like xxxxx.md5). (I'm trying not to make the server do much of anything so I can use some generic host for that role). So, if someone is responsible for copying these two files onto the server (via ftp, etc.) and happens to copy the fingerprint *before* the image, then it is possible that the "device" could examine that fingerprint (coincidentally) and make its download/update decision before the actual image has been copied (uploaded to the server) >> Or, the image is updated but the fingerprint >> is stale when the device examines it. As a result, >> the image is NOT seen as "new". > > No biggy. The device will check again sometime, right? Yes. But then the frequency at which the device re-checks the server becomes more significant. Ideally, I would "signal()" the devices when I *knew* the new image (and fingerprint) was available. But, that's another action that someone (or something) must perform. More resilient if the devices see to their own needs without needing external prodding. >> [you could get around this if you implemented network >> file locking -- but that is more complexity and leaves >> open the possibility of stale locks, etc.] > > Have you considered your own simple protocol? E.g. > * client identifies to server and asks for update since $TIMESTAMP > * server says yes or no > * if yes, start streaming Yes, but that means I have to have something "special" running on the server. I.e., a way of asking for an update "since $TIMESTAMP", a way for the server to respond, etc. I am trying to do those things without developing any special server-side tools (which would have to be ported to whichever server hosted these images, etc.) If, instead, I did something like: - image file name is MAC address of device (so each device knows which image it should be interested in) with ".image" extension - fingerprint is MAC address of device with ".fingerprint" (e.g., ".md5") - device fetches fingerprint via TFTP (since it is a tiny file, you can get the entire file in one packet!) - check against device's *computed* fingerprint - device decides (on its own!) that it needs the new image - device fetches image using (some existing) protocol This fits within the services offered by most machines. It requires the "image builder" (person + software) to create these two files (since the image is built by compiling sources, it seems appropriate to burden that person with the task of creating the fingerprint) and somehow get them onto the server in the right place. (this ignores security issues, spoofing, etc.) >> This could be worked around by forcing the server to >> recompute the fingerprint each time an image is added. >> But, that still leaves a race window > > Only if the image is already accessible while the fingerprint isn't > there yet. And why would you do that? If the server is "recomput[ing] the fingerprint" from the image, then the image is *there*. You would need to make sure it wasn't yet visible to the "client". E.g., create a "MAC" subdirectory called MAC.temp. Put image in there. Compute fingerprint and put it in there, too. Then, mv(1) the subdirectory to "MAC" (where it would be visible to the client). (this still leaves problems to resolve). <frown> I'm just trying to figure out how to get something for nothing! :> >> *and* requires >> the server to be aware of the introduction of the >> new image file(s). >> >> The simplest compromise would seem to be having the >> device track the timestamp of the image file and >> check *that*. If changed, then *assume* the image >> actually has changed (of course, touch(1)-ing the >> image file would then force the device to consider >> the image file as changed -- this could be an >> advantage?). The device could then either >> fingerprint the image itself *or* naively assume >> it to be a new image and begin the (secure) update >> procedure. >> >> Security then is another facet to be addressed. E.g., >> given that the devices *don't* have enough resources to >> store an entire image before flashing, the protocol >> would have to be interruptible. E.g., encrypted >> packets so they can't be spoofed. But, accepting >> the possibility that the entire image might not >> become available "when needed" for the reflash. >> I.e., fall back to a secure boot loader that >> can do the update without the rest of the application >> being available. >> >> Note that this loader should *not* restart the upload >> but, instead, *continue* where it apparently left off, >> previously -- even in light of any intervening power >> cycles. This saves time, reduces flash wear and is >> just "smarter", in general. :> >> >> But, it seems like this *still* requires server side >> locking so the file isn't changed *while* it is being >> doled out. > > Why would you do that? Each unique image should have a unique name. Well, it depends on how you then inform the client of the "right" file to access! I.e., you need something that maps client ID's to image filenames. And then you have to make sure that is updated and the whole race issue with keeping that synchronous with the image/fingerprint files. (unless you link(1) a client-specific name to a particular set of files, etc.) There are lots of ways you *can* do this. I'm just looking for one that is clean and free from as many hazzards as possible. :< So far, I think putting the fingerprint *in* the image is the best approach. Then, maybe using something like sftp to fetch it. So, client *always* starts to fetch the image. After it has the first portion of it (seems wisest to put the fingerprint at the head of the image file so you can get to it quickly), it extracts fingerprint and decides if it wants to continue the sftp session -- followed by a flashing -- or abort it. That way, if the file is deleted/renamed on the server, an instance of it should still be "link(1)-ed" to the sftp session (so it doesn't disappear in the middle of the xfer) >> E.g., something like TFTP would be >> inappropriate as it doesn't reliably lock the file >> from one packet to the next. >> >> Shirley, this sort of thing has been done before (?). >> Pointers? |