USB drives, caching and sync
in [Embedded]
|
Prev: Software Development opportunity - High Availability / Embedded/ Middleware / Linux
Next: Looking for programming languages that compile to C
From: Noob on 14 Jan 2010 08:52 Boudewijn Dijkstra wrote: > In fact, all OS calls should be reentrant/thread-safe unless specified > otherwise. I like this solution :-) So, if two threads are concurrently streaming data to the disk, and one thread calls sync, all the data (and metadata) written so far by both threads should be flushed to disk, right? If the second thread is "in the middle" of a write, the OS would queue the flush, so that everything happens as expected, right? Regards.
From: Boudewijn Dijkstra on 14 Jan 2010 09:46 Op Thu, 14 Jan 2010 14:52:29 +0100 schreef Noob <root(a)127.0.0.1>: > Boudewijn Dijkstra wrote: > >> In fact, all OS calls should be reentrant/thread-safe unless specified >> otherwise. > > I like this solution :-) It is, I believe, common practice. > So, if two threads are concurrently streaming data to the disk, and one > thread calls sync, all the data (and metadata) written so far by both > threads should be flushed to disk, right? Yes. > If the second thread is "in the middle" of a write, the OS would queue > the flush, so that everything happens as expected, right? Either the OS has received the data to be written, or it hasn't. When it hasn't, a sync cannot affect it. -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/ (remove the obvious prefix to reply by mail)
From: Paul Keinanen on 14 Jan 2010 10:57 On Wed, 13 Jan 2010 12:18:48 +0100, "Boudewijn Dijkstra" <sp4mtr4p.boudewijn(a)indes.com> wrote: >Op Mon, 11 Jan 2010 14:36:43 +0100 schreef Noob <root(a)127.0.0.1>: >> Boudewijn Dijkstra wrote: >>> Noob wrote: >>> >> [...] >> >>>> When I'm done writing my file, I do call fflush and close on the file >>>> descriptor (the fflush should be redundant) but that does not seem >>>> sufficient to convince the OS to flush the file's data and metadata to >>>> the disk. (Perphaps the OS keeps a copy of the FAT in memory, and only >>>> commits that on sync?) >>> >>> Your documentation should state which calls affect data only and which >>> affect also metadata. >> >> I would assume that most writes require an update of the FAT, which one >> might consider metadata? > >Yes sorry, most writes require a metadata update at some point, but there >are only a few ways of forcing the system to do it immediately. > >>>> What happens if two threads call sync "at the same time"? >>>> Is sync supposed to be reentrant? or thread-safe? >>>> (I suppose it will depend on the OS?) >>> >>> POSIX calls should be reentrant/thread-safe unless specified otherwise. >> >> I don't think OS21/OS+ claims POSIX conformance. I will have to check. > >In fact, all OS calls should be reentrant/thread-safe unless specified >otherwise. On a single processor system, how could two threads call sync simultaneously ? Unless the system call is explicitly made preemptable, assuming user/kernel mode switching during the call, this could not happen. At least in most operating system, calls to OS services is implemented with some kind of change mode to kernel traps, which are served in a more or less similar way as interrupts in kernel mode. Some hardware even have instructions with names like "software interrupt" to activate the service routine in kernel mode. No other user mode thread will get control, unless this "interrupt" returns one way or an other, so there should not be such problems. Of course the "OS call" seen by the user may do some jobs in the user mode, before switching to kernel mode, so this code may be vulnerable to task switching, if not implemented properly.
From: Nobody on 15 Jan 2010 03:44 On Thu, 14 Jan 2010 17:57:22 +0200, Paul Keinanen wrote: > On a single processor system, how could two threads call sync > simultaneously ? > > Unless the system call is explicitly made preemptable, assuming > user/kernel mode switching during the call, this could not happen. It would be a pretty lousy OS if sync() wasn't pre-emptible, given that it has to be one of the slowest system calls there is. Think about it: sync() spends most of its time blocked waiting for the disk to indicate either that it's ready for more data or has completed outstanding writes.
From: Noob on 15 Jan 2010 04:42
Boudewijn Dijkstra wrote: > Noob wrote: > >> So, if two threads are concurrently streaming data to the disk, and >> one thread calls sync, all the data (and metadata) written so far by >> both threads should be flushed to disk, right? > > Yes. > >> If the second thread is "in the middle" of a write, the OS would queue >> the flush, so that everything happens as expected, right? > > Either the OS has received the data to be written, or it hasn't. When it > hasn't, a sync cannot affect it. Consider a scenario where thread A needs to write 100 MB to the hard disk drive i.e. thread A calls write(fd, buf, 100*1000*1000); When the OS has "processed" approximately half the data, another thread calls sync. Would the sync operation be queued and carried out only after the write completes? Or would the OS perform the sync operation "immediately", flushing data and metadata cached up to this point? In that case, when the write completes, the first half of the data would be guaranteed to be written to disk, while parts of the second half may still be cached. (I think the second scenario is more plausible.) In other words, I think I'm asking: is there some form of atomicity of file system calls? Regards. |