USB drives, caching and sync
in [Embedded]
|
Prev: Software Development opportunity - High Availability / Embedded/ Middleware / Linux
Next: Looking for programming languages that compile to C
From: David Brown on 19 Jan 2010 15:18 Noob wrote: > Boudewijn Dijkstra wrote: > >> Noob wrote: >> >>> Paul Keinanen wrote: >>> >>>> Of course, writing to the file and fsyncing should be done from the >>>> same thread. >>> >>> OK, but many threads may be writing to the disk "at the same time" >> >> but not to the same file I hope? > > Each thread writes to its own file, in its own directory. > >>> i.e. concurrently, whereas vfs_sync is supposed to flush everything >>> to disk (even data and metadata from other threads). >> >> Perhaps you're reading over the initial 'f' of fsync. > > I don't quite understand. > > I think you're saying fsync works on a file, whereas sync works on an > entire volume, but I'm not getting your point. > Yes, this is what he is saying - fsync() asks the OS to make sure that the specific file has been completely written to the disk, while sync() asks for /all/ outstanding data to be flushed (sync() also does a little more housekeeping). On a number of systems, fsync() is actually implemented by flushing all outstanding writes on that file system. With various sorts of optimisations, write combines, re-orders, hard disk caches, native command queueing, etc., it can be extremely difficult for the OS to know exactly what should be written to ensure that the metadata and file data of a file is fully written. The only sure and safe way is then to flush /everything/. Thus it is perfectly reasonable for your system's fsync() to be a full flush. There was a lot of online discussion about this sort of thing recently, especially in the context of KDE writing many small files on ext4, which then got lost during crashes (while testing experimental and alpha systems). The debate raged about whether it was better to have safe but slow fsyncs(), or continue with the fast but theoretically unreliable ext3 behaviour, and whether it was enough for calls like these to do what posix said they must do or whether they had to do what developers thought they did. > On my system, fsync is called vfs_fflush, and I've tried calling it > right before closing a file, but the OS still runs fsck when I replug > the drive. Calling sync seems to make the OS happy. > The difference is that calling sync will clear the drives "dirty" flag, which is set whenever something is about to be written to the disk. An fsync, even if it flushes everything in the write buffer, will not clear this flag. The OS uses it on startup to see that the system has had an unclean shutdown, and therefore needs extra checking on the file system. |