copy a file while it is actively being written
in [General Linux]
|
Prev: nice bandwidth usage
Next: unexplained bianry "diff"
From: Jerry Peters on 3 Feb 2010 16:20 unruh <unruh(a)wormhole.physics.ubc.ca> wrote: > On 2010-02-02, Douglas Mayne <invalid(a)invalid.com> wrote: >> On Mon, 01 Feb 2010 15:08:04 -0800, jinzishuai(a)gmail.com wrote: >> >>> Hi there, >>> >>> I am wondering if serious problem could happen if I copy a file that is >>> actively being written to. >>> I guess some file corruption might happen, right? >>> >> My _guess_ is that any "corruption" would show up as a failed backup >> attempt. The original file is hopefully locked by its owner for exclusive >> write access; thus, any external read attempt will not interfere with it. >> However, the backup attempt will be "blurry" at best. > > File locking is something that depends totally on the program, and is a > voluntary thing-- ie the reading program must check to see if the file > is locked. ARAIK, there is no mechanism in Linux to lock a file so that > nothing else can read it. Actually there is, see man fcntl: Mandatory locks are enforced for all processes. If a process tries to perform an incompatible access (e.g., read(2) or write(2)) on a file region that has an incompatible mandatory lock, then the result depends upon whether the O_NONBLOCK flag is enabled for its open file descrip- tion. If the O_NONBLOCK flag is not enabled, then system call is blocked until the lock is removed or converted to a mode that is com- patible with the access. If the O_NONBLOCK flag is enabled, then the system call fails with the error EAGAIN or EWOULDBLOCK. To make use of mandatory locks, mandatory locking must be enabled both on the file system that contains the file to be locked, and on the file itself. Mandatory locking is enabled on a file system using the "-o mand" option to mount(8), or the MS_MANDLOCK flag for mount(2). Manda- tory locking is enabled on a file by disabling group execute permission on the file and enabling the set-group-ID permission bit (see chmod(1) and chmod(2)). The Linux implementation of mandatory locking is unreliable. See BUGS below. Of course this still requires that the writing program use fcntl lock. Jerry
From: Douglas Mayne on 5 Feb 2010 12:21 On Tue, 02 Feb 2010 22:30:29 +0000, unruh wrote: > On 2010-02-02, Douglas Mayne <invalid(a)invalid.com> wrote: >> On Mon, 01 Feb 2010 15:08:04 -0800, jinzishuai(a)gmail.com wrote: >> >>> Hi there, >>> >>> I am wondering if serious problem could happen if I copy a file that is >>> actively being written to. >>> I guess some file corruption might happen, right? >>> >> My _guess_ is that any "corruption" would show up as a failed backup >> attempt. The original file is hopefully locked by its owner for exclusive >> write access; thus, any external read attempt will not interfere with it. >> However, the backup attempt will be "blurry" at best. > > File locking is something that depends totally on the program, and is a > voluntary thing-- ie the reading program must check to see if the file > is locked. ARAIK, there is no mechanism in Linux to lock a file so that > nothing else can read it. > Thanks for the correction. I see now that the *nix permission model does not include flags for locks. _Services_ which run on top of *nix which want file locking (such as Samba), are what give the impression that locks are implemented natively, when they are not. -- Douglas Mayne
From: Chris Davies on 5 Feb 2010 15:32 Douglas Mayne <doug(a)localhost.localnet> wrote: > Thanks for the correction. I see now that the *nix permission model does > not include flags for locks. You're looking in the wrong place. *nix-based systems /do/ provide locking, but it's rarely anything to do with permissions. See flock(1), flock(2), and lockf(3) for the relevant documentation. Chris
From: unruh on 5 Feb 2010 21:38 On 2010-02-05, Chris Davies <chris-usenet(a)roaima.co.uk> wrote: > Douglas Mayne <doug(a)localhost.localnet> wrote: >> Thanks for the correction. I see now that the *nix permission model does >> not include flags for locks. > > You're looking in the wrong place. *nix-based systems /do/ provide > locking, but it's rarely anything to do with permissions. See flock(1), > flock(2), and lockf(3) for the relevant documentation. > > Chris man 2 flock flock - apply or remove an advisory lock on an open file .... flock() places advisory locks only; given suitable permissions on a file, a process is free to ignore the use of flock() and perform I/O on the file.
From: despen on 5 Feb 2010 22:44
"jinzishuai(a)gmail.com" <jinzishuai(a)gmail.com> writes: > Hi there, > > I am wondering if serious problem could happen if I copy a file that > is actively being written to. > I guess some file corruption might happen, right? Wrong. Most files are written without locking. If you are familiar with using the command: tail -f you'd know that reading a file as it's being written causes no problems. > More specifically, I am dealing with the snapshot of a qcow2 file. > # qemu-img snapshot -c snap base.qcow2 > # cp base.qcow2 snap.qcow2 > # qemu-img snapshot -a snap snap.qcow2 > Do you think I will get an image snap.qcow2 that is completely > consistent with the base.qcow2 image at the time when the snapshot is > taken? Some buffering will take place. > For my case, it is OK if new information written to the disk after > snapshot taken is lost or corrupted since all I want to copy is the > backed up content. But would this copy during write process actually > mess up the whole file and make it even un-readable? Sounds like you might want to be using the syslog facilities? |