Variable I/O Performance with dd vs. cat
in [General Linux]
|
From: Douglas Mayne on 17 Jan 2010 10:40 I had to zero some disk partitions yesterday. I used this command: # dd if=/dev/zero of=/dev/sdb5 & # P=$! Then I checked its progress periodically, # kill -USR1 $P The progress checks showed significantly variable output speed, from 6 to 80 MB/s. I retested the same problem with a similar command, # cat /dev/zero >/dev/sdb5 This does not allow progress monitoring, but it does complete with an elapsed time consistent with the fastest disk I/O (i.e. 80 MB/s.) I am curious to why there is such a big difference for what would seem to be very similar I/O commands. Any comments? Other information: I ran this on Slackware 13.0 with kernel 2.6.30.10. The hardware was a Core 2 motherboard and Seagate SATA (3 Gb/s) hard disk. TIA -- Douglas Mayne
From: Grant on 17 Jan 2010 16:05 On Sun, 17 Jan 2010 15:40:49 +0000 (UTC), Douglas Mayne <doug(a)localhost.localnet> wrote: >I had to zero some disk partitions yesterday. I used this command: > ># dd if=/dev/zero of=/dev/sdb5 & ># P=$! > >Then I checked its progress periodically, > ># kill -USR1 $P > >The progress checks showed significantly variable output speed, from 6 to >80 MB/s. I retested the same problem with a similar command, > ># cat /dev/zero >/dev/sdb5 > >This does not allow progress monitoring, but it does complete with an >elapsed time consistent with the fastest disk I/O (i.e. 80 MB/s.) > >I am curious to why there is such a big difference for what would seem to >be very similar I/O commands. Any comments? dd runs much faster if you specify a large block size, for example: # dd if=/dev/zero bs=1M of=/dev/sdb5 & Sometimes I use bs=4k -- anything's better than the default 512 byte block writes ;) Grant. -- http://bugs.id.au
From: E. Lazardo on 17 Jan 2010 17:49 On Sun, 17 Jan 2010 15:40:49 +0000, Douglas Mayne wrote: > I had to zero some disk partitions yesterday. I used this command: > > # dd if=/dev/zero of=/dev/sdb5 & > # P=$! > > Then I checked its progress periodically, > > # kill -USR1 $P > > The progress checks showed significantly variable output speed, from 6 > to 80 MB/s. I retested the same problem with a similar command, > > # cat /dev/zero >/dev/sdb5 > > This does not allow progress monitoring, but it does complete with an > elapsed time consistent with the fastest disk I/O (i.e. 80 MB/s.) > > I am curious to why there is such a big difference for what would seem > to be very similar I/O commands. Any comments? > > Other information: I ran this on Slackware 13.0 with kernel 2.6.30.10. > The hardware was a Core 2 motherboard and Seagate SATA (3 Gb/s) hard > disk. > > TIA Block size is too small for a valid test. Try a range while keeping the aggregate data size constant to see the effects of block size. For example, the following will use a constant of 8GB which should avoid most desktop memory caching artifacts: in=/dev/zero out=/dev/sdb5 for bs in 1 2 4 8 16 32 64 128; do count=8192/$bs echo "echo 3 > /proc/sys/vm/drop_caches" sync dd if=$in of=$out bs=${bs}k count=$count iflag=noatime done Zone recording effects (google:ZCAV) can be gaged by rerunning with a wrapper using values from /proc/partitions. This example uses a 4GB partition: blocks=4008186 ((blocks-=100)) # prevents overrun loc=0 while [ $loc -lt $blocks ]; do for bs in 8 32 128; do count=8192/$bs echo "echo 3 > /proc/sys/vm/drop_caches" sync dd if=$in of=$out bs=${bs}k count=$count skip=$loc iflag=noatime done ((loc+=400818)) done For write testing, use 'seek' instead of 'skip' and 'oflag' instead of 'iflag'. Cheers,
From: Eef Hartman on 18 Jan 2010 07:28 In alt.os.linux.slackware Douglas Mayne <doug(a)localhost.localnet> wrote: > I had to zero some disk partitions yesterday. I used this command: > > # dd if=/dev/zero of=/dev/sdb5 & I think with dd you should use a significant larger blocksize (default is 512 bytes), with a option like "bs=1M" because /dev/zero is basiccally a character device (gives one byte at a time). > The progress checks showed significantly variable output speed, from 6 to > 80 MB/s. I retested the same problem with a similar command, dd first "fills" the block from the input, then writes it to the output. With a larger block the "filling" is done (it _is_ double buffered) while it's writing the previous block. I've noted the speed difference before when copying one disk to another (identical) one. -- ******************************************************************* ** Eef Hartman, Delft University of Technology, dept. SSC/ICT ** ** e-mail: E.J.M.Hartman(a)tudelft.nl - phone: +31-15-278 82525 ** *******************************************************************
From: Joseph Rosevear on 18 Jan 2010 18:23 In alt.os.linux.slackware Douglas Mayne <doug(a)localhost.localnet> wrote: > I had to zero some disk partitions yesterday. I used this command: > # dd if=/dev/zero of=/dev/sdb5 & > # P=$! Great. I didn't know about $!. > Then I checked its progress periodically, > # kill -USR1 $P I tried this and it killed the process. E.g., konqueror& P=$! kill -USR1 $P Is dd immune to being killed? What's up here? (I tried to look up some documentation on "kill -USR1". I couldn't find -USR1 mentioned in "man kill" or "help kill". Argh.) > The progress checks showed significantly variable output speed, from 6 to > 80 MB/s. I retested the same problem with a similar command, > # cat /dev/zero >/dev/sdb5 Great. I didn't know you could do this. > This does not allow progress monitoring, but it does complete with an > elapsed time consistent with the fastest disk I/O (i.e. 80 MB/s.) > I am curious to why there is such a big difference for what would seem to > be very similar I/O commands. Any comments? > Other information: I ran this on Slackware 13.0 with kernel 2.6.30.10. > The hardware was a Core 2 motherboard and Seagate SATA (3 Gb/s) hard disk. > TIA > -- > Douglas Mayne I guess I didn't help you with your question... unless the answer to your question is connected to my question. -Joe
|
Next
|
Last
Pages: 1 2 Prev: Software reset of ps2/usb adapter? Next: difference between striping using mdadm and LVM |