cp slows machine
in [Slackware]
|
From: Grant on 14 Dec 2009 14:32 On Mon, 14 Dec 2009 16:23:35 GMT, Mike Jones <Not(a)Arizona.Bay> wrote: >Responding to tapp: > >> Mike Jones <Not(a)Arizona.Bay> [Sun, 13 Dec 2009 23:49:53 +0000] >> >>> I've noticed a slowdown in things when setting cp to copy chunks of >>> data. >> >> With "chunks" you mean "100s of MB"? Because once your I/O is saturated, >> a certain slowdown at interactive desktop level is inevitable, with all >> OSs. The only way to get rid of this would be using SCSI disks, I guess. > > >Ah. Now I have a better idea of how things work. Ta. What kernel you running? Recently on lkml there was a discussion about heavy disk I/O upsetting foreground (desktop) responsiveness, perhaps a later kernel would better behave for you? > > >>> I'm thinking about sticking a 'nice' parameter in my .bashrc for cp >> >> That won't help much, but ionice(1) might. Also, check the kernel >> parameter CONFIG_HZ: setting it to 1000 will get you some more >> responsiveness, while losing a little throughput. (I don't know if this >> parameter can be changed without kernel recompile though.) Stock slackware-13 kernel has: $ egrep ^CONFIG_HZ /boot/config-generic-2.6.29.6 CONFIG_HZ_1000=y CONFIG_HZ=1000 $ egrep ^CONFIG_HZ /boot/config-huge-2.6.29.6 CONFIG_HZ_1000=y CONFIG_HZ=1000 > >Didn't even know I had ionice. I'll read up on it and play around with >it, see what happens. Cheers! Dunno about ionice, but the VM (virtual memory) and file writeout system (VFS?) has several tuning knobs one might play with, the description of them is under linux-($uname -r)/Documentation/* somewhere, I've not been there recently. Grant. -- http://bugsplatter.id.au
From: Grant on 14 Dec 2009 14:50 On Mon, 14 Dec 2009 16:22:06 GMT, Mike Jones <Not(a)Arizona.Bay> wrote: >Responding to Keith Keller: .... >> What's your disk subsystem like? LVM? md RAID? SATA, PATA, SAS? What >> do the various disk testers (e.g. vmstat, iostat) say? Are you swapping >> at all? .... >2 SATA HDDs and sometimes a fair bit of data transfer across the LAN to >and from these disks, and from disk to disk. Can you narrow down wehat causes the slowdown to either HDD file transfer, or over the LAN? > Have multiple swap files (4) >adding up to RAM size, but this "dragging" thing appears with or without >swap files or swap partitions. Four swapfiles on two HDDs makes little sense, unless you really need the swap space. I place one swap partition on each drive in a multi-drive box, and add ",pri=1" after 'defaults' to the appropriate /etc/fstab lines so that they're treated like RAID0. For example: $ cat /etc/fstab # /etc/fstab for slackware64-13.0 on pooh64 -- 2009-10-07 # /dev/sda3 / reiserfs defaults 0 0 /dev/sdb3 /usr reiserfs defaults 0 0 /dev/sdc2 /home/data reiserfs defaults,noauto 0 0 # /dev/sda5 swap swap defaults,pri=1 0 0 /dev/sdb5 swap swap defaults,pri=1 0 0 # #/dev/cdrom /mnt/cdrom auto noauto,owner,ro 0 0 /dev/fd0 /mnt/floppy auto noauto,owner 0 0 # devpts /dev/pts devpts gid=5,mode=620 0 0 proc /proc proc defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 # deltree:/home/common /home/common nfs hard,intr deltree:/home/mirror /home/mirror nfs hard,intr,noauto,user black:/home/buffer /home/buffer nfs hard,intr,noauto,user # No matter what size swap you have available, it's the swap usage that counts, use top, free or cat /proc/swaps to see usage. If you're regularly using lots of swap, it's better to add real memory. Grant. -- http://bugsplatter.id.au
From: Mike Jones on 14 Dec 2009 18:14 Responding to Grant: > On Mon, 14 Dec 2009 16:22:06 GMT, Mike Jones <Not(a)Arizona.Bay> wrote: > >>Responding to Keith Keller: > ... >>> What's your disk subsystem like? LVM? md RAID? SATA, PATA, SAS? >>> What do the various disk testers (e.g. vmstat, iostat) say? Are you >>> swapping at all? > ... >>2 SATA HDDs and sometimes a fair bit of data transfer across the LAN to >>and from these disks, and from disk to disk. > > Can you narrow down wehat causes the slowdown to either HDD file > transfer, or over the LAN? > >> Have multiple swap files >> (4) >>adding up to RAM size, but this "dragging" thing appears with or without >>swap files or swap partitions. > > Four swapfiles on two HDDs makes little sense, unless you really need > the swap space. > > I place one swap partition on each drive in a multi-drive box, and add > ",pri=1" after 'defaults' to the appropriate /etc/fstab lines so that > they're treated like RAID0. For example: > > $ cat /etc/fstab > # /etc/fstab for slackware64-13.0 on pooh64 -- 2009-10-07 # > /dev/sda3 / reiserfs defaults > 0 0 /dev/sdb3 /usr reiserfs defaults > 0 0 /dev/sdc2 /home/data reiserfs defaults,noauto > 0 0 # > /dev/sda5 swap swap defaults,pri=1 > 0 0 /dev/sdb5 swap swap defaults,pri=1 > 0 0 # > #/dev/cdrom /mnt/cdrom auto noauto,owner,ro > 0 0 /dev/fd0 /mnt/floppy auto noauto,owner > 0 0 # > devpts /dev/pts devpts gid=5,mode=620 > 0 0 proc /proc proc defaults > 0 0 tmpfs /dev/shm tmpfs defaults > 0 0 # > deltree:/home/common /home/common nfs hard,intr > deltree:/home/mirror /home/mirror nfs hard,intr,noauto,user > black:/home/buffer /home/buffer nfs hard,intr,noauto,user # > > No matter what size swap you have available, it's the swap usage that > counts, use top, free or cat /proc/swaps to see usage. > > If you're regularly using lots of swap, it's better to add real memory. > > Grant. Its not a LAN thing. Its when I set cp off copy\moving whole dir trees of data about. Doesn/t matter if I cp. use MC , Thunar, whatever. Its the data shifting that sucks up processing capacity. As for swap, I was messing about with a default auto-setup script that generates an emergency 500MiB swapfile, and just whacked in three more to make up the 2GiB RAM I've got on this machine. I don't think its ever used any of it. Thats what surprises me, that a machine with this kind of capacity should even feel moving data about in the background. -- *=( http://www.thedailymash.co.uk/ *=( For all your UK news needs.
From: Mike Jones on 14 Dec 2009 18:15 Responding to Grant: > On Mon, 14 Dec 2009 16:23:35 GMT, Mike Jones <Not(a)Arizona.Bay> wrote: > >>Responding to tapp: >> >>> Mike Jones <Not(a)Arizona.Bay> [Sun, 13 Dec 2009 23:49:53 +0000] >>> >>>> I've noticed a slowdown in things when setting cp to copy chunks of >>>> data. >>> >>> With "chunks" you mean "100s of MB"? Because once your I/O is >>> saturated, a certain slowdown at interactive desktop level is >>> inevitable, with all OSs. The only way to get rid of this would be >>> using SCSI disks, I guess. >> >> >>Ah. Now I have a better idea of how things work. Ta. > > What kernel you running? Recently on lkml there was a discussion about > heavy disk I/O upsetting foreground (desktop) responsiveness, perhaps a > later kernel would better behave for you? > > >> >>>> I'm thinking about sticking a 'nice' parameter in my .bashrc for cp >>> >>> That won't help much, but ionice(1) might. Also, check the kernel >>> parameter CONFIG_HZ: setting it to 1000 will get you some more >>> responsiveness, while losing a little throughput. (I don't know if >>> this parameter can be changed without kernel recompile though.) > > Stock slackware-13 kernel has: > $ egrep ^CONFIG_HZ /boot/config-generic-2.6.29.6 CONFIG_HZ_1000=y > CONFIG_HZ=1000 > $ egrep ^CONFIG_HZ /boot/config-huge-2.6.29.6 CONFIG_HZ_1000=y > CONFIG_HZ=1000 >> >>Didn't even know I had ionice. I'll read up on it and play around with >>it, see what happens. Cheers! > > Dunno about ionice, but the VM (virtual memory) and file writeout system > (VFS?) has several tuning knobs one might play with, the description of > them is under linux-($uname -r)/Documentation/* somewhere, I've not been > there recently. > > Grant. Slackware 12.2 with default hugesmp.s kernel. I'll take a look at that documentation. Cheers. -- *=( http://www.thedailymash.co.uk/ *=( For all your UK news needs.
From: Aaron W. Hsu on 14 Dec 2009 22:37 Mike Jones <Not(a)Arizona.Bay> writes: >I've noticed a slowdown in things when setting cp to copy chunks of data. >This mostly affects resource hungry stuff like Seamonkey, making things >feel as if I've lost a CPU. >I'm thinking about sticking a 'nice' parameter in my .bashrc for cp >Now is the time to say "Argh! Don't do that!" and\or "Try this..." >Ideas? It's one of those tricks that the modern computer vendors play on you. See, you do have lots of RAM and lots of processing power in your system, but many systems have an I/O bottleneck when it comes to reading data to and from the disks. It is always slower to do this sort of thing. Unfortunately, many of the Desktop Environments in use today make rather heavy usage of the disk, and some programs, like SeaMonkey, do as well. This means that when you are saturating your I/O throughput with something like the cp program, you're also going to be slowing down programs that rely on disk I/O. In other words, your processors and memory probably are not saturated, but your disk throughput probably is. My solution to this is using programs that don't rely on the disk unless it makes sense. That way, even if the disk is under heavy load, most programs remain responsive, because they don't need to write to the disk or read from it. Aaron W. Hsu -- A professor is one who talks in someone else's sleep.
First
|
Prev
|
Next
|
Last
Pages: 1 2 3 4 5 Prev: Bluetooth and Slack 13 Next: Re (3): HOW2 implement a daemon? |