Help with pdflush latency / congestion on older kernel (2.6.18)
in [Kernel]
|
Prev: [regression] 2.6.34-rc1 to -rc3 on zaurus: no longer boots
Next: [PATCH -v2] Only honor the FIGETBSZ ioctl for regular files and directories
From: Erik Slagter on 14 Jun 2010 10:20 Hi all, First of all I must apologise that this relates to an older kernel version. This kernel is used on a satellite top-box with a closed source module of it's vendor that handles the specific hardware, so we cannot use a new kernel, sadly. The good news is that the problem has nothing to do with the closed source module. I was hoping that maybe this is a known problem and has been fixed in newer kernel version and so I can try to backport it (or maybe that has already been done...) This is the problem. Until recently a 2.6.9 kernel was used (yep, I hesitate to admit...) The vendor now switched to 2.6.18 and since then we have a sort of disk write congestion problem, at least on NFS but maybe also on local (ata) disks. It appears that pdflush waits too long before it starts writing the dirty pages, which results in the processes that fetch and write the data, are being blocked at a certain point. In the meantime, data keeps coming in (satellite tv being notorious for it's lack of flow control ;-)) and the hardware and/or os level queues overflow, because they're not fetched in time. This is especially true when more than one recording takes place and/or a simultaneous playback takes place. I think changing the user space process (like using additional threads) will not solve the problem completely and also would be rather a workaround instead. We've done lot's of testing and tweaking in the meantime, detailed information is available at request! Things we tried already: - various incantations of posix_fadvise on the file to be written, only once or on a frequent base (posix_fadvise(...DONT_NEED...)) - opening using O_SYNC - tweaking /proc/vm/: dirty_background_ratio, dirty_expire_centisecs, dirty_ratio, dirty_writeback_centisecs It appears that dirty_writeback_centisecs cannot be set to any value below 5 seconds. We believe that the possibility to wake up pdflush earlier actually would help in this case. Would using O_DIRECT be a possible solution? I cannot test that at the moment though, because the out-of-the-box doesn't have it enabled in the config, but I probably can get hold a kernel that has it enabled. Can anyone supply some help, clues, hints, tips, etc., it's very much appreciated! Thanks very much! -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |