fuse: support splice() reading from fuse device
in [Kernel]
|
Prev: sched: change nohz idle load balancing logic to push model
Next: Another set of ioctl bkl pushdown, almost the end
From: Jens Axboe on 20 May 2010 07:30 On Thu, May 20 2010, Miklos Szeredi wrote: > This continues zero copy I/O support on the fuse interface. The first > part of the patchset (splice write support on fuse device) was posted > here: > > http://lkml.org/lkml/2010/4/28/215 > > With Jens' pipe growing patch and additional fuse patches it was > possible to achieve a 20GBytes/s write throghput on my laptop in a > "null" filesystem (no page cache, data goes to /dev/null). Do you have some numbers on how that compares to the same test with the default 16 page pipe size? -- Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Miklos Szeredi on 20 May 2010 07:50 On Thu, 20 May 2010, Jens Axboe wrote: > On Thu, May 20 2010, Miklos Szeredi wrote: > > This continues zero copy I/O support on the fuse interface. The first > > part of the patchset (splice write support on fuse device) was posted > > here: > > > > http://lkml.org/lkml/2010/4/28/215 > > > > With Jens' pipe growing patch and additional fuse patches it was > > possible to achieve a 20GBytes/s write throghput on my laptop in a > > "null" filesystem (no page cache, data goes to /dev/null). > > Do you have some numbers on how that compares to the same test with the > default 16 page pipe size? With the default 64k pipe size it's about 4 times slower than with a pipe size of 1MB. Miklos -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Miklos Szeredi on 20 May 2010 15:00 On Thu, 20 May 2010, Linus Torvalds wrote: > Are there actual real loads that get improved? I don't care if it means > that the improvement goes from three orders of magnitude to just a couple > of percent. The "couple of percent on actual loads" is a lot more > important than "many orders of magnitude on a made-up benchmark". The reason I've been looking at zero copy for fuse is that embedded people have been complaining about fuse's large CPU overhead for I/O. So large in fact that it was having a performance impact even for relatively slow devices. And most of that overhead comes from copying data around. So it's not the 20GB/s throughput that's interesting but the reduced CPU overhead, especially on slower processors. Apart from cache effects 20GB/s throughput with a null filesystem means 1% CPU at 200MB/s transfer speed with _any_ filesystem. Without bigger requests that translates to 4% overhead and without zero copy about 15%. That's on a core2/1.8GHz, with an embedded CPU the overhead reduction would be even more significant. Miklos -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Linus Torvalds on 20 May 2010 15:30 On Thu, 20 May 2010, Miklos Szeredi wrote: > > So it's not the 20GB/s throughput that's interesting but the reduced > CPU overhead, especially on slower processors. Apart from cache > effects 20GB/s throughput with a null filesystem means 1% CPU at > 200MB/s transfer speed with _any_ filesystem. No it doesn't. Really. It means 1% CPU at 200MB _IF_ you trigger the zero copy and nothing else! But that's a damn big if. Does it ever trigger in practice? I doubt it. In practice, you'll have to fill the pages with something in the first place. In practice, the destination of the data is such that you'll often end up copying anyway - it won't be /dev/null. That's why I claim your benchmark is meaningless. It does NOT even say what you claim it says. It does not say 1% CPU on a 200MB/s transfer, exactly the same way my stupid pipe zero-copy didn't mean that people could magically get MB/s throughput with 1% CPU on pipes. It says nothing at all, in short. You need to have a real source, and a real destination. Not some empty filesystem and /dev/null destination. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Miklos Szeredi on 20 May 2010 16:10
On Thu, 20 May 2010, Linus Torvalds wrote: > But that's a damn big if. Does it ever trigger in practice? I doubt it. In > practice, you'll have to fill the pages with something in the first place. > In practice, the destination of the data is such that you'll often end up > copying anyway - it won't be /dev/null. > > That's why I claim your benchmark is meaningless. It does NOT even say > what you claim it says. It does not say 1% CPU on a 200MB/s transfer, > exactly the same way my stupid pipe zero-copy didn't mean that people > could magically get MB/s throughput with 1% CPU on pipes. I'm talking about *overhead* not actual CPU usage. And I know that caches tend to reduce the effect of multiple copies, but that depends on a lot of things as well (size of request, delay between copies, etc.) Generally I've seen pretty significant reductions in overhead for eliminating each copy. I'm not saying it will always be zero copy all the way, I'm saying that less copies will tend to mean less overhead. And the same is true for making requests larger. > It says nothing at all, in short. You need to have a real source, and a > real destination. Not some empty filesystem and /dev/null destination. Sure, I will do that. It's just a lot harder to measure the effects on hardware I have access to, where the CPU speed is just damn too large compared to I/O speed. Miklos -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |