Frontswap (was Transcendent Memory): overview
in [Kernel]
|
Prev: firewire: core: check for 1394a compliant IRM, fix inaccessibility of Sony camcorder
Next: firewire: core: check for 1394a compliant IRM, fix inaccessibility of Sony camcorder
From: Nitin Gupta on 30 May 2010 14:20 On 05/28/2010 11:10 PM, Dan Magenheimer wrote: > [PATCH V2 0/4] Frontswap (was Transcendent Memory): overview > > Changes since V1: > - Rebased to 2.6.34 (no functional changes) > - Convert to sane types (per Al Viro comment in cleancache thread) > - Define some raw constants (Konrad Wilk) > - Performance analysis shows significant advantage for frontswap's > synchronous page-at-a-time design (vs batched asynchronous speculated > as an alternative design). See http://lkml.org/lkml/2010/5/20/314 > I think zram (http://lwn.net/Articles/388889/) is a more generic solution and can also achieve swap-to-hypervisor as a special case. zram is a generic in-memory compressed block device. To get frontswap functionality, such a device (/dev/zram0) can be exposed to a VM as a 'raw disk'. Such a disk can be used for _any_ purpose by the guest, including use as a swap disk. This method even works for Windows guests. Please see: http://www.vflare.org/2010/05/compressed-ram-disk-for-windows-virtual.html Here /dev/zram0 of size 2GB was created and exposed to Windows VM as a 'raw disk' (using VirtualBox). This disk was detected in the guest and NTFS filesystem was created on it (Windows cannot swap directly to a partition; it always uses swap file(s)). Then Windows was configured to swap over a file in this drive. Obviously, the same can be done with Linux guests. Thus, zram is useful in both native and virtualized environments with different use cases. Thanks, Nitin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Dan Magenheimer on 31 May 2010 13:20 > On 05/28/2010 11:10 PM, Dan Magenheimer wrote: > > [PATCH V2 0/4] Frontswap (was Transcendent Memory): overview > > > > Changes since V1: > > - Rebased to 2.6.34 (no functional changes) > > - Convert to sane types (per Al Viro comment in cleancache thread) > > - Define some raw constants (Konrad Wilk) > > - Performance analysis shows significant advantage for frontswap's > > synchronous page-at-a-time design (vs batched asynchronous > speculated > > as an alternative design). See http://lkml.org/lkml/2010/5/20/314 > > > > I think zram (http://lwn.net/Articles/388889/) is a more generic > solution > and can also achieve swap-to-hypervisor as a special case. > > zram is a generic in-memory compressed block device. To get frontswap > functionality, such a device (/dev/zram0) can be exposed to a VM as > a 'raw disk'. Such a disk can be used for _any_ purpose by the guest, > including use as a swap disk. Hi Nitin -- Though I agree zram is cool inside Linux, I don't see that it can be used to get the critical value of frontswap functionality in a virtual environment, specifically the 100% dynamic control by the hypervisor of every single page attempted to be "put" to frontswap. This is the key to the "intelligent overcommit" discussed in the previous long thread about frontswap. Further, by doing "guest-side compression" you are eliminating possibilities for KSM-style sharing, right? So while zram may be a great feature, it is NOT a more generic solution than frontswap, just a different solution that has a different set of objectives. Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Nitin Gupta on 31 May 2010 15:20 Hi Dan, On 05/31/2010 10:44 PM, Dan Magenheimer wrote: >> On 05/28/2010 11:10 PM, Dan Magenheimer wrote: >>> [PATCH V2 0/4] Frontswap (was Transcendent Memory): overview >>> >>> Changes since V1: >>> - Rebased to 2.6.34 (no functional changes) >>> - Convert to sane types (per Al Viro comment in cleancache thread) >>> - Define some raw constants (Konrad Wilk) >>> - Performance analysis shows significant advantage for frontswap's >>> synchronous page-at-a-time design (vs batched asynchronous >> speculated >>> as an alternative design). See http://lkml.org/lkml/2010/5/20/314 >>> >> >> I think zram (http://lwn.net/Articles/388889/) is a more generic >> solution >> and can also achieve swap-to-hypervisor as a special case. >> >> zram is a generic in-memory compressed block device. To get frontswap >> functionality, such a device (/dev/zram0) can be exposed to a VM as >> a 'raw disk'. Such a disk can be used for _any_ purpose by the guest, >> including use as a swap disk. > > > Though I agree zram is cool inside Linux, I don't see that it can > be used to get the critical value of frontswap functionality in a > virtual environment, specifically the 100% dynamic control by the > hypervisor of every single page attempted to be "put" to frontswap. > This is the key to the "intelligent overcommit" discussed in the > previous long thread about frontswap. > Yes, zram cannot return write/put failure for arbitrary pages but other than that what additional benefits does frontswap bring? Even with frontswap, whatever pages are once given out to hypervisor just stay there till guest reads them back. Unlike cleancache, you cannot free them at any point. So, it does not seem anyway more flexible than zram. One point I can see is additional block layer overhead in case of zram. For this, I have not yet done detailed measurements. > Further, by doing "guest-side compression" you are eliminating > possibilities for KSM-style sharing, right? > With zram, whether compression happens within guest or on the host, depends on how it is used. When zram device(s) are exported as raw disk(s) to a guest, pages written to them are sent to host and they are compressed on host an not within the guest. Also, I'm planning to include de-duplication support for zram too (which will be separate from KSM). > So while zram may be a great feature, it is NOT a more generic > solution than frontswap, just a different solution that has a > different set of objectives. > frontswap is a particular use case of zram disks. However, we still need to work on some issues with zram: - zram cannot return write/put failures for arbitrary pages. OTOH, frontswap can consult host before every put and may forward pages to in-guest swap device when put fails. - When a swap slot is freed, the notification from guest does not reach zram device(s) as exported from host. OTOH, frontswap calls frontswap_flush() which frees corresponding page from host memory. - Being a block device, it is potentially slower than frontswap approach. But being a generic device, its useful for all kinds of guest OS (including windows etc). Thanks, Nitin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Dan Magenheimer on 31 May 2010 20:30
> From: Nitin Gupta [mailto:ngupta(a)vflare.org] > frontswap is a particular use case of zram disks. However, we still > need to work on some issues with zram: > - zram cannot return write/put failures for arbitrary pages. OTOH, > frontswap can consult host before every put and may forward pages to > in-guest swap device when put fails. > - When a swap slot is freed, the notification from guest does > not reach zram device(s) as exported from host. OTOH, frontswap calls > frontswap_flush() which frees corresponding page from host memory. > - Being a block device, it is potentially slower than frontswap > approach. But being a generic device, its useful for all kinds > of guest OS (including windows etc). Hi Nitin -- This is a good list (not sure offhand it is complete or not) of the key differences between zram and frontswap. Unless/until zram solves each of these issues -- which are critical to the primary objective of frontswap (namely intelligent overcommit) -- I simply can't agree that frontswap is a particular use case of zram. Zram is just batched asynchronous I/O to a fixed-size device with a bonus of on-the-fly compression. Cool, yes. Useful, yes. Useful in some cases in a virtualized environment, yes. But a superset/replacement of frontswap, no. > Yes, zram cannot return write/put failure for arbitrary pages but other > than that what additional benefits does frontswap bring? Even with > frontswap, > whatever pages are once given out to hypervisor just stay there till > guest > reads them back. Unlike cleancache, you cannot free them at any point. > So, > it does not seem anyway more flexible than zram. The flexibility is that the hypervisor can make admittance decisions on each individual page... this is exactly what allows for intelligent overcommit. Since the pages "just stay there until the guest reads them back", the hypervisor must be very careful about which and how many pages it accepts and the admittance decisions must be very dynamic, depending on a lot of factors not visible to any individual guest and not timely enough to be determined by the asynchronous "backend I/O" subsystem of a host or dom0. Thanks, Dan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |