VFIO driver: Non-privileged user level PCI drivers
in [Kernel]
|
From: Tom Lyon on 3 Jun 2010 17:50 OK, in the interest of making progress, I am about to embark on the following: 1. Create a user-iommu-domain driver - opening it will give a new empty domain. Ultimately this can also populate sysfs with the state of its world, which would also be a good addition to the base iommu stuff. If someone closes the fd while in use, the domain stays valid anyway until users drop off. 2. Add DOMAIN_SET and DOMAIN_UNSET ioctls to the vfio driver. Require that a domain be set before using the VFIO_DMA_MAP_IOVA ioctl (this is the one that KVM wants). However, the VFIO_DMA_MAP_ANYWHERE ioctl is the one which uses the dma_sg interface which has no expicit control of domains. I intend to keep it the way it is, but expect only non-hypervisor programs would want to use it. 3. Clean up the docs and other nits that folks have found. Comments? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Michael S. Tsirkin on 6 Jun 2010 06:00 On Thu, Jun 03, 2010 at 02:41:38PM -0700, Tom Lyon wrote: > OK, in the interest of making progress, I am about to embark on the following: > > 1. Create a user-iommu-domain driver - opening it will give a new empty domain. > Ultimately this can also populate sysfs with the state of its world, which would > also be a good addition to the base iommu stuff. > If someone closes the fd while in use, the domain stays valid anyway until users > drop off. > > 2. Add DOMAIN_SET and DOMAIN_UNSET ioctls to the vfio driver. Require that > a domain be set before using the VFIO_DMA_MAP_IOVA ioctl Require domain to be set before you allow any access to the device: mmap, write, read. IMO this is the only safe way to make sure userspace does not corrupt memory, and this removes the need to special-case MSI memory, play with bus master enable and hope it can be cleared without reset, etc. > (this is the one > that KVM wants). Not sure I understand. I think that MAP should be done on the domain, not the device, this handles pinning pages correctly and this way you don't need any special checks. > However, the VFIO_DMA_MAP_ANYWHERE ioctl is the one > which uses the dma_sg interface which has no expicit control of domains. I > intend to keep it the way it is, but expect only non-hypervisor programs would > want to use it. If we support MAP_IOVA, why is MAP_ANYWHERE useful? Can't non-hypervisors just pick an address? > 3. Clean up the docs and other nits that folks have found. > > Comments? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Avi Kivity on 6 Jun 2010 09:50 On 06/02/2010 07:53 PM, Chris Wright wrote: > * Avi Kivity (avi(a)redhat.com) wrote: > >> The interface would only work for clients which support it: kvm, >> vhost, and iommu/devices with restartable dma. >> > BTW, there is no such thing as restartable dma. There is a provision in > new specs (read: no real hardware) that allows a device to request pages > before using them. So it's akin to demand paging, but the demand is an > explicit request rather than a page fault. > Thanks for the correction. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Tom Lyon on 7 Jun 2010 15:10 On Sunday 06 June 2010 02:54:51 am Michael S. Tsirkin wrote: > On Thu, Jun 03, 2010 at 02:41:38PM -0700, Tom Lyon wrote: > > OK, in the interest of making progress, I am about to embark on the following: > > > > 1. Create a user-iommu-domain driver - opening it will give a new empty domain. > > Ultimately this can also populate sysfs with the state of its world, which would > > also be a good addition to the base iommu stuff. > > If someone closes the fd while in use, the domain stays valid anyway until users > > drop off. > > > > 2. Add DOMAIN_SET and DOMAIN_UNSET ioctls to the vfio driver. Require that > > a domain be set before using the VFIO_DMA_MAP_IOVA ioctl > > Require domain to be set before you allow any access to the device: > mmap, write, read. IMO this is the only safe way to make sure userspace > does not corrupt memory, and this removes the need to special-case > MSI memory, play with bus master enable and hope it can be cleared without > reset, etc. Michael - the light bulb finally lit for me and I now understand what you've been saying the past few weeks. Of course you're right - we need iommu set before any register access. I had thought that was done by default but now I see that the dma_map_sg routine only attaches to the iommu on demand. So I will torpedo the MAP_ANYWHERE stuff. I'd like to keep the MAP_IOVA ioctl with the vfio fd so that the user can still do everything with one fd. I'm thinking the fd opens and iommu bindings could be done in a program before spinning out the program with the user driver. > > > (this is the one > > that KVM wants). > > Not sure I understand. I think that MAP should be done on the domain, > not the device, this handles pinning pages correctly and > this way you don't need any special checks. > > > However, the VFIO_DMA_MAP_ANYWHERE ioctl is the one > > which uses the dma_sg interface which has no expicit control of domains. I > > intend to keep it the way it is, but expect only non-hypervisor programs would > > want to use it. > > If we support MAP_IOVA, why is MAP_ANYWHERE useful? Can't > non-hypervisors just pick an address? > > > 3. Clean up the docs and other nits that folks have found. > > > > Comments? > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Michael S. Tsirkin on 8 Jun 2010 17:30
On Mon, Jun 07, 2010 at 12:01:04PM -0700, Tom Lyon wrote: > On Sunday 06 June 2010 02:54:51 am Michael S. Tsirkin wrote: > > On Thu, Jun 03, 2010 at 02:41:38PM -0700, Tom Lyon wrote: > > > OK, in the interest of making progress, I am about to embark on the following: > > > > > > 1. Create a user-iommu-domain driver - opening it will give a new empty domain. > > > Ultimately this can also populate sysfs with the state of its world, which would > > > also be a good addition to the base iommu stuff. > > > If someone closes the fd while in use, the domain stays valid anyway until users > > > drop off. > > > > > > 2. Add DOMAIN_SET and DOMAIN_UNSET ioctls to the vfio driver. Require that > > > a domain be set before using the VFIO_DMA_MAP_IOVA ioctl > > > > Require domain to be set before you allow any access to the device: > > mmap, write, read. IMO this is the only safe way to make sure userspace > > does not corrupt memory, and this removes the need to special-case > > MSI memory, play with bus master enable and hope it can be cleared without > > reset, etc. > > Michael - the light bulb finally lit for me and I now understand what you've been > saying the past few weeks. Of course you're right - we need iommu set before any > register access. I had thought that was done by default but now I see that the > dma_map_sg routine only attaches to the iommu on demand. > > So I will torpedo the MAP_ANYWHERE stuff. I'd like to keep the MAP_IOVA ioctl > with the vfio fd so that the user can still do everything with one fd. I'm thinking the > fd opens and iommu bindings could be done in a program before spinning out the > program with the user driver. This would kind of break Avi's idea that mappings are programmed at the domain and shared by multiple devices, won't it? > > > > > (this is the one > > > that KVM wants). > > > > Not sure I understand. I think that MAP should be done on the domain, > > not the device, this handles pinning pages correctly and > > this way you don't need any special checks. > > > > > However, the VFIO_DMA_MAP_ANYWHERE ioctl is the one > > > which uses the dma_sg interface which has no expicit control of domains. I > > > intend to keep it the way it is, but expect only non-hypervisor programs would > > > want to use it. > > > > If we support MAP_IOVA, why is MAP_ANYWHERE useful? Can't > > non-hypervisors just pick an address? > > > > > 3. Clean up the docs and other nits that folks have found. > > > > > > Comments? > > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |