VFIO driver: Non-privileged user level PCI drivers
in [Kernel]
|
From: Tom Lyon on 1 Jul 2010 00:40 On Wednesday 30 June 2010 09:16:23 pm Alex Williamson wrote: > On Tue, 2010-06-08 at 14:21 -0700, Tom Lyon wrote: > > +int vfio_dma_unmap_dm(struct vfio_listener *listener, struct vfio_dma_map *dmp) > > +{ > > + unsigned long start, npage; > > + struct dma_map_page *mlp; > > + struct list_head *pos, *pos2; > > + int ret; > > + > > + start = dmp->vaddr & ~PAGE_SIZE; > > + npage = dmp->size >> PAGE_SHIFT; > > + > > + ret = -ENXIO; > > + mutex_lock(&listener->vdev->dgate); > > + list_for_each_safe(pos, pos2, &listener->dm_list) { > > + mlp = list_entry(pos, struct dma_map_page, list); > > + if (dmp->vaddr != mlp->vaddr || mlp->npage != npage) > > + continue; > > + ret = 0; > > + vfio_dma_unmap(listener, mlp); > > + break; > > + } > > Hi Tom, > > Shouldn't we be matching the mlp based on daddr instead of vaddr? We > can have multiple dma address pointing at the same virtual address, so > dma address is the unique element. I'm also nervous about this dm_list. > For qemu device assignment, we're potentially statically mapping many GB > of iova space. It seems like this could get incredibly bloated and > slow. Thanks, > > Alex In weird circumstances, differing user vaddrs could reolve to the same physical address, so the uniqueness of any mapping is the <vaddr,len>. Yes, a linear list is slow, but does qemu need a lot of mappings, or just big ones? > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Alex Williamson on 1 Jul 2010 01:20 On Wed, 2010-06-30 at 21:30 -0700, Tom Lyon wrote: > On Wednesday 30 June 2010 09:16:23 pm Alex Williamson wrote: > > On Tue, 2010-06-08 at 14:21 -0700, Tom Lyon wrote: > > > +int vfio_dma_unmap_dm(struct vfio_listener *listener, struct vfio_dma_map *dmp) > > > +{ > > > + unsigned long start, npage; > > > + struct dma_map_page *mlp; > > > + struct list_head *pos, *pos2; > > > + int ret; > > > + > > > + start = dmp->vaddr & ~PAGE_SIZE; > > > + npage = dmp->size >> PAGE_SHIFT; > > > + > > > + ret = -ENXIO; > > > + mutex_lock(&listener->vdev->dgate); > > > + list_for_each_safe(pos, pos2, &listener->dm_list) { > > > + mlp = list_entry(pos, struct dma_map_page, list); > > > + if (dmp->vaddr != mlp->vaddr || mlp->npage != npage) > > > + continue; > > > + ret = 0; > > > + vfio_dma_unmap(listener, mlp); > > > + break; > > > + } > > > > Hi Tom, > > > > Shouldn't we be matching the mlp based on daddr instead of vaddr? We > > can have multiple dma address pointing at the same virtual address, so > > dma address is the unique element. I'm also nervous about this dm_list. > > For qemu device assignment, we're potentially statically mapping many GB > > of iova space. It seems like this could get incredibly bloated and > > slow. Thanks, > > > > Alex > > In weird circumstances, differing user vaddrs could reolve to the same physical address, > so the uniqueness of any mapping is the <vaddr,len>. That sounds like another argument for using daddr to me, no? There's only one address space on the PCI bus, daddr. There's a 1:1 mapping of daddr to physical page, but an N:1 mapping of vaddr to a physical page. > Yes, a linear list is slow, but does qemu need a lot of mappings, or just big ones? I'm not sure yet, the first interface I tried seems to be giving me handfuls of pages. We sometimes reprogram the guest physical address (daddr) to a new virtual address (vaddr), but I don't know what the old virtual address was, so I can't unmap it. That's when I ran into the issue above. Thanks, Alex -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Alex Williamson on 1 Jul 2010 11:30 On Tue, 2010-06-08 at 14:21 -0700, Tom Lyon wrote: > +The VFIO_DMA_MASK ioctl is used to set the maximum permissible DMA address > +(device dependent). It takes a single unsigned 64 bit integer as an argument. > +This call also has the side effect of enabling PCI bus mastership. Hi Tom, This interface doesn't make sense for the MAP_IOVA user. Especially in qemu, we have no idea what the DMA mask is for the device we're assigning. It doesn't really matter though because the guest will use bounce buffers internally once it loads the device specific drivers and discovers the DMA mask. This only seems relevant if we're using a DMA_MAP call that gets to pick the dmaaddr, so I'd propose we only make this a required call for that interface, and create a separate ioctl for actually enabling bus master. Thanks, Alex -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Michael S. Tsirkin on 1 Jul 2010 11:40 On Thu, Jul 01, 2010 at 09:29:04AM -0600, Alex Williamson wrote: > On Tue, 2010-06-08 at 14:21 -0700, Tom Lyon wrote: > > +The VFIO_DMA_MASK ioctl is used to set the maximum permissible DMA address > > +(device dependent). It takes a single unsigned 64 bit integer as an argument. > > +This call also has the side effect of enabling PCI bus mastership. > > Hi Tom, > > This interface doesn't make sense for the MAP_IOVA user. Especially in > qemu, we have no idea what the DMA mask is for the device we're > assigning. It doesn't really matter though because the guest will use > bounce buffers internally once it loads the device specific drivers and > discovers the DMA mask. This only seems relevant if we're using a > DMA_MAP call that gets to pick the dmaaddr, so I'd propose we only make > this a required call for that interface, and create a separate ioctl for > actually enabling bus master. Thanks, > > Alex I expect there's no need for a separate ioctl to do this: you can do this by write to the control register. -- MST -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Alex Williamson on 1 Jul 2010 11:50
On Thu, 2010-07-01 at 18:31 +0300, Michael S. Tsirkin wrote: > On Thu, Jul 01, 2010 at 09:29:04AM -0600, Alex Williamson wrote: > > On Tue, 2010-06-08 at 14:21 -0700, Tom Lyon wrote: > > > +The VFIO_DMA_MASK ioctl is used to set the maximum permissible DMA address > > > +(device dependent). It takes a single unsigned 64 bit integer as an argument. > > > +This call also has the side effect of enabling PCI bus mastership. > > > > Hi Tom, > > > > This interface doesn't make sense for the MAP_IOVA user. Especially in > > qemu, we have no idea what the DMA mask is for the device we're > > assigning. It doesn't really matter though because the guest will use > > bounce buffers internally once it loads the device specific drivers and > > discovers the DMA mask. This only seems relevant if we're using a > > DMA_MAP call that gets to pick the dmaaddr, so I'd propose we only make > > this a required call for that interface, and create a separate ioctl for > > actually enabling bus master. Thanks, > > > > Alex > > I expect there's no need for a separate ioctl to do this: > you can do this by write to the control register. Nope, vfio only allows direct writes to the memory and io space bits of the command register, all other bits are virtualized. I wonder if that's necessary though since we require the device to be attached to an iommu domain before we allow config space access. Alex -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |