mm: iommu: The Virtual Contiguous Memory Manager
in [Kernel]
|
Prev: [PATCH v2]kernel.h Move preprocessor #warning about using kernel headers in userpsace to types.h
Next: scsi/sg: remove casts from void*
From: Hari Kanigeri on 1 Jul 2010 19:00 > The VCMM takes the long view. Its designed for a future in which the > number of IOMMUs will go up and the ways in which these IOMMUs are > composed will vary from system to system, and may vary at > runtime. Already, there are ~20 different IOMMU map implementations in > the kernel. Had the Linux kernel had the VCMM, many of those > implementations could have leveraged the mapping and topology management > of a VCMM, while focusing on a few key hardware specific functions (map > this physical address, program the page table base register). > -- Sounds good. Did you think of a way to handle the cases where one of the Device that is using the mapped address crashed ? How is the physical address unbacked in this case ? Hari -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Andi Kleen on 1 Jul 2010 19:10 > The VCMM provides a more abstract, global view with finer-grained > control of each mapping a user wants to create. For instance, the > symantics of iommu_map preclude its use in setting up just the IOMMU > side of a mapping. With a one-sided map, two IOMMU devices can be Hmm? dma_map_* does not change any CPU mappings. It only sets up DMA mapping(s). > Additionally, the current IOMMU interface does not allow users to > associate one page table with multiple IOMMUs unless the user explicitly That assumes that all the IOMMUs on the system support the same page table format, right? As I understand your approach would help if you have different IOMMus with an different low level interface, which just happen to have the same pte format. Is that very likely? I would assume if you have lots of copies of the same IOMMU in the system then you could just use a single driver with multiple instances that share some state for all of them. That model would fit in the current interfaces. There's no reason multiple instances couldn't share the same allocation data structure. And if you have lots of truly different IOMMUs then they likely won't be able to share PTEs at the hardware level anyways, because the formats are too different. > The VCMM takes the long view. Its designed for a future in which the > number of IOMMUs will go up and the ways in which these IOMMUs are > composed will vary from system to system, and may vary at > runtime. Already, there are ~20 different IOMMU map implementations in > the kernel. Had the Linux kernel had the VCMM, many of those > implementations could have leveraged the mapping and topology management > of a VCMM, while focusing on a few key hardware specific functions (map > this physical address, program the page table base register). The standard Linux approach to such a problem is to write a library that drivers can use for common functionality, not put a middle layer inbetween. Libraries are much more flexible than layers. That said I'm not sure there's all that much duplicated code anyways. A lot of the code is always IOMMU specific. The only piece which might be shareable is the mapping allocation, but I don't think that's very much of a typical driver In my old pci-gart driver the allocation was all only a few lines of code, although given it was somewhat dumb in this regard because it only managed a small remapping window. -Andi -- ak(a)linux.intel.com -- Speaking for myself only. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zach Pfeffer on 2 Jul 2010 02:20 Andi Kleen wrote: >> The VCMM provides a more abstract, global view with finer-grained >> control of each mapping a user wants to create. For instance, the >> semantics of iommu_map preclude its use in setting up just the IOMMU >> side of a mapping. With a one-sided map, two IOMMU devices can be > > Hmm? dma_map_* does not change any CPU mappings. It only sets up > DMA mapping(s). Sure, but I was saying that iommu_map() doesn't just set up the IOMMU mappings, its sets up both the iommu and kernel buffer mappings. > >> Additionally, the current IOMMU interface does not allow users to >> associate one page table with multiple IOMMUs unless the user explicitly > > That assumes that all the IOMMUs on the system support the same page table > format, right? Actually no. Since the VCMM abstracts a page-table as a Virtual Contiguous Region (VCM) a VCM can be associated with any device, regardless of their individual page table format. > > As I understand your approach would help if you have different > IOMMus with an different low level interface, which just > happen to have the same pte format. Is that very likely? > > I would assume if you have lots of copies of the same IOMMU > in the system then you could just use a single driver with multiple > instances that share some state for all of them. That model > would fit in the current interfaces. There's no reason multiple > instances couldn't share the same allocation data structure. > > And if you have lots of truly different IOMMUs then they likely > won't be able to share PTEs at the hardware level anyways, because > the formats are too different. See VCM's above. > >> The VCMM takes the long view. Its designed for a future in which the >> number of IOMMUs will go up and the ways in which these IOMMUs are >> composed will vary from system to system, and may vary at >> runtime. Already, there are ~20 different IOMMU map implementations in >> the kernel. Had the Linux kernel had the VCMM, many of those >> implementations could have leveraged the mapping and topology management >> of a VCMM, while focusing on a few key hardware specific functions (map >> this physical address, program the page table base register). > > The standard Linux approach to such a problem is to write > a library that drivers can use for common functionality, not put a middle > layer in between. Libraries are much more flexible than layers. That's true up to the, "is this middle layer so useful that its worth it" point. The VM is a middle layer, you could make the same argument about it, "the mapping code isn't too hard, just map in the memory that you need and be done with it". But the VM middle layer provides a clean separation between page frames and pages which turns out to be infinitely useful. The VCMM is built in the same spirit, It says things like, "mapping is a global problem, I'm going to abstract entire virtual spaces and allow people arbitrary chuck size allocation, I'm not going to care that my device is physically mapping this buffer and this other device is a virtual, virtual device." > > That said I'm not sure there's all that much duplicated code anyways. > A lot of the code is always IOMMU specific. The only piece > which might be shareable is the mapping allocation, but I don't > think that's very much of a typical driver > > In my old pci-gart driver the allocation was all only a few lines of code, > although given it was somewhat dumb in this regard because it only managed a > small remapping window. I agree that its not a lot of code, and that this layer may be a bit heavy, but I'd like to focus on is a global mapping view useful and if so is something like the graph management that the VCMM provides generally useful. -- Sent by an employee of the Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zach Pfeffer on 2 Jul 2010 03:10 Hari Kanigeri wrote: >> He demonstrated the usage of his code in one of the emails he sent out >> initially. Did you go over that, and what (or how many) step would you >> use with the current code to do the same thing? > > -- So is this patch set adding layers and abstractions to help the User ? > > If the idea is to share some memory across multiple devices, I guess > you can achieve the same by calling the map function provided by iommu > module and sharing the mapped address to the 10's or 100's of devices > to access the buffers. You would only need a dedicated virtual pool > per IOMMU device to manage its virtual memory allocations. Yeah, you can do that. My idea is to get away from explicit addressing and encapsulate the "device address to physical address" link into a mapping. -- Sent by an employee of the Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Zach Pfeffer on 2 Jul 2010 03:40
Hari Kanigeri wrote: >> The VCMM takes the long view. Its designed for a future in which the >> number of IOMMUs will go up and the ways in which these IOMMUs are >> composed will vary from system to system, and may vary at >> runtime. Already, there are ~20 different IOMMU map implementations in >> the kernel. Had the Linux kernel had the VCMM, many of those >> implementations could have leveraged the mapping and topology management >> of a VCMM, while focusing on a few key hardware specific functions (map >> this physical address, program the page table base register). >> > > -- Sounds good. > Did you think of a way to handle the cases where one of the Device > that is using the mapped address crashed ? > How is the physical address unbacked in this case ? Actually the API takes care of that by design. Since the physical space is managed apart from the mapper the mapper can crash and not affect the physical memory allocation. -- Sent by an employee of the Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |