pci: pciehp update the slot bridge res to get big range for pcie devices
in [Kernel]
|
Prev: Broken ondemand scheduler in Linux 2.6.30+ on Pentium IVs
Next: [BUGFIX] crypto, ghash_clmul_ni, Fix building failure on x86_32
From: Kenji Kaneshige on 26 Oct 2009 01:00 Yinghai Lu wrote: > move out bus_size_bridges and assign resources out of pciehp_add_bridge() > and at last do them all together one time including slot bridge, to avoid to call > assign resources several times, when there are several bridges under the slot bridge. > > need to introduce pci_bridge_assign_resources there. > > handle the case the first bridge that doesn't get pre-allocated big enough res from FW. > for example pcie devices need 256M, but the bridge only get preallocated 2M... > Though I have not looked at the patch deeply yet (sorry), I have some questions and concerns about this change. Please correct me if my understanding is not correct. - Your patch doesn't seems to have the code to free resources. If we need to expand the resource range, don't we need to free preallocated resource before allocating the new one? - Your patch seems to update BARs for bridge itself. I think it would break the bridge's driver (port service driver) that if it controls the device's capability by using IO/Mem, though I don't know if such driver or capabilities exists now. And one comment about pci_configure_slot(). We need to program hotplug parameters before the device start working. So we need to call pci_configure_slot() before calling pci_bus_add_devices(). Thanks, Kenji Kaneshige > pci_is_enabled must be removed in pci_setup_bridge(), otherwise update res is not > updated to bridge BAR. and we are safe to do that, because only have one bus_size > and assigned resources are called. > > Signed-off-by: Yinghai Lu <yinghai(a)kernel.org> > > --- > drivers/pci/hotplug/pciehp_pci.c | 29 ++++++++++++---- > drivers/pci/setup-bus.c | 70 +++++++++++++++++++++++++++++++++++++-- > include/linux/pci.h | 1 > 3 files changed, 90 insertions(+), 10 deletions(-) > > Index: linux-2.6/drivers/pci/hotplug/pciehp_pci.c > =================================================================== > --- linux-2.6.orig/drivers/pci/hotplug/pciehp_pci.c > +++ linux-2.6/drivers/pci/hotplug/pciehp_pci.c > @@ -53,19 +53,18 @@ static int __ref pciehp_add_bridge(struc > busnr = pci_scan_bridge(parent, dev, busnr, pass); > if (!dev->subordinate) > return -1; > - pci_bus_size_bridges(dev->subordinate); > - pci_bus_assign_resources(parent); > - pci_enable_bridges(parent); > - pci_bus_add_devices(parent); > + > return 0; > } > > int pciehp_configure_device(struct slot *p_slot) > { > struct pci_dev *dev; > - struct pci_bus *parent = p_slot->ctrl->pcie->port->subordinate; > + struct pci_dev *bridge = p_slot->ctrl->pcie->port; > + struct pci_bus *parent = bridge->subordinate; > int num, fn; > struct controller *ctrl = p_slot->ctrl; > + int retval; > > dev = pci_get_slot(parent, PCI_DEVFN(0, 0)); > if (dev) { > @@ -96,12 +95,28 @@ int pciehp_configure_device(struct slot > (dev->hdr_type == PCI_HEADER_TYPE_CARDBUS)) { > pciehp_add_bridge(dev); > } > - pci_configure_slot(dev); > pci_dev_put(dev); > } > > - pci_bus_assign_resources(parent); > + pci_bus_size_bridges(parent); > + pci_bridge_assign_resources(bridge); > + retval = pci_reenable_device(bridge); > + pci_set_master(bridge); > + pci_enable_bridges(parent); > pci_bus_add_devices(parent); > + > + for (fn = 0; fn < 8; fn++) { > + dev = pci_get_slot(parent, PCI_DEVFN(0, fn)); > + if (!dev) > + continue; > + if ((dev->class >> 16) == PCI_BASE_CLASS_DISPLAY) { > + pci_dev_put(dev); > + continue; > + } > + pci_configure_slot(dev); > + pci_dev_put(dev); > + } > + > return 0; > } > > Index: linux-2.6/drivers/pci/setup-bus.c > =================================================================== > --- linux-2.6.orig/drivers/pci/setup-bus.c > +++ linux-2.6/drivers/pci/setup-bus.c > @@ -27,6 +27,44 @@ > #include <linux/slab.h> > #include "pci.h" > > +static void pdev_assign_resources_sorted(struct pci_dev *dev) > +{ > + struct resource *res; > + struct resource_list head, *list, *tmp; > + int idx; > + u16 class = dev->class >> 8; > + > + head.next = NULL; > + > + /* Don't touch classless devices or host bridges or ioapics. */ > + if (class == PCI_CLASS_NOT_DEFINED || > + class == PCI_CLASS_BRIDGE_HOST) > + return; > + > + /* Don't touch ioapic devices already enabled by firmware */ > + if (class == PCI_CLASS_SYSTEM_PIC) { > + u16 command; > + pci_read_config_word(dev, PCI_COMMAND, &command); > + if (command & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) > + return; > + } > + > + pdev_sort_resources(dev, &head); > + > + for (list = head.next; list;) { > + res = list->res; > + idx = res - &list->dev->resource[0]; > + if (pci_assign_resource(list->dev, idx)) { > + res->start = 0; > + res->end = 0; > + res->flags = 0; > + } > + tmp = list; > + list = list->next; > + kfree(tmp); > + } > +} > + > static void pbus_assign_resources_sorted(const struct pci_bus *bus) > { > struct pci_dev *dev; > @@ -144,9 +182,6 @@ static void pci_setup_bridge(struct pci_ > u32 l, bu, lu, io_upper16; > int pref_mem64; > > - if (pci_is_enabled(bridge)) > - return; > - > dev_info(&bridge->dev, "PCI bridge, secondary bus %04x:%02x\n", > pci_domain_nr(bus), bus->number); > > @@ -550,6 +585,35 @@ void __ref pci_bus_size_bridges(struct p > } > EXPORT_SYMBOL(pci_bus_size_bridges); > > +void __ref pci_bridge_assign_resources(const struct pci_dev *bridge) > +{ > + struct pci_bus *b; > + > + pdev_assign_resources_sorted((struct pci_dev *)bridge); > + > + b = bridge->subordinate; > + if (!b) > + return; > + > + pci_bus_assign_resources(b); > + > + switch (bridge->class >> 8) { > + case PCI_CLASS_BRIDGE_PCI: > + pci_setup_bridge(b); > + break; > + > + case PCI_CLASS_BRIDGE_CARDBUS: > + pci_setup_cardbus(b); > + break; > + > + default: > + dev_info(&bridge->dev, "not setting up bridge for bus " > + "%04x:%02x\n", pci_domain_nr(b), b->number); > + break; > + } > +} > +EXPORT_SYMBOL(pci_bridge_assign_resources); > + > void __ref pci_bus_assign_resources(const struct pci_bus *bus) > { > struct pci_bus *b; > Index: linux-2.6/include/linux/pci.h > =================================================================== > --- linux-2.6.orig/include/linux/pci.h > +++ linux-2.6/include/linux/pci.h > @@ -756,6 +756,7 @@ ssize_t pci_write_vpd(struct pci_dev *de > int pci_vpd_truncate(struct pci_dev *dev, size_t size); > > /* Helper functions for low-level code (drivers/pci/setup-[bus,res].c) */ > +void pci_bridge_assign_resources(const struct pci_dev *bridge); > void pci_bus_assign_resources(const struct pci_bus *bus); > void pci_bus_size_bridges(struct pci_bus *bus); > int pci_claim_resource(struct pci_dev *, int); > > -- > To unsubscribe from this list: send the line "unsubscribe linux-pci" in > the body of a message to majordomo(a)vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Yinghai Lu on 26 Oct 2009 02:00 Kenji Kaneshige wrote: > Yinghai Lu wrote: >> move out bus_size_bridges and assign resources out of pciehp_add_bridge() >> and at last do them all together one time including slot bridge, to >> avoid to call >> assign resources several times, when there are several bridges under >> the slot bridge. >> >> need to introduce pci_bridge_assign_resources there. >> >> handle the case the first bridge that doesn't get pre-allocated big >> enough res from FW. >> for example pcie devices need 256M, but the bridge only get >> preallocated 2M... >> > > Though I have not looked at the patch deeply yet (sorry), I have > some questions and concerns about this change. Please correct me > if my understanding is not correct. > > - Your patch doesn't seems to have the code to free resources. > If we need to expand the resource range, don't we need to free > preallocated resource before allocating the new one? that is done with pci_bus_size_bridges ==> pbus_size_io/pbus_size_mem ==> find_free_bus_resource ==> release_resource. > > - Your patch seems to update BARs for bridge itself. I think it > would break the bridge's driver (port service driver) that if > it controls the device's capability by using IO/Mem, though I > don't know if such driver or capabilities exists now. port service driver will be AER and pciehotplug. it seems those driver are not use those BAR... those BAR are supposed for the devices under the pcie bridge. > > And one comment about pci_configure_slot(). We need to program > hotplug parameters before the device start working. So we need > to call pci_configure_slot() before calling pci_bus_add_devices(). > Thanks, will correct that. YH YH -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Kenji Kaneshige on 26 Oct 2009 03:50 Yinghai Lu wrote: > Kenji Kaneshige wrote: >> Yinghai Lu wrote: >>> move out bus_size_bridges and assign resources out of pciehp_add_bridge() >>> and at last do them all together one time including slot bridge, to >>> avoid to call >>> assign resources several times, when there are several bridges under >>> the slot bridge. >>> >>> need to introduce pci_bridge_assign_resources there. >>> >>> handle the case the first bridge that doesn't get pre-allocated big >>> enough res from FW. >>> for example pcie devices need 256M, but the bridge only get >>> preallocated 2M... >>> >> Though I have not looked at the patch deeply yet (sorry), I have >> some questions and concerns about this change. Please correct me >> if my understanding is not correct. >> >> - Your patch doesn't seems to have the code to free resources. >> If we need to expand the resource range, don't we need to free >> preallocated resource before allocating the new one? > > that is done with pci_bus_size_bridges ==> pbus_size_io/pbus_size_mem ==> find_free_bus_resource ==> release_resource. > I didn't noticed that find_free_bus_resource() was changed to call release_resource() recently... By the way, does this (release_resource() by find_bus_resource()) change the resource assignment by BIOS also for bridges other than the ports with hotplug slot (switch upstreamport, for example)? >> - Your patch seems to update BARs for bridge itself. I think it >> would break the bridge's driver (port service driver) that if >> it controls the device's capability by using IO/Mem, though I >> don't know if such driver or capabilities exists now. > > port service driver will be AER and pciehotplug. > it seems those driver are not use those BAR... > those BAR are supposed for the devices under the pcie bridge. > I understand that there are only two port service drivers (AER and PCIe hotplug) and both doesn't use BAR. But I still have a concern that changing bridge's BARs might cause problems in the future. In my understanding, what you need is expanding IO/Mem base and limit of root or switch downstream ports. If so, I think we should only touch IO/Mem base/limit, and should not touch bridge's BARs. Thanks, Kenji Kaneshige -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Yinghai Lu on 26 Oct 2009 04:30 Kenji Kaneshige wrote: > Yinghai Lu wrote: >> Kenji Kaneshige wrote: >>> Yinghai Lu wrote: >>>> move out bus_size_bridges and assign resources out of >>>> pciehp_add_bridge() >>>> and at last do them all together one time including slot bridge, to >>>> avoid to call >>>> assign resources several times, when there are several bridges under >>>> the slot bridge. >>>> >>>> need to introduce pci_bridge_assign_resources there. >>>> >>>> handle the case the first bridge that doesn't get pre-allocated big >>>> enough res from FW. >>>> for example pcie devices need 256M, but the bridge only get >>>> preallocated 2M... >>>> >>> Though I have not looked at the patch deeply yet (sorry), I have >>> some questions and concerns about this change. Please correct me >>> if my understanding is not correct. >>> >>> - Your patch doesn't seems to have the code to free resources. >>> If we need to expand the resource range, don't we need to free >>> preallocated resource before allocating the new one? >> >> that is done with pci_bus_size_bridges ==> pbus_size_io/pbus_size_mem >> ==> find_free_bus_resource ==> release_resource. >> > > I didn't noticed that find_free_bus_resource() was changed to call > release_resource() recently... > > By the way, does this (release_resource() by find_bus_resource()) > change the resource assignment by BIOS also for bridges other than > the ports with hotplug slot (switch upstreamport, for example)? yes. BIOS preallocate small range for the bridge, and leave the BAR for the device under that bridge uninitialized. > >>> - Your patch seems to update BARs for bridge itself. I think it >>> would break the bridge's driver (port service driver) that if >>> it controls the device's capability by using IO/Mem, though I >>> don't know if such driver or capabilities exists now. >> >> port service driver will be AER and pciehotplug. >> it seems those driver are not use those BAR... >> those BAR are supposed for the devices under the pcie bridge. >> > > I understand that there are only two port service drivers (AER and > PCIe hotplug) and both doesn't use BAR. But I still have a concern > that changing bridge's BARs might cause problems in the future. In > my understanding, what you need is expanding IO/Mem base and limit > of root or switch downstream ports. If so, I think we should only > touch IO/Mem base/limit, and should not touch bridge's BARs. those bridge BAR are for devices under that bridge. the port device is not supposed to use them. if we don't touch the bridge's BAR, the hw will not forward the io for those devices under it. YH -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Kenji Kaneshige on 26 Oct 2009 06:30
Yinghai Lu wrote: > Kenji Kaneshige wrote: >> Yinghai Lu wrote: >>> Kenji Kaneshige wrote: >>>> Yinghai Lu wrote: >>>>> move out bus_size_bridges and assign resources out of >>>>> pciehp_add_bridge() >>>>> and at last do them all together one time including slot bridge, to >>>>> avoid to call >>>>> assign resources several times, when there are several bridges under >>>>> the slot bridge. >>>>> >>>>> need to introduce pci_bridge_assign_resources there. >>>>> >>>>> handle the case the first bridge that doesn't get pre-allocated big >>>>> enough res from FW. >>>>> for example pcie devices need 256M, but the bridge only get >>>>> preallocated 2M... >>>>> >>>> Though I have not looked at the patch deeply yet (sorry), I have >>>> some questions and concerns about this change. Please correct me >>>> if my understanding is not correct. >>>> >>>> - Your patch doesn't seems to have the code to free resources. >>>> If we need to expand the resource range, don't we need to free >>>> preallocated resource before allocating the new one? >>> that is done with pci_bus_size_bridges ==> pbus_size_io/pbus_size_mem >>> ==> find_free_bus_resource ==> release_resource. >>> >> I didn't noticed that find_free_bus_resource() was changed to call >> release_resource() recently... >> >> By the way, does this (release_resource() by find_bus_resource()) >> change the resource assignment by BIOS also for bridges other than >> the ports with hotplug slot (switch upstreamport, for example)? > > yes. > > BIOS preallocate small range for the bridge, and leave the BAR for the device under that bridge uninitialized. > Does this happen at the boot time regardless of hot-plug? >>>> - Your patch seems to update BARs for bridge itself. I think it >>>> would break the bridge's driver (port service driver) that if >>>> it controls the device's capability by using IO/Mem, though I >>>> don't know if such driver or capabilities exists now. >>> port service driver will be AER and pciehotplug. >>> it seems those driver are not use those BAR... >>> those BAR are supposed for the devices under the pcie bridge. >>> >> I understand that there are only two port service drivers (AER and >> PCIe hotplug) and both doesn't use BAR. But I still have a concern >> that changing bridge's BARs might cause problems in the future. In >> my understanding, what you need is expanding IO/Mem base and limit >> of root or switch downstream ports. If so, I think we should only >> touch IO/Mem base/limit, and should not touch bridge's BARs. > > those bridge BAR are for devices under that bridge. the port device is not supposed to use them. > Do you mean you touch only BARs of the devices under the bridge? > if we don't touch the bridge's BAR, the hw will not forward the io for those devices under it. > I understand you need to touch I/O base/limit and Mem base/limit. But I don't understand why you also need to update bridge's BARs. Could you please explain a little more about it? Just in case, my terminology "bridge's BARs" is Base Address Register 0 (offset 0x10) and Base Address Register 1 (offset 0x14) in the (type 1) configuration space header of the bridge. Thanks, Kenji Kaneshige -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |