Add a helper function in PCI IOV to get VF device
in [Kernel]
|
Prev: linux-next: manual merge of the drbd tree with Linus' tree
Next: [RFC] lp_events: an lternitive to suspend blocker user mode and kernel API
From: Williams, Mitch A on 2 Jun 2010 15:00 >-----Original Message----- >From: Frank Pan [mailto:frankpzh(a)gmail.com] > >I'm not talking about PF's pci device, because PF >driver knows it. I'm talking about VF's. >The root cause is PF cannot get VF's bdf, so it has >no idea where VF is. IMO it's abnormal. > >Make symlinks is just a use case. When a userspace >app only knows VF's bdf, it has no idea how to get >into the VF's interface-like directories. >(/sys/class/net/ethX/vf[1-7]) >Because of: >1. Userspace app cannot get PF's bdf from VF's bdf >2. Userspace app cannot guess the interface name of >pf(ethX) >So a symlink from /sys/bus/pci/devices/xxxxx to >/sys/class/net/ethX/vf[1-7] is useful. >This is a real issue. > Frank, I'm not sure what you're trying to accomplish here. All of the information you need is already in sysfs. Given the PF device, you can look at /sys/class/net/ethX/device/virtfnX to get the bus/device/function of each of the VF devices. If the VF driver is loaded in your kernel, then given the bus/device/function of the vf device, you can look at /sys/class/net/ethX/device/virtfnX/net to see which interface corresponds to that VF. Furthermore, if the VF driver is loaded, you can find the PF device by looking at /sys/class/net/ethX/device/physfn, and you can find out which interface it is by looking at /sys/class/net/ethX/device/physfn/net So it's all there. You don't need anything else. The current PF drivers (both ixgbe and igb) do not directly manipulate sysfs at all, so there is no /sys/class/net/ethX/vfX. If you see this setup, you are using a very, very old version of the igb driver, which is not supported at all. Please upgrade to a recent kernel/driver combination. -Mitch >Thanks for reply. > >-- >Frank Pan > >Computer Science and Technology >Tsinghua University -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Frank Pan on 2 Jun 2010 23:00 hi Mitch, > Frank, I'm not sure what you're trying to accomplish here. All of the information you need is already in sysfs. Given the PF device, you can look at /sys/class/net/ethX/device/virtfnX to get the bus/device/function of each of the VF devices. Yes, that's the most funny part. Sysfs can only be traversed in usespace. So the thing userspace knows isn't known by driver(pf driver have no idea about vf's bdf), while the thing driver knows isn't known by userspace(one cannot infer pf's bdf from vf's bdf). Please think kernel/userspace as 2 system, they can hardly communicate these informations. IMHO give a syscall/ioctl telling these is funny and strange. > If the VF driver is loaded in your kernel, then given the bus/device/function of the vf device, you can look at /sys/class/net/ethX/device/virtfnX/net to see which interface corresponds to that VF. VF driver will never be loaded on physical machine, it can only be loaded in a virtual machine. On a physical machine, VF won't have any interface. > Furthermore, if the VF driver is loaded, you can find the PF device by looking at /sys/class/net/ethX/device/physfn, and you can find out which interface it is by looking at /sys/class/net/ethX/device/physfn/net So, when VF has no interface, this path is also unavailable. That's why I say given vf's bdf, one cannot infer pf's. > The current PF drivers (both ixgbe and igb) do not directly manipulate sysfs at all, so there is no /sys/class/net/ethX/vfX. If you see this setup, you are using a very, very old version of the igb driver, which is not supported at all. Please upgrade to a recent kernel/driver combination. You are right, vfX is not available now. Anyway, I'm hacking igb driver to provide informations about vf into userspace. When driver don't know vf's bdf and userspace don't know pf's bdf, this thing cannot be done. That's the motivation of this patch. Don't forget another one: PF cannot access VF's PCI space. This is also a <maybe common> use case. I need this to dump/change the VF's PCI space. Thanks for reply. -- Frank Pan Computer Science and Technology Tsinghua University -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Frank Pan on 2 Jun 2010 23:20 > So, when VF has no interface, this path is also unavailable. That's > why I say given vf's bdf, one cannot infer pf's. Sorry I just missed something. Userspace can find pf's bdf by physfn. So, the motivation is provide some information about vf from pf driver by sysfs. I cannot avoid to create one directory for each vf, like vf0, vf1, etc. Although userspace can find these directories, but it has no idea about which one to use. Userspace just knows vf's bdf, it won't care about/know vf's id. So I have the issue to create a symlink from each vf's /sys/bus/pci/devices/xxxx/ to their corresponding directories. And then comes the issue: PF cannot get VF's bdf in kernel space. This is not a strange issue, hacking every SR-IOV device can met. Thanks. -- Frank Pan Computer Science and Technology Tsinghua University -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Williams, Mitch A on 3 Jun 2010 13:40 >-----Original Message----- >From: Frank Pan [mailto:frankpzh(a)gmail.com] >Sent: Wednesday, June 02, 2010 8:11 PM [snip] >So, the motivation is provide some information about vf from pf driver by >sysfs. > >I cannot avoid to create one directory for each vf, like vf0, vf1, >etc. Although userspace can find these directories, but it has no idea >about which one to use. Userspace just knows vf's bdf, it won't care >about/know vf's id. So I have the issue to create a symlink from each >vf's /sys/bus/pci/devices/xxxx/ to their corresponding directories. > >And then comes the issue: PF cannot get VF's bdf in kernel space. > >This is not a strange issue, hacking every SR-IOV device can met. > Frank, I still don't understand what you're trying to do here. What's the use case? What is the end goal that you're trying to get to? What information do you need about the VF, and why? SR-IOV works properly today without this stuff. If we can understand what you're trying to do, then maybe we can help. -Mitch >Thanks. > >-- >Frank Pan > >Computer Science and Technology >Tsinghua University -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Chris Wright on 3 Jun 2010 14:00
* Frank Pan (frankpzh(a)gmail.com) wrote: > hi Mitch, > > > Frank, I'm not sure what you're trying to accomplish here. All of the information you need is already in sysfs. Given the PF device, you can look at /sys/class/net/ethX/device/virtfnX to get the bus/device/function of each of the VF devices. > > Yes, that's the most funny part. Sysfs can only be traversed in > usespace. So the thing userspace knows isn't known by driver(pf driver > have no idea about vf's bdf), while the thing driver knows isn't known > by userspace(one cannot infer pf's bdf from vf's bdf). > Please think kernel/userspace as 2 system, they can hardly communicate > these informations. IMHO give a syscall/ioctl telling these is funny > and strange. > > > If the VF driver is loaded in your kernel, then given the bus/device/function of the vf device, you can look at /sys/class/net/ethX/device/virtfnX/net to see which interface corresponds to that VF. > > VF driver will never be loaded on physical machine, it can only be > loaded in a virtual machine. On a physical machine, VF won't have any > interface. VF is often loaded on the physical machine. There's also a networking specific mechanism for querying and configuring a VF via the PF. While your patch is simple, it's unclear to me what your end goal is. The patch itself only adds a function. if you showed how you are planning to use it, that would really help. It's especially confusing that you are comparing your patch with symlinks visible in sysfs. thanks, -chris -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |