[tgt]: Add BSG v4 backstore support to usr/bs_sg.c
in [Kernel]
|
Prev: Add a new structure for skb buffer from external.
Next: [PATCH 0001] Add Lenovo Thinkpad T400 to the whitelist
From: Nicholas A. Bellinger on 7 Jun 2010 07:20 On Mon, 2010-06-07 at 13:35 +0300, Boaz Harrosh wrote: > On 06/07/2010 12:37 PM, Nicholas A. Bellinger wrote: > > On Mon, 2010-06-07 at 12:27 +0300, Boaz Harrosh wrote: > >> On 06/07/2010 12:02 PM, Nicholas A. Bellinger wrote: > >>> On Mon, 2010-06-07 at 11:40 +0300, Boaz Harrosh wrote: > >>>> On 06/07/2010 06:50 AM, Nicholas A. Bellinger wrote: > >>>>> From: Nicholas Bellinger <nab(a)linux-iscsi.org> > >>>>> > >>>>> This patch adds initial support for the block layer implementation of the sg v4 interface > >>>>> (BSG) -> STGT with a new struct backingstore_template bsg_bst sharing code with the > >>>>> original sg_bst. It adds for new function bs_bsg_cmd_submit() for incoming write I/O > >>>>> and bs_bsg_cmd_complete() for polling read I/O using include/linux/bsg.h:struct sg_io_v4. > >>>>> > >>>>> This patch also splits up bs_sg_open() to support both BSG and SG looking for the > >>>>> path "/dev/bsg" in bs_sg_open(), and getting max_queue using SG_GET_COMMAND_Q and > >>>>> calling SG_SET_RESERVED_SIZE (following the original bs_sg code in init_sg_device) > >>>>> > >>>>> Also chk_bsg_device() currently using a hardcoded 254 major as def for BSG does not > >>>>> appear in include/linux/major.h just yet.. > >>>>> > >>>>> So far this has been tested with STGT/iSCSI <-> BSG <-> TCM_Loop SPC-4 iSCSI Target > >>>>> Port emulation and is able to mkfs, fsck and mount a filesystem from a TCM/IBLOCK > >>>>> Linux LVM kernel level backstore > >>>>> > >>>>> Signed-off-by: Nicholas A. Bellinger <nab(a)linux-iscsi.org> > >>>>> --- > >>>>> usr/bs_sg.c | 178 +++++++++++++++++++++++++++++++++++++++++++++++++++--- > >>>>> usr/scsi_cmnd.h | 7 ++ > >>>>> 2 files changed, 175 insertions(+), 10 deletions(-) > >>>>> > >>>>> diff --git a/usr/bs_sg.c b/usr/bs_sg.c > >>>>> index 23676ef..d071714 100644 > >>>>> --- a/usr/bs_sg.c > >>>>> +++ b/usr/bs_sg.c > >>> > >>> <SNIP> > >>> > >>>>> + > >>>>> +static int bs_bsg_cmd_submit(struct scsi_cmd *cmd) > >>>>> +{ > >>>>> + struct scsi_lu *dev = cmd->dev; > >>>>> + int fd = dev->fd; > >>>>> + struct sg_io_v4 *io_hdr = &cmd->cmd_bsg_hdr; > >>>>> + int err = 0; > >>>>> + /* > >>>>> + * Following linux/include/linux/bsg.h > >>>>> + */ > >>>>> + /* [i] 'Q' to differentiate from v3 */ > >>>>> + io_hdr->guard = 'Q'; > >>>>> + io_hdr->protocol = BSG_PROTOCOL_SCSI; > >>>>> + io_hdr->subprotocol = BSG_SUB_PROTOCOL_SCSI_CMD; > >>>>> + io_hdr->request_len = cmd->scb_len; > >>>>> + io_hdr->request = (unsigned long )cmd->scb; > >>>>> + > >>>>> + if (scsi_get_data_dir(cmd) == DATA_WRITE) { > >>>> > >>>> Why the if? both sides are fully bidi just remove the ifs > >>>> and: OUT <= OUT, IN <= IN > >>>> > >>> > >>> Whoops, thanks for pointing this one out (again). Tested with > >>> unidirection commands and included into my local tgt.git/pass that I > >>> will send along to Tomo-san.. > >>> > >>> Btw, I still need to sort out BIDI support for TCM_Loop at some point in > >>> the not too distant future. What would be a good way to generate BIDI > >>> traffic into STGT or a local SCSI LLD..? > >> > >> scsi_debug has some emulation for XOR commands, does that help? > >> > > > > Great, thanks for the pointer on the LLD side. Is there any userspace > > code around that will generate XDWRITEREAD_* CDBs for bulk I/O or do I > > need to code this part myself..? > > > > Best, > > > > --nab > > > > You could use iscsi as LLD and generate OSD heavy traffic. (With a real osd_tgtd > on the end side). But that will need more work, right? Hmmm, suppose I need to read up a bit on how to generate osd_tgtd traffic.. > what does the TCM_Loop > do exactly. Please forgive my laziness, I never took the time to find out. > TCM_Loop is a TCM fabric module that appears under /sys/kernel/config/target/loopback/ that emulates SAS, FC and iSCSI WWPN Ports and their assocated I_T Nexus for SPC-4 PR and ALUA operations. TCM_Loop registers as a Linux/SCSI LLD (a la scsi_debug) that presents TCM backstores (IBLOCK, FILEIO, RAMDISK) as a local struct scsi_device with a complete set of WWN port naming identifiers (sg_inq -i) and TransportIDs (for PR). With TCM v4 code in lio-core-2.6.git/lio-4.0, TCM_Loop has been converted along with the rest of the v4.x fabric modules to use a generic set of struct config_group logic from drivers/target/target_core_fabric_configfs.c. There is a bit of information on the wiki at: http://linux-iscsi.org/index.php/TCM_loop but as you will notice the screenshots are still referencing the original tests from last year with VMware Workstation. Things have changed quite a bit since then as TCM_Loop has gone from using internal SAS WWPN+Nexus emulation to providing multi-protocol support based on configfs provided WWPNs using the new generic naming handers from drivers/target/target_core_fabric_lib.c Best, --nab -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Boaz Harrosh on 7 Jun 2010 07:30
On 06/07/2010 02:07 PM, Nicholas A. Bellinger wrote: > On Mon, 2010-06-07 at 13:35 +0300, Boaz Harrosh wrote: >> On 06/07/2010 12:37 PM, Nicholas A. Bellinger wrote: >>> On Mon, 2010-06-07 at 12:27 +0300, Boaz Harrosh wrote: >>>> On 06/07/2010 12:02 PM, Nicholas A. Bellinger wrote: >>>>> On Mon, 2010-06-07 at 11:40 +0300, Boaz Harrosh wrote: >>>>>> On 06/07/2010 06:50 AM, Nicholas A. Bellinger wrote: >>>>>>> From: Nicholas Bellinger <nab(a)linux-iscsi.org> >>>>>>> >>>>>>> This patch adds initial support for the block layer implementation of the sg v4 interface >>>>>>> (BSG) -> STGT with a new struct backingstore_template bsg_bst sharing code with the >>>>>>> original sg_bst. It adds for new function bs_bsg_cmd_submit() for incoming write I/O >>>>>>> and bs_bsg_cmd_complete() for polling read I/O using include/linux/bsg.h:struct sg_io_v4. >>>>>>> >>>>>>> This patch also splits up bs_sg_open() to support both BSG and SG looking for the >>>>>>> path "/dev/bsg" in bs_sg_open(), and getting max_queue using SG_GET_COMMAND_Q and >>>>>>> calling SG_SET_RESERVED_SIZE (following the original bs_sg code in init_sg_device) >>>>>>> >>>>>>> Also chk_bsg_device() currently using a hardcoded 254 major as def for BSG does not >>>>>>> appear in include/linux/major.h just yet.. >>>>>>> >>>>>>> So far this has been tested with STGT/iSCSI <-> BSG <-> TCM_Loop SPC-4 iSCSI Target >>>>>>> Port emulation and is able to mkfs, fsck and mount a filesystem from a TCM/IBLOCK >>>>>>> Linux LVM kernel level backstore >>>>>>> >>>>>>> Signed-off-by: Nicholas A. Bellinger <nab(a)linux-iscsi.org> >>>>>>> --- >>>>>>> usr/bs_sg.c | 178 +++++++++++++++++++++++++++++++++++++++++++++++++++--- >>>>>>> usr/scsi_cmnd.h | 7 ++ >>>>>>> 2 files changed, 175 insertions(+), 10 deletions(-) >>>>>>> >>>>>>> diff --git a/usr/bs_sg.c b/usr/bs_sg.c >>>>>>> index 23676ef..d071714 100644 >>>>>>> --- a/usr/bs_sg.c >>>>>>> +++ b/usr/bs_sg.c >>>>> >>>>> <SNIP> >>>>> >>>>>>> + >>>>>>> +static int bs_bsg_cmd_submit(struct scsi_cmd *cmd) >>>>>>> +{ >>>>>>> + struct scsi_lu *dev = cmd->dev; >>>>>>> + int fd = dev->fd; >>>>>>> + struct sg_io_v4 *io_hdr = &cmd->cmd_bsg_hdr; >>>>>>> + int err = 0; >>>>>>> + /* >>>>>>> + * Following linux/include/linux/bsg.h >>>>>>> + */ >>>>>>> + /* [i] 'Q' to differentiate from v3 */ >>>>>>> + io_hdr->guard = 'Q'; >>>>>>> + io_hdr->protocol = BSG_PROTOCOL_SCSI; >>>>>>> + io_hdr->subprotocol = BSG_SUB_PROTOCOL_SCSI_CMD; >>>>>>> + io_hdr->request_len = cmd->scb_len; >>>>>>> + io_hdr->request = (unsigned long )cmd->scb; >>>>>>> + >>>>>>> + if (scsi_get_data_dir(cmd) == DATA_WRITE) { >>>>>> >>>>>> Why the if? both sides are fully bidi just remove the ifs >>>>>> and: OUT <= OUT, IN <= IN >>>>>> >>>>> >>>>> Whoops, thanks for pointing this one out (again). Tested with >>>>> unidirection commands and included into my local tgt.git/pass that I >>>>> will send along to Tomo-san.. >>>>> >>>>> Btw, I still need to sort out BIDI support for TCM_Loop at some point in >>>>> the not too distant future. What would be a good way to generate BIDI >>>>> traffic into STGT or a local SCSI LLD..? >>>> >>>> scsi_debug has some emulation for XOR commands, does that help? >>>> >>> >>> Great, thanks for the pointer on the LLD side. Is there any userspace >>> code around that will generate XDWRITEREAD_* CDBs for bulk I/O or do I >>> need to code this part myself..? >>> >>> Best, >>> >>> --nab >>> >> >> You could use iscsi as LLD and generate OSD heavy traffic. (With a real osd_tgtd >> on the end side). But that will need more work, right? > > Hmmm, suppose I need to read up a bit on how to generate osd_tgtd > traffic.. > Generating is easy, there is the exofs filke system that given an osd will export a VFS filesystem over it. but... >> what does the TCM_Loop >> do exactly. Please forgive my laziness, I never took the time to find out. >> > > TCM_Loop is a TCM fabric module that appears under /sys/kernel/config/target/loopback/ > that emulates SAS, FC and iSCSI WWPN Ports and their assocated I_T Nexus > for SPC-4 PR and ALUA operations. TCM_Loop registers as a Linux/SCSI > LLD (a la scsi_debug) that presents TCM backstores (IBLOCK, FILEIO, > RAMDISK) as a local struct scsi_device with a complete set of WWN port > naming identifiers (sg_inq -i) and TransportIDs (for PR). > > With TCM v4 code in lio-core-2.6.git/lio-4.0, TCM_Loop has been > converted along with the rest of the v4.x fabric modules to use a > generic set of struct config_group logic from > drivers/target/target_core_fabric_configfs.c. There is a bit of > information on the wiki at: > > http://linux-iscsi.org/index.php/TCM_loop > > but as you will notice the screenshots are still referencing the > original tests from last year with VMware Workstation. Things have > changed quite a bit since then as TCM_Loop has gone from using internal > SAS WWPN+Nexus emulation to providing multi-protocol support based on > configfs provided WWPNs using the new generic naming handers from > drivers/target/target_core_fabric_lib.c > OK, so from what I understand, from a SCSI stack perspective it is the lowest end target, using the local machine's backing store. That will not help at all with OSD. With osd_tgtd (In user mode) we use the iscsi protocol driver, but define a new disk-type, "osd" and a new backing-store "osdemu" which does a complete OSD-CDB decoding and implementation. Current code is not good for you because it's heavily user-mode dependent including an sqlite3 DB for storing attributes. Cheers Boaz > Best, > > --nab > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |