Analysis to develop an ISCSI Layer inside Microsoft Windows NT/2K/XP Ing. Franco Ometti While1 S.r.l.
Description The activity is related to develop a special device driver that implements a SCSI miniport paradigm. In this case the module doesn t interface one specific local hardware. In fact, it is necessary to contact a specific storage equipment that includes some physical devices connected to the network. The SCSI HBA driver must redirect any I/O operation to the network using a specific ISCSI protocol in order to reach the configurated remote disks. To realize these features, it is necessary that the HBA driver interfaces a network driver via TDI interface (Transport Driver Interface). In this document there are a couple of figures that describes the Windows NT Lan architecture. iscsi that we will be define in this proposal is related to the current IETF specifications provided into the official document (Draft 08). iscsi is SCSI over TCP/IP providing up to eight remote SCSI targets with each target capable of supporting eight separate LUNs. Remote SCSI targets may be on one or multiple Remote Disk Devices (i.e. Cisco SN5420), up to a maximum of eight devices. In order to reach this target under Win 2000 systems, it is necessary to develop a Miniport SCSI driver that contains the ISCSI protocol implementation. This component, from now named BIPIPE (Basic iscsi Pipeline), must contain two different levels respectively related to manage the SCSI Miniport interface and the TDI client semantics. In this special case this miniport cannot handle a physical interrupt. In order to solve the asynchronous end of the iscsi operation, it is necessary to combine the call-back effect provided by the TDI interface and one internal Timer armed during the init phase and responsible to manage any status change of the connection (Down, Up, Active, etc.). To handle any setting related to the behavior of the iscsi miniport, it is also necessary to provide a proprietary configuration utility. This program should be called during the installation phase and during run-rime as well. Related to the iscsi environment, it is necessary to identify any remote device that are requested. To do this, it must be possible to handle the IP address, the login parameters, etc. During installation of iscsi driver, the administrator will define the controlled target IP addresses. These will be saved and returned to Windows at the next boot. At that time, via Disk Administrator utility, the administrator will define partitions. Even these data are to be saved via iscsi administration utility in order to complete the scenario returned to Windows in next reboots. Windows 2000 has not networking up at boot time, so it is necessary to set configuration in order to answer correctly to SCSI Inquiry phase. This information will be stored in Registry. When Windows will start inquiry cycle, iscsi driver will answer getting information from stored data. In that way, Windows will keep track of the devices in its internal device database.
Moreover, iscsi subsystem has to know all the data about partitions since Win 2000 does not know them until it can access to the devices. To assure this information, iscsi will save the main configuration details. These will be returned during boot phase. This kind of data presentation will last until iscsi will be able to access all the device IP addresses. During that period, no direct access to network device will be allowed by iscsi driver. Main characteristics In order to define the layout of the iscsi miniport driver and the TDI connector, now we present the list of the main characteristics that we include in this level: SW architecture Ready for Windows Server.net, Windows XP, Windows 2000 Server Transport Level directly called by the miniport and not by the user process. Simultaneous operation of iscsi and NDIS miniport operations Multiprocessor/Multithread implementation Use of Tagging and Queuing for better throughput Data Zero Copy Win 2000 miniport major features Support normal SCSI Miniport functions to find and Initialize iscsi Software/Hardware HBAs Support Plug and Play Support WMI subset for SCSI Miniports Support SRB full decode and Tagged Queueing Support Windows SRB IOCTLs Support special IOCTLs ISCSI Initiator and Target Topologies Multiple SCSI Targets behind iscsi Target Multiple LUNs per Target Multiple hosts per Target Multiple Sessions per NIC Multiple Connection per Session Multiple NIC per host
Conformant with the IETF draft 0.8 or latest available Login Logout Async events Authentication SLP service location protocol NDT Iscsi Naming and Discovery Requirements Certification UNH plugfest Stress Performace Test (i.e. IoMeter, ZDBenckmarks) Interoperability Interoperates with storage systems that respects the iscsi protocols (i.e. SN5420 Storage Router) Interoperates with selected Hard Drive, CD-ROM and RAID target devices Interoperates with System Storage management SW Applications (like a normal HD). System Procedures To complete this project, it is necessary to provide the installation/removal procedures and one configuration utility. Installation Procedure We provide an installation directory that must include all the required files. This directory can be stored into a CDROM or a FloppyDisk It is necessary to activate the installation using the SCSI Adapter entry point from the Main Control Panel. During the installation it is necessary to activate directly the installation utility in order to set any data required to configure the iscsi (i.e. target IP addresses). This action is necessary to set into the registry any information that must be used at the next boot to activate all disks provided by the network servers. Restart Windows 2000.
When the system is restarted, it is possible to run "Disk Administrator" to manage any new iscsi devices. With this utility you can set partitions, to format file-systems, etc. Removal Procedure In order to remove the iscsi driver you have to activate the SCSI Adapter driver control panel and select the remove procedure. Reboot the system. Configuration Procedure The iscsi Configuration Utility that we can provide in this first version will be able to configure the following: Target Host Names / IP Addresses 1. Add 2. Remove 3. Status 4. ReLogin 5. ReScan DisplayVersions All commands provided for Host name and Ip Addresses management are: Add Used to define the IP addresses of iscsi target nodes. Remove Used to remove already defined target node IP addresses. Status Used to display iscsi connection status to the target nodes and the target names at those nodes. ReLogin Used to cause reconnections to all of the target nodes without requiring a system reboot. This helps when the target node did not respond properly during login and the target devices were not seen correctly.
ReScan Used to cause the discovery to all of the target nodes without requiring a system reboot. This helps when the target node did not respond properly during login and the target devices were not seen correctly. We can develop the software to prepare the environment already set to host next steps: like showing a System Device Tree similar to Win2K DeviceManager GUI display; showing NICs and iscsi Soft/SNICs configured in the system as well as remote iscsi Targets that are in Full/Partial iscsi Login mode. Now we try to describe the main algorithm. To reach this target we present some graphics schema that includes all major phases.
BIPIPE Macro layout DPIPE User controller SNMP Dpipe Agent User space Kernel space I/O subsystem modules IOCTL Interface PORT SCSI SUB-SYSTEM Port Interface Ioctl Interface SNMP manager EndJob manager INIT manager SRB command manager Connect manager Timer Handler CallBackmanager TDI Client connector
BIPIPE Port Interface Port level DriverEntry BpipeHwInitialize BpipeResetBus Interrupt and Status not handled Init Internal Res. (timers, services) Nothing to do BpipeFindAdapter Prepare ConfigInfo and in this case the controller is always found BpipeStartIo BpipeIoDone DriverEntry init interface SRB IOCTL BIpipeSrbHandle r BIpipeIoctlHandle r To BIpipeInitManager To BIpipeSrbManager To BIpipeIoctlManager From BIpipeEndJob
BIPIPE Init Manager Reset Tables RemoteAddressCache ConnectionsTable (SCSI id Connection ID) MultiServer, MultiTarget, MultiLun Allocate buffers pool for any connection entry Get info from conf DB Get/Put data from/to Registry Activate internal Timers Call BipipeIntTimer Activate internal Threads Call BipipeServiceThread Init TDI Connector Call TCInit Exit
BIPIPE Service Thread Check Discovery Mode (get from Registry) AUTO SLP service location protocol MANUAL NDT Naming/Discovery Requirements Loop for any connection (Scan connection table) Conn Run Conn go DOWN Conn go UP Set SYS Device Tab (HD, CD, Remov, etc.) Upgrade Conn Table (Set new Down state) Upgrade registry Db (HD, CD, Remov, etc.) Conn LOGOUT Retry Login (Set new Down state) Upgrade Conn Table (Up, iscsi-id, etc.) Upgrade Conn Table (Set new Down state) ReSet SYS DeviceTab (HD, CD, Remov, etc.) Assign drive letter (Preset INQ tab is used) Execute Logout (SendLogout -TCDisconnect) Set SYS Device Tab (HD, CD, Remov, etc.) ReSet SYS DeviceTab (HD, CD, Remov, etc.)
BIPIPE Srb Manager Check Connection State (Get fron ConnTab) Connection NOT available Connection available Check Cmd Queue Check INQUIRY Cmd YES NO Exit(KO) Upgrade INQUIRY Table No pending Some pending Exit(OK) Check Tag Cmd Mode No Tag Exit(KO) Update Cmd Queue Prepare iscsi PDU Check Direction DATA IN Call TCEnqueInCmd DATA OUT Call TCEnqueOutCmd Exit Exit
BIPIPE Call Back Data Check for Errors Return Code Sense Data SCSI Error (Check, Parity, Busy,etc.) Exit(KO) Sense Key Check Response Type OP_NOOP_IN Handle Noop In Resp OP_SCSI_RSP Handle OutCmd Resp Build Miniport Reply (Reply, Sense, etc.) OP_SCSI_TASK_MGT_RSP Handle Management Resp OP_LOGIN_RSP Handle Login Resp Handle negotiation Update Connect Table (a1) (*) Exit -> BipipeIoDone and IoRequestDpc are required
(a1) OP_RTT_RSP Handle RxRtt Resp OP_SCSI_DATA_RSP Handle Tx DATA Resp Build Miniport Reply ( Reply, Sense, etc.) Copy Data to Port buffer OP_PING_RSP Handle Ping Resp Update Connect Table OP_ASYNC_EVENT Handle Async Resp Update Connect Table OP_RTT_RSP Handle RTT Resp OP_TEXT_RSP Handle TEXT Resp Exit Invalid Respose (*)Exit -> BipipeIoDone and IoRequestDpc are required
BIPIPE TDI Connector TCInit Open TCP protocol Prepare all call-back Set connection Table Prepare Packet Pool TdiBuildInternalDeviceControlIrp Main Call Back DataInEvent Exec Receive (TdiBuildReceive) Call BpipeCallBackData TDI TCP Async TdiCall TCConnect Open single Channel Set connection Table Execute connect TdiBuildConnect Build Address TdiBuildAssociatedAddress IoCallDriver ConnectEvent Update connection table Async TdiCall
TCDisconnect Update connection Table Execute Disconnect TdiBuildDisconnect IoCallDriver Main call back DiscEvent Update connection table (Conn Down) TDI TCP Async TdiCall TCEnqueOutCmd Prepare IRP Prepare Buffer Exec TCP Send TdiBuildSend IoCallDriver DataOutEvent Release resources (Irp, Buffer, etc.) Check reply code Update con table Async TdiCall TCEnqueInCmd Prepare IRP Exec TCP Send TdiBuildSend IoCallDriver TdiCall