SCSI The Protocol for all Storage Architectures

Transcription

1 SCSI The Protocol for all Storage Architectures David Deming, Solution Technology April 12, 2005

2 Abstract SCSI: The Protocol for All Storage Architectures This session will appeal to System Administrators, Storage Administrators, Storage Architects, and those that are seeking a fundamental understanding of SCSI Protocol and how it benefits your IT storage applications. The session will delve into the SCSI model, it s protocol, and how storage applications benefit from having a single high level protocol. The audience will receive the fundamental understanding of why SCSI is used as the storage industry s main language. SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 2

3 SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 3

4 Objectives Brief understanding of SCSI and its history Understanding of industry standards and the role of the SCSI Architecture Understanding the SCSI Architecture model Understanding the I/O Operation and the importance behind SCSI Protocol Learn how parallel SCSI, Fibre Channel, and iscsi rely on the SCSI Architecture to support storage applications SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 4

5 Brief History SCSI-1: In the beginning 1987 SCSI was adapted from the SASI Interface (1979) Specified a physical interface, transport protocol, and standard command for disk, tape, and other storage devices SCSI-2: Second coming of SCSI Speed and bus width enhancements Multi-threading commands and transport protocol cleanup Added more storage device type command sets SCSI-3: Today and beyond More speed: Ultra-2/Ultra-160/Ultra-320 Separation of Physical Interface, Transport Protocols, and the SCSI Command Sets SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 5

6 SCSI Today and the Future SCSI Language Every architecture uses the SCSI command sets Every architecture uses the SCSI upper layer protocol model SCSI is the Language of Love SCSI Transport Defines protocol mapping and transfer of SCSI language Fibre Channel Protocol (FCP), Serial Storage Architecture (SSA), Serial Bus Protocol (SBP), iscsi, Serial ATA (SATA), Serial Attached SCSI (SAS) SCSI Physical Architectures Serial Attached SCSI next generation SCSI physical interface Fibre Channel, SSA, 1394, ATAPI, TCP/IP SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 6

7 Introduction to Standards SCSI Standards and their impact on all storage interfaces

8 Standards Organizations International Organization for Standards (ISO) International Organization for Standards (ISO) American National Standards Institute (ANSI) American National Standards Institute (ANSI) IEEE Ethernet (802.x) IEEE-1394 (Firewire) IETF Internet Protocol iscsi FCIP ifcp INCITS T10 SCSI T10 SCSI T11 FC T11 FC T13 ATA T13 ATA SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 8

9 SCSI Standards Device Specific Command Sets Shared Command Set Architectural Model Transport Protocols Physical Interfaces SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 9

10 SCSI-3 Physical Interfaces Initiator I/O System SCSI-3 Class Driver (Disk Device Driver) SCSI-3 Class Driver (Tape Device Driver) Interface-Independent Software Programming Interface (Miniport Interface, CAM, ASPI, IO) SCSI-3 FCP Port Driver SCSI Parallel Port Driver iscsi IP Port Driver SAS Port Driver SCSI Serial Bus Protocol (SBP-2) Port Driver Fibre Channel Port SCSI Parallel Port Ethernet Port SAS Serial Port IEEE-1394 (Firewire) Port Fibre Channel Physical I/F SCSI BUS Fast/Wide/Ultra/etc. Ethernet Serial Attached SCSI Physical I/F FireWire Physical I/F SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 10

11 Protocol Service Model Application Layer Client I/O System SCSI SCSI Application Application SCSI Application Protocol Server I/O System SCSI SCSI Application Application Command Set Standards Protocol Service Interface Transport Protocol Layer SCSI SCSI Transport Transport Protocol Protocol Services Services SCSI Transport Protocol SCSI SCSI Transport Transport Protocol Protocol Services Services Transport Protocol Standards Interconnect Service Interface Interconnect Layer Interconnect Interconnect Services Services Interconnect Interconnect Services Services Interconnect Standards Application layer: Clients and servers that originate and process SCSI I/O operations by means of a SCSI application protocol Transport protocol layer: Services and protocols through which clients and servers communicate Interconnect layer: Services, signaling mechanism and interconnect subsystem needed for the physical transfer of data from sender to receiver. SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 11

12 SCSI Command Sets The most appealing features behind the SCSI Architecture, i.e. the SCSI language Includes every type of storage device manufactured today; disk, tape, CD, storage enclosures, etc Commands are manufacturer independent and are common for all types of devices SCSI device drivers and operating systems have utilized SCSI Command Sets since the late 80 s Can be used in any storage architecture including Fibre Channel, iscsi, InfiniBand, ATA/IDE, SATA (Serial ATA), SAS (Serial Attached SCSI), 1394 (Firewire) SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 12

13 SCSI Architectural Model Characteristics of the SCSI-3 Architecture

14 The SCSI Distributed Model SCSI is a client-server protocol. The client is called the Initiator (usually the OS I/O subsystem) and issues requests to the server. The server is called the Target (usually the SCSI controller that is part of a storage device) and receives, executes and returns Initiator requests and their associated responses. Client-Server Transaction Client Initiator Server Request Server Response Protocol Service Interface Service Delivery Subsystem Server Target SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 14

15 Client-Server Model A single Initiator can have multiple Application Clients. Targets have ONE Task Manager and one or more Logical Units (LU), which are numbered (LUN). The Task Manager: controls the sequencing of one or more tasks within a LU carries out the task management functions has the authority to modify Service Requests that have already been received by the target The Device Server processes operations and directs them to a specific LUN. Initiator Target Application Application Application Application Clients Task Management Request Task Management Response Device Server Request Device Server Response Task Manager Device Server Logical Units SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 15

16 Re-cap SCSI Terminology SCSI is a standard that defines an interface between an Initiator (usually a computer) and a Target (usually a storage device such as a hard disk, tape backup, or storage array). Interface refers to connectors, cables, electrical signals, optical signals and the command protocol that allow initiators and targets to communicate. Logical Units are a subset of Target devices which can allow for scalability. The Queue (or Task Set) is used to hold pending Commands (Tasks) that the Target will execute Initiator Target Logical Units Interface Queue Task Sets SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 16

17 Task Attributes There are four types of Task attributes that can affect how the Target executes each task (Command) Simple Target can execute in any order Target will typically apply a performance algorithm to numerous simple tasks Ordered Target must execute all ordered tasks in the order they are received Any task prior to ordered must be executed before ordered task Head of queue Informs Target to insert the task into the front of the queue Auto Contingent Allegiance (ACA) Used when the Target enters into an error condition for a command that has previously executed SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 17

18 I/O Operations Characteristics of SCSI I/O Operations

19 I/O Operation Model There are two categories of Protocol Services: - Execute Command and Confirmation Services - Data Transfer Services There are three main phases of an I/O operation that includes a data transfer: 1.Command: Send required command and parameters via a Command Descriptor Block (CDB) 2.Data: Transfer data in accordance with the command 3.Status: Receive confirmation of command execution SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 19

20 SCSI I/O Operations At a minimum, SCSI I/O Operations consist of: An Initiator issuing a SCSI Command A Target returning completion Status There is no Data transfer between Initiator and Target Types of Commands that do not move Data Test Unit Ready Start/Stop Unit Rewind Initiator SCSI Command SCSI Status Interconnecting Interconnecting Subsystem Subsystem Target SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 20

21 Operations that move Data When an Initiator and Target need to exchange information They utilize a Data phase Data In transmits information from the Target to the Initiator Data Out transmits information from the Initiator to the Target Data can be transmitted all at once or take numerous Data phases to complete information transfers Types of Commands Read or Write Inquiry Initiator SCSI Command Read or Write Data SCSI Status Interconnecting Interconnecting Subsystem Subsystem Target SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 21

22 Command Descriptor Block A Command is executed by sending a Command Descriptor Block (CDB) to the Target For each CDB The first byte of the CDB is the Operation Code The last byte of the CDB is the Control Byte The format of the Operation Code and Control Byte is identical for Example Six Byte CDB every SCSI Command in the SCSI Universe bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 byte 0 Operation code byte 1 Reserved (MSB) byte 2 Logical block address (if required) byte 3 (LSB) Transfer length (if required) byte 4 Parameter list length (if required) Allocation length (if required) How many bytes to transfer byte 5 Control byte Informs the Target where to start the transfer How many blocks (512 bytes) to transfer SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 22

23 Other CDB Formats CBDs can be: 10 bytes 12 bytes 16 bytes or even variable bytes in length Ten Byte CDB bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 byte 0 Operation code byte 1 Reserved Service Action (if required) byte 2 (MSB) byte 3 Logical block address (if required) byte 4 byte 5 (LSB) byte 6 Reserved byte 7 (MSB) Transfer length (if required) Parameter List Length (if required) byte 8 Allocation length (if required) (LSB) byte 9 Control byte Twelve Byte CDB bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 byte 0 Operation code byte 1 Reserved Service Action (if required) byte 2 (MSB) byte 3 Logical block address (if required) byte 4 byte 5 (LSB) byte 6 (MSB) byte 7 Transfer length (if required) byte 8 Parameter list length (if required) byte 9 Allocation length (if required) (LSB) byte 10 Reserved byte 11 Control byte SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 23

24 SCSI Commands all Devices Standard SCSI Commands: Disk Tape Storage Enclosures Disk Arrays CD WORM Media Changers Every device imaginable or not yet created Op. Code Command Name Type 00h TEST UNIT READY M 03h REQUEST SENSE Z 12h INQUIRY M 15h MODE SELECT(6) Z 18h COPY O 1Ah MODE SENSE(6) Z 1Ch RECEIVE DIAGNOSTIC RESULTS O 1Dh SEND DIAGNOSTIC O 39h COMPARE O 3Ah COPY AND VERIFY O 3Bh WRITE BUFFER Z 3Ch READ BUFFER O 4Ch LOG SELECT O 4Dh LOG SENSE O 55h MODE SELECT(10) Z 5Ah MODE SENSE(10) Z SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 24

25 SCSI Status Any time a SCSI Command is sent to a Target The Initiator expects a completion Status This status can reflect successful or unsuccessful completion of the command The Status may indicate Busy or Not Ready Error condition exists for another command Targets task set is full SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 25

26 Parallel SCSI

27 Parallel SCSI Characteristics Information can only go in one direction at a time Each piece (byte) of information is acknowledged Information transfers are interlocked A connection protocol is used before any information transfers Arbitration, Selection, and Message phases Uses SCSI ID s to identify devices Utilizes protocol bus phases to accomplish information transfers Command Phase to deliver the CDB Data Phase to deliver customer data Status Phase to deliver completion status Can multi-task by disconnecting SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 27

28 Read Command Example Initiator connects to Target Once Initiator connects to Target, the Target is in control of the I/O Process Initiator sends CDB information via Command Phases Target returns requested information via Data Phase Target returns completion information via Status Phase Target Disconnects via Message Phase Initiator Connection Arbitrate and Select CDB 0x A Command Phase Requested Read Information Data Phase Requested Read Information Data Phase Completion Information 0x00 Status Phase Disconnection 0x00 Message Phase Target SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 28

29 Fibre Channel

30 Fibre Channel Protocol Characteristics Fibre Channel creates frames to deliver SCSI Commands, Data, and Status information units (IU). FCP Frames include: FCP_CMD frame Equivalent to parallel SCSI Command Phase FCP_XFER_RDY frame No actual parallel SCSI Equivalent FCP_DATA frame Equivalent to parallel SCSI Data Phase FCP_RSP frame Equivalent to parallel SCSI Status Phase SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 30

31 Fibre Channel Frame Fill Word Fill Word Transmission Words Start-of-Frame Frame Header Optional Header Optional Header Data Field Data Field Payload (Information being transported) FCP_CMD IU FCP_Data IU FCP_XFER_RDY IU FCP_RESP IU Fill Bytes CRC End-of-Frame Fill Word Fill Word Fill Word Frame type and content/function Class-specific control information Protocol Type in this frame Sequence this frame belongs to Originator Exchange ID Multi-purpose parameter field R_CTL Destination Address (D_ID) CS_CTL Source Address (S_ID) TYPE Frame Control (F_CTL) SEQ_ID DF_CTL SEQ_CNT OX_ID RX_ID Parameter Field (PARM) Where frame is being sent to Where the frame came from Frame Control field Sequential count of frames Responder Exchange ID SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 31

32 Write Command Example Before devices can communicate in FC they must establish a session via login protocol Originator sends Write command to responder Responder replies with a ready to receive response Originator sends Data frames When all Data is transferred, Responder sends Status to Originator Session could end at this point but typically doesn t Originator Establish Session (outside scope of this presentation) FCP_CMD Frame (Write) Command Phase FCP_XFER_RDY Frame (Responder is ready for Data) FCP_DATA Frame Data Phase FCP_DATA Frame Data Phase FCP_RSP Frame Status Phase Remove Session (outside scope of this presentation) Responder SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 32 Exchange Sequence

33 iscsi IP Network Gateway FC SAN

34 iscsi Characteristics iscsi is a SCSI transport protocol for mapping of block oriented storage data over TCP/IP networks. Storage generally identified for this application include disk, tape arrays and tape libraries. IP networks most applicable for this purpose are Gigabit Ethernet and in the future 10 Gigabit Ethernet, however for low performance applications 10/100bT will work. The iscsi layer encapsulates the SCSI CDB into a iscsi Protocol Data Unit (PDU) and forwards it to the Transmission Control Protocol (TCP) layer. The communications between the Initiator and Target will occur over one or more TCP connections. The TCP connections form a session and will carry the iscsi PDU s. The sessions are given an ID called a Connection ID (CID). There are two parts of the ID, Initiator Session ID (ISID) and Target ID (TSID) and together make up an I_T nexus. SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 34

35 iscsi Encapsulation Initiator Target SCSI SCSI Application Application SCSI Application Protocol SCSI SCSI Application Application Data Data Data Protocol Service Interface iscsi Transport iscsi Transport Protocol Services Protocol Services iscsi Transport Protocol iscsi Transport iscsi Transport Protocol Services Protocol Services PDU Header PDU Data Interconnect Service Interface Interconnect Interconnect Services Services TCP TCP Interconnect Interconnect Services Services TCP TCP TCP Header TCP Segment CHK IP IP Data Link Data Link IP IP Data Link Data Link Ethernet Header IP Header IP Packet (Datagrams) Ethernet Frame FCS IP Network SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 35

36 Write Command Example Before devices can communicate in iscsi they must establish a session via login protocol Initiator sends Write command to Target Target replies with a ready to transmit R2T response Initiator sends Data frames When all Data is transferred, Target sends Status to Initiator Session could end at this point but typically doesn t Initiator Establish Session (outside scope of this presentation) Command PDU (Write) Command Phase R2T PDU (Target is ready for Data) Data-Out PDU Data Phase Data-Out PDU Data Phase Response PDU Status Phase Remove Session (outside scope of this presentation) Target Sequence SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 36

37 Read Command Example Before devices can communicate in iscsi they must establish a session via login protocol Initiator Establish Session (outside scope of this presentation) Command PDU (Read) Command Phase Target Initiator sends Read command to Target Target sends Data frames When all Data is transferred, Target sends Status to Initiator Session could end at this point but typically doesn t Sequence Data-In PDU Data Phase Data-In PDU Data Phase Response PDU Status Phase Remove Session (outside scope of this presentation) SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 37

38 Summary SCSI is the language of LOVE Every storage architecture utilizes the SCSI model SCSI Commands sets are available for every type of storage device even including host-to-host An I/O Process is interface independent and is made up of: Command Phase Optional Data Phase and a Status Phase SCSI language is here to stay and will be used in every storage architecture including: SATA SAS For more information on standards see: incits.org, t10.org, t11.org, t13.org, ietf.org, scsita.org, fibrechannel.org SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 38

39 Q&A / Feedback Please send any questions or comments on this presentation to SNIA: track-storage@snia.org Many thanks to the following individuals for their contributions to this tutorial. SNIA Education Committee David Deming Elaine Silber SCSI: The Protocol for all Storage Architectures, April 2005 Storage Networking Industry Association. All Rights Reserved. 39