Lowering Cloud & Data Center TCO with SAS Based Storage. Seagate Mark E. Wojtasiak. Xyratex Michael K. Connolly Michael A. Hoard Linda Tillis, P.E.

Transcription

1 Lowering Cloud & Data Center TCO with SAS Based Storage Seagate Mark E. Wojtasiak Xyratex Michael K. Connolly Michael A. Hoard Linda Tillis, P.E.

2 Notices The information in this document is subject to change without notice. While every effort has been made to ensure that all information in this document is accurate, the document is provided as-is and Xyratex accepts no liability for any errors that may arise. Reliance on this document and its content is at the sole risk of the viewer or reader Xyratex (the trading name of Xyratex Technology Limited). Registered Office: Langstone Road, Havant, Hampshire, PO9 1SA, England. Registered number No part of this document may be transmitted or copied in any form, or by any means, for any purpose, without the written permission of Xyratex. Xyratex is a trademark of Xyratex Technology Limited. All other brand and product names are registered marks of their respective proprietors. For more information please contact marketing@xyratex.com or visit Issue 1.0 September 2013

3 Introduction 1 In Seagate s and Xyratex s last paper Achieving Rapid Scale in Enterprise and Cloud Data Centers with SAS it was discussed how: 1 Storage requirements are continuing to grow exponentially with no slowdown in sight. 2 Desktop drive designs do not accommodate data center and multi-spindle environments with sustained performance, scalability, and reliability as forefront drivers. 3 Not all data centers and infrastructures are necessarily the same and the related performance ability of any environment must match the requirements necessary for critical applications. 4 Organizations expecting to purchase or lease reliable sustained storage performance at scale are advised to seek out and demand the better-sustained match of high-performance enterprise drives (nearline SAS) to the needs. While it is important to discuss how Serial AT Attachment (SATA) drives are not inherently designed to provide the best option for enterprise class workloads, it is equally important to understand why Serial Attached SCSI (SAS) provides a better high performance storage solution today and represents a critical enabler to lowering data center total cost of ownership (TCO) over the next decade. The case for SAS in the enterprise becomes evident when assessing the emerging performance and technology directions influencing next generation enterprise data centers and cloud deployments, costs, and operational power and cooling demands. Historical Perspective Consider the historical evolution of enterprise-class storage devices over the years. Indeed, in the early days, storage related decisions were simpler for organizations and related IT departments. If the desire was for enterprise class disk drives for servers or storage arrays, there was only one choice: a 3.5-inch (3.5 ) 10,000- RPM SCSI drive. As servers and storage evolved, drive performance became critical, which resulted in the development and introduction of 15,000-RPM drives. In addition to the need for performance, the overall power requirements within datacenter and equipment asset(s) was a large concern. This provided the platform for the development of the 2.5-inch (2.5 ) drive. Moving to a smaller form factor, 2.5 drives enabled more efficient power consumption and greater input-output (I/O) throughput performance per drive, enabling enterprise applications requiring high transaction rates and best available input-output per second (IOPS) capability. Overall, the transition to the 2.5 form factor served the specific power efficiency and high performance needs, but did not allow for the same capacity growth equivalent of 3.5 drives. This caused a distinct separation between enterprise high performance drives, which continued to focus on the 2.5 form factor featuring higher I/O rates operating at either 10,000-RPM or 15,000-RPM. These drives catered to high-end enterprise applications requiring Figure 1: Interface Evolution (Source: SCSI Trade Association) the full set of system robustness attributes, including numerous data integrity features, multi-spindle robustness characteristics, and improved power efficiency. However, with strict focus on high performance and power consumption, the tradeoff was a lower data storage capacity per drive. The continuing industry wide explosion of data creation and volume created an opportunity leading to the ultimate development of more than one drive class focused on enterprise applications requiring the highest

4 2 attainable data storage capacity with moderate performance and varying degrees in system robustness attributes. This led to the introduction of the 7,200-RPM enterprise drives, or the nearline drive class. Today, nearline enterprise drives make up a majority of drive usage and capacity demand in the enterprise space. Indeed, high capacity with moderate performance per drive for enterprise applications are primary characterizations of nearline drives. Further separation within the nearline drive class relates to the levels of system robustness attributes detailed in the table below, including data integrity features and suitability for multi-spindle integration. Drive Type Interface Interface Speed per port Form Factor(s) Pricing Capacity Performance Data Integrity Rotational Vibration Tolerance Multi-spindle Bit Error Rate Annual Failure Rate Desktop Drive SATA (Single Port) 3 & 6 Gb/s 3.5 $ High Low 7,200-RPM N/A Low Not suitable 1 sector per 10E14 High* Enterprise Value (Nearline Lite) SATA (Single Port) 3 & 6 Gb/s 3.5 $$ High Low to Moderate 7,200-RPM Minimal Moderate Minimal suitability 1 sector per 10E15 Moderate Enterprise Capacity (Nearline) SAS (Dual Port) 3 & 6 Gb/s 2.5 & 3.5 $$$ High Very Good to Moderate 7,200-RPM Full High Excellent suitability 1 sector per 10E15 Low Enterprise Performance SAS (Dual Port) 3 & 6 Gb/s 2.5 $$$$ Moderate High 10,000-RPM 15,000-RPM Full Highest Excellent suitability 1 sector per 10E16 Lowest * Based on a general enterprise workload The decision of where to utilize one drive type over another is not abundantly or immediately clear, since all types share the same interface speeds, form factors, and are plug-in compatible with similar connector assemblies. The primary purpose of this paper is providing perspectives on future trends where the separation between value and usage widens the distinction between drive types. Future system architects, organizations, and end users will need to pay close attention to these developments. Dual Port SAS based Enterprise Capacity Nearline is closely associated with the higher end of the enterprise spectrum, featuring highest available capacity, very good to moderate performance per drive and the highest levels in system robustness attributes. This combination of attributes supports excellent multi-drive suitability built-within the core of the system, starting with the drives themselves. Using best in class system design and integration methods, end users benefit from enhanced internal system quality, high performance and excellent system availability, resulting in significantly less hardware to satisfy requirements, lower power consumption, lower hardware failure rates and lower operational costs. Market and End User Requirements There are many catalysts to the widespread use of nearline, but probably none more than Cloud Computing more specifically Cloud Storage. The cloud ushers in a new era of highly scalable data centers. Indeed, the emergence of sophisticated virtualization capabilities drove a complete rethinking, and redesign, of large-scale data center infrastructures as well as computing and storage. These factors drove the inception of what the industry now knows as Anything and Everything as a Service deployments. These new philosophies abstract the user experience and application from the underlying hardware with the goal of reducing acquisition, design, deployment, and operational costs. While the premise of using hardware with lowest possible cost has worked well in the short term, will this strategy continue to make sense long term? One area where SATA nearline lite does make sense relates to the industry trend known as Cold Archival Storage, which relates to the storage of older data, still requiring accessibility, in less expensive and as needed platforms. In a cold storage framework, infrequent or rarely accessed data resides in less expensive, archival storage assets required to store and/or retrieve the data. These assets receive power and attention only when needed based on user-required access to the data. How does this relate to storage requirements and disk drive capabilities? Overall, this sounds remarkably similar to the present day single I/O requests inherent with SATA, which already allows for significantly more storage capacity with the least overhead infrastructure. The net result, there is no need to improve the SATA interface as what is available today is sufficient for future needs. Even though the ability to store and utilize significantly more data, with less infrastructure, is a driving engine of change in the cloud industry, there are other factors to consider in gaining maximum productivity for future needs. In the very near future, when the underlying hardware components deliver higher levels of multicore processing capabilities within a single chip, the demands for increased storage performance becomes more critical as indicated by the industry and the community.

5 Storage Performance Considerations 3 What is regarded today as supercomputing will ultimately be common in everyday computing. Why is this the case? Consider how multi-core processing, networks, and memory technologies all share a similar characteristic: lanes. Multi-core development increases independent threads, or processing lanes, while network and memory bandwidth expands communication lanes, or channels. The natural tendency is increasing, geometrically, independent initiator threads and communication lanes as these areas grow together in unison. These mutually synergistic technologies become more important when considering how to support next generation virtualization platforms. In order to gain high levels of productivity in the future, the requirement for consistent underlying and built-in hypermultilane capabilities, suited for scale, is a crucial facet. To meet the demands of cloud data centers from a TCO and workload perspective, cloud service providers ( CSP s) and data center managers require cost effective upgrades to technology much faster and more often. With SATA reaching its apogee at 6Gb/s, the architecture is at its maximum signaling rate. Example System Integration Techniques An example design method, applicable to either SAS nearline or SATA nearline Lite based storage systems, is extensive use of middleware and virtualization to abstract lower level hardware errors from the user level experience. Even though, this provides required levels of performance and availability measured at the storage system level, it does not decrease the incidence of hardware failures. On the contrary, end users may enjoy what appears to be transparent use of the system with little or no impact, in the presence of numerous ongoing failover events. This is where the user needs to be careful since ongoing failover events are not representative of a no impact scenario. Indeed, the replacement of failed drives raises operational costs. Depending on the system design, drive rebuild events cause performance slow down and increased vulnerability to application outage or data loss. In the end, users pay for the full system and all operational costs, since there is no abstracting net-realized flow of expenses to the purchaser. Selecting drives with lower incidence of failure helps reduce operational costs. Selecting drives with higher incidence of failure may appear to have no impact due to the system design, but this increases operational costs. Application of state of the art system integration techniques may enable acceptable results for SATA nearline lite based systems, whereas these same techniques result in vastly superior overall gains when applied to SAS nearline based systems. The point being storage systems embody many design trades. The choices of disk drive type and system design methods represent key factors that end users should take into account in their deployment decisions. With SAS, the technology roadmap is more robust with 12Gb/s and 24Gb/s SAS signaling rates on the horizon. For data center infrastructures requiring increased throughput, SAS offers the flexibility to switch out 6Gb/s SAS drives with the latest 12Gb/s SAS drives at a lower technology refresh cost than afforded by SATA. When considering the current, and ongoing, drive industry investment around SAS innovation, there is really no comparison with SATA. SAS comes with more features than available with SATA now, and the SAS roadmap indicates higher signaling rates positioned to yield more pronounced throughput and robustness gains coupled with broader high performance feature-set additions over the course of the next 10 years. Single Port SATA based Enterprise Value Nearline Lite is focused on high capacity and affordability, featuring the highest available capacity, lowest 3.5-inch hard drive power consumption, low to moderate performance, and a minimized set of enterpriseclass features in comparison to traditional Nearline SAS hard drives, which makes it more suitable to cloud based infrastructures. This combination of attributes may not inherently support all enterprise workloads; therefore, the application of other system design and integration practices is necessary to compensate and work around any limitations. Using adequate work around techniques, end users will benefit from favorable initial system affordability and power savings, with acceptable system performance; however, these systems will inherently require more hardware or software to satisfy more robust enterprise application requirements resulting in higher maintenance actions and operational cost. Figure 2: SAS Roadmap (Source: SCSI Trade Association 2013)

6 4 According to the SCSI Trade Association (STA) presentation SAS Standards and Technology Update 1 the STA commits to preserving existing SAS architectures as well as 3Gb/s SAS & 6Gb/s SAS usage models while maintaining backward compatibility with current 3Gb/s & 6Gb/s SATA via SAS backplane device connectors. In addition, the STA stresses focus on 6Gb/s SAS and continued SAS compatibility encouraging improved storage system reliability, availability, and serviceability ( RAS ) attributes, doubling transfer rates, improving cost-toperformance and powerto-bandwidth ratios, and reducing the number of per Gb/s connections. This focus aids in maximizing link utilization when using devices operating at less than 12Gb/s. All of this, in conjunction with maintaining and supporting the SAS Advanced Connectivity roadmap, promotes broad adoption of Mini-SAS HD and supporting MultiLink SAS implementations. According to another STA presentation 2, by Marty Figure 2: Seagate Step up to SAS Czekalski, Senior Staff Program Manager at Seagate Technology and President of STA, the performance benefits of SAS theoretically double with each improvement in interface speed. For example, with 6Gb/s SAS in full duplex, four (4) ports could, in essence, deliver 48Gb/s of available bandwidth. The beauty of this architecture is that 12Gb/s SAS is, on the short-term horizon, capable of delivering 96Gb/s per every four (4) port duplex configuration resulting in even higher I/O per second (IOPS) and lower latency storage. The SAS feature-set, depicted in Figure 3 and created by Seagate Technology, is the step up to SAS pyramid. As indicated, SAS has enterprise features, which are impractical on a SATA drive including: Full Duplex I/O Full duplex link capability supports concurrent writes and reads. Two concurrent data channels SAS drives allow execution of two data commands simultaneously. Read/Read or Write/Write or Read/Write using both ports or Read/Write on one port. Enterprise Command Queuing The command queue supports up to 128 commands from up to 16 separate hosts/initiators. Problems with the execution of any one command will not affect any other command in the queue. End-to-end data integrity Supports data integrity checks in tandem with the host. The host appends an 8-byte check field to the data it sends to the drive. The drive checks the data it receives against the check field and signals the host if an error condition exists. This check field writes to medium with the data and returns to the host for validation on a Read. Full SCSI command set SAS drives support the same comprehensive command set and inherently compatible with available and future SAS controllers ensuring compatibility among disk drive vendors for years to come. Variable sector size from 512 bytes to 4160 bytes in 8-byte increments. This has beneficial impact on advanced protocol, security, and data integrity. 1 SAS Standards and Technology Update, SAS & SATA Change the Face of Enterprise Storage,

7 Full IOECC Error Correction for all drive electronic errors (silicon failures) is available on both Read and Write operations. 5 Dual port - SAS provides dual path capability without the need for a multiplexer or SAS bridge. Signal Integrity - SAS provides more robust signalling Cost - SAS based RAID controllers are the same cost as SATA What about using a SAS to SATA bridge to take advantage of the SAS controller or backplane, but utilizing SATA drives for the extra cost savings? The use of SAS-SATA bridge does not provide an exact implementation of native SAS is many areas: Aborting a task will have a much greater impact on the outstanding commands from different initiators. Non-512B LBA size and use of PI will have a negative performance impact. Limited queue depth not improved (split across two (2) paths.) Still half duplex communications Enterprise SAS versus SATA drive cost differential is less than cost of SAS to SATA bridge card System Customization When customizing server and storage systems in a cloud data center, SATA based architecture, as mentioned previously has limitations. With SATA, data center and cloud service architects are limited to lower performance spin-speeds: 7,200-RPM nearline, as well as 5,900 and 5,400-RPM enterprise hard drives. With a SAS based architecture, cloud architects can leverage single system architecture into multiple server and storage configurations based on workload demand simply by changing out the hard disk drives (HDDs). Enterprise-class HDD SAS SATA 5,400-RPM No Yes 5,900-RPM No Yes 7,200-RPM Yes Yes 10,000-RPM Yes No 15,000-RPM Yes No Further, a SAS-based server and storage architecture allows for optimal customization, especially in consideration of application requirements and changes in terms of workload or performance IOPS. With a SAS based design, a cloud architect can switch out 7,200-RPM drives for 10,000 or 15,000-RPM drives based on the needs of the application. This flexibility allows data centers and service providers to customize standardized system designs for multiple use cases and is only possible through the utilization of an all SAS design. System Integration The Bigger Picture The existing quandary for CSPs and enterprise organizations is providing high performance, availability, reliability, and interoperability to their clients and infrastructure, but maintaining a low TCO. Indeed, availability, agility, and low cost are crucial components for CSPs in order to compete in the marketplace. Many CSP and organizations, that can no longer effectively integrate traditional enterprise IT vendor storage solutions, now incorporate a build their own structure opting, in many cases, to use SATA drives with higher capacities and lower cost per drive metrics to meet their TCO goals. However, this approach does not take the significant need for higher availability, reliability, flexibility, scalability, and drive longevity into consideration, especially within larger CSPs maintaining critical always on real time performance and service level agreements (SLAs) to their customers. SAS nearline drives offer CSPs and enterprise infrastructures benefits not capable with SATA nearline lite and lower cost drives. The inherent multi-port capability of SAS drives allows a redundant, fail-over path for

8 6 each drive and SAS based storage systems support active-active frameworks. In addition, the design of SAS drives takes heavy loads and use into consideration with Mean Time Between Failure ( MTBF ) ratings over 1 million hours and warranties between 3 to 5 years. These factors alone, over SATA MTBFs and warranties, allow for a better availability and TCO in the long term. In terms of flexibility and interoperability, SAS offers the capability to utilize SATA drives as well as SAS drives in the same environment. While SATA has its place in the cloud and the enterprise, the SAS benefits of backward compatibility with the SCSI command set and interoperability as well as SATA bring to bear higher levels of cost savings and adaptability. Further, SAS leverages the continuing roadmap evolution with increased performance. Conclusion - Key Takeaways SAS is the mainstream server storage interface of choice, assuring investment protection by guaranteeing at least two generations of accessibility, said Chris Lyon, Executive Director, SCSI Trade Association. Companies such as Xyratex and Seagate are enabling the robust capabilities of SAS with new storage enclosures and application storage servers that continue to propel the market ahead as well as extending SAS architecture deeper into enterprise storage. Seagate s and Xyratex s first white paper focused on how SAS can help achieve rapid scale in enterprise and cloud data centers. This second paper focuses on how a SAS based storage architecture can potentially lower TCO in both the near-term and, but more importantly, the long-term as it relates to system upgrades, customization, and integration. In order to meet data center demands from a TCO and workload perspective, data center managers require cost effective upgrades to technology much faster and more often; best achieved deploying all SAS architecture. The flexibility of SAS based storage allows data center operators to customize standardized system designs for multiple use cases thereby extending the life of the system and thus, lowering TCO. When designing a data center, system architects and end users need to pay close attention to the development of the SAS interface and how it best addresses availability, reliability, flexibility, scalability, and longevity requirements; especially within large CSPs maintaining critical always on real time performance and SLAs with their customers. Stay connected to Xyratex and Seagate for future white papers focused on designing, building, deploying and operating scalable storage architectures for cloud, high performance computing and big data environments. Xyratex & Seagate Partnership Xyratex and Seagate are recognized leaders in high quality, enterprise-class data storage technologies. Xyratex s research and development expertise in storage and SAS technology creates solutions with unmatched price-to-performance and TCO benefits for its OEM clients. Xyratex solutions span storage enclosures and application storage servers with its OneStor family of products as well as integrated, high performance, scale out solutions with its high-performance ClusterStor product line; specifically suited for data intensive applications in the research, energy, defense, and life sciences markets. Xyratex s rigorously tested storage platforms and solutions provide increased levels of reliability, availability, and serviceability (RAS) as well as TCO not available with other manufacturers. Further, this experience and patented testing methodologies allow Xyratex to satisfy the requirements of Cloud Service Providers and enterprise datacenters for superior and robust storage infrastructure, which is critical to sustain successful SLAs and meet application uptime guarantees. Learn more at Seagate contends that as storage capacity continues to shift to cloud architectures, data centers must evolve to deliver the highest quality of service at the lowest possible total cost of ownership (TCO). Seagate has the industry s broadest selection of enterprise storage devices for cloud data centers. From high-performance solid state drives to high-capacity hard disk drives, choosing the right storage device or mix of storage devices can have a direct effect on data center TCO. Seagate and our partners help data center customers choose the right storage device or mix of storage devices to best meet their requirements through Seagate s Cloud Builder Alliance. The Cloud Builder Alliance focuses on strategic relationships with leading edge partners building the next-generation cloud using Seagate technology, devices, features, and services. Learn more at seagate.com/solutions/cloud/.

9 1 United Kingdom Headquarters Langstone Road Havant Hampshire PO9 1SA United Kingdom T: +44(0) F: +44(0) Principal US Office Lakeview Blvd. Fremont CA USA T: F: Xyratex (The trading name of Xyratex Technology Limited). Registered in England & Wales. Company no: Registered Office: Langstone Road, Havant, Hampshire PO9 1SA, England. The information given in this brochure is for marketing purposes and is not intended to be a specification nor to provide the basis for a warranty. The products and their details are subject to change. For a detailed specification or if you need to meet a specific requirement please contact Xyratex. No part of this document may be transmitted, copied, or used in any form, or by any means, for any purpose (other than for the purpose for which it was disclosed), without the written permission of Xyratex Technology Limited.