Flash Storage Gets Priority with Emulex ExpressLane Quality of service (QoS) feature ensures highest I/O performance for mission-critical applications Emulex Host Bus Adapters
Abstract Flash storage is often deployed for mission-critical applications where data access latency needs to be the absolute lowest. However, Storage Area Network (SAN) traffic to other, slower devices can result in congestion points, defeating the purpose of flash deployments and losing the value of flash (and decreasing return on investment (ROI)) due to network traffic congestion. Moreover, heavy traffic often creates inconsistent response time (i.e., jitter) that can cause problems for certain applications (e.g., high definition video). This whitepaper explores the new storage tiering model (flash and non-flash) that creates congestion potential due to mixed traffic and introduces a quality of service (Q0S) solution based on Gen 5 Fibre Channel (FC). Table of Contents Introduction...3 SSD history and future...3 SSD deployment considerations...4 SSD latency due to congestion on the SAN...4 Emulex ExpressLane...5 ExpressLane enablement...5 Conclusion.......................................................................................................................................8 2 EMULEX WHITE PAPER Flash Storage Gets Priority with Emulex ExpressLane
Introduction Application performance in today s data center is affected by three areas: bandwidth, throughput and latency. As shown in Figure 1, missioncritical applications span a variety of industries and technologies, and have different requirements across these three areas. For example, video applications require high bandwidth, while databases require high throughput and high frequency traders require low latency. Solid state drives (SSDs) are the latest in tiered storage technology and are used to increase application performance by removing storage I/O bottlenecks. Figure 1. Application performance requirements. Emulex delivers reliable and high performance connectivity to enable optimum throughput, latency and bandwidth for today s most demanding data centers. With features aimed at enhancing storage tiering strategies, the Emulex LightPulse Gen 5 FC Host Bus Adapter (HBA), coupled with Emulex OneCommand Manager for centralized management of all Emulex adapters in the tiered storage environment, is critical to any I/O solution. Adding a new storage tiering feature, ExpressLane, to its base Gen 5 FC HBA, Emulex continues to build on the value of FC in a company s infrastructure with flash technology services. ExpressLane is available as an included feature on every Emulex Gen 5 FC HBA and creates a priority queue for an application where frames from the application are sent out sooner and thus do not get caught in host congestion, allowing high performance flash storage to be fully utilized. SSD history and future SSD or flash storage, called tier 0, has been around for many years. Early adopters were data centers with I/O-intensive applications, such as Oracle databases, credit card processing and stock trader applications. As more and more companies realize that storage based on mechanical disk is becoming antiquated (i.e., while CPU (and hence servers and networks) get denser, cheaper and faster, disk only gets denser, cheaper and slower), performance-hungry data center managers look for faster access to storage. In addition, in the past, many companies were hesitant to run applications in the cloud because of the latency between virtual servers and storage. SSDs address this concern with fast access to stored data, making flash a key storage trend for not only cloud applications, but also clustered applications. Today, flash storage is regarded as a mainstream technology, accessible to more businesses than ever before. Gartner has forecast that an all-flash storage market (i.e., storage with no spinning disk at all) is going to reach $4 billion by 2015. 1 In fact, this past year showed a lot of activity on the vendor side with more flash storage product activity, stock market fluctuations, acquisitions and legal wrangling. 1 Innovation Insights, All-Flash Storage Not Just Next -- It s Now, February 4, 2013 3 EMULEX WHITE PAPER Flash Storage Gets Priority with Emulex ExpressLane
SSD deployment considerations Advanced SSD technology makes enhanced tiered storage a reality for improving mission-critical application performance. This technology allows a company to better align its storage needs with its business needs and is often attached to a SAN. These solutions automate the movement and placement of data across storage resources as needs change over time, enabling continuous optimization of applications by eliminating trade-offs between capacity and performance, while lowering cost and delivering higher service levels at the same time. The Storage Networking Industry Association (SNIA) reports that the hardest part of implementing a tiered storage solution is categorizing the data 2 which may be based on levels of protection needed, performance requirements, frequency of use and other considerations. Flash storage arrays attached to a SAN often use Gen 5 FC HBAs because the superior performance means faster storage access. Typical Gen 5 FC solutions will deliver 1.2 million I/O operations per second (IOPS), 3 giving twice the data throughput as former 8GFC solutions and cutting response time (application latency) in half. Gen 5 FC also increases virtual machine (VM) density per server and enables missioncritical applications to be virtualized. With higher availability for reliable storage tiering, Gen 5 FC deployments are essential to ensure no downtime for critical applications. Using HBAs from a single vendor (e.g, Emulex Gen 5 FC) across the data center also streamlines HBA management for reduced costs and enterprise scalability. SSD latency due to congestion on the SAN While SSDs solve a storage performance issue, deployment can create an unforeseen performance issue for those critical applications that rely on fast access to the data. With the wider deployment of SSDs, mixed workloads are now residing on the same SAN. As flash storage gets deployed into mixed environments and among hybrid storage arrays, the combination of data going to rotating media and flash devices causes congestion on the SAN. Figure 2 shows the following scenario of two competing VMs trying to access data: 1. VM1 on server 1 requests low priority I/O from disk array 1 2. VM2 requests high priority I/O from disk flash array 3. Congestion occurs at the HBA where VM2 traffic is stuck behind slower traffic Figure 2. Congestion point at an HBA due to competing VMs requesting data. 2 Computer Weekly, Tiered storage strategies and best practices, March, 2010 3 Demartek, Emulex LPe16000B 16Gb Fibre Channel HBA Evaluation, October, 2012 4 EMULEX WHITE PAPER Flash Storage Gets Priority with Emulex ExpressLane
Because the host s normal queuing mechanism assigns priority to I/O in the order received, and sends requests one at a time, high priority applications wait for their turn behind non-crucial jobs. Furthermore, when the bandwidth is full at peak times, the VMs that have access to fast storage Logical Unit Numbers (LUNs) are blocked in host congestion which can be the difference in completing some high revenue processes or not. This also leaves the flash storage array underutilized, reducing ROI. I/O QoS is critical across the data center, but especially so for cloud and software-defined storage since they often have Service Level Agreements (SLAs) and thus require flexibility in priority and performance for all their users. Additionally, East-West storage traffic that uses acceleration, clustering and high availability for virtualization/cloud and databases relies on caching and tiering in both servers and arrays/ appliances and is affected by improperly balanced QoS, having a direct impact on business revenues (e.g., trading firms). Emulex ExpressLane Recognizing that tired storage strategies are impacted in mixed workloads connected to a SAN, Emulex developed ExpressLane which enables high priority (mission-critical) flash workloads to be tagged as high priority and moved in front of less time-sensitive workloads and traffic, thereby lowering latency, reducing jitter and increasing customer satisfaction. Referring back to Figure 2, by providing prioritized queuing in the driver and ASIC so that the traffic for the prioritized LUN moves to the front of the queue, ExpressLane ensures the full benefit of SSDs or high performance storage arrays. ExpressLane can be enabled to provide prioritized paths on both flash storage arrays and traditional spinning media arrays. In fact, the prioritized queuing path results in individual I/Os having consistent performance, that is, there are no performance spikes or jitter. ExpressLane enables higher QoS in the following ways: n Insurance in critical time periods Every data center performs differently and experiences different fluctuations of peak I/O performance. If a critical time period for I/O lines up with a peak performance period, crucial information flow can be impacted adversely. The flash storage meant to speed up application processing becomes underutilized while waiting for the SAN fabric to catch up. ExpressLane ensures the right information flow by giving priority in peak conditions. Since ExpressLane is managed by OneCommand Manager, it can be implemented immediately by an administrator with a few clicks. n Large throughput clog on the host machine When data output from essential applications is significantly delayed, the business suffers. Situations where this becomes a problem include high definition video, data warehousing and backup operations. All of these situations can use up the full network pipe and take a relatively long time to finish. During these long operations, other applications on the host are impacted regardless of their importance. Critical delays can ruin effective utilization of high performance fabrics. Having the fastest storage will not change anything if the pipe being used is preventing the application from using the ultra-fast flash storage. ExpressLane fixes this problem by allowing the administrator to elevate the flash LUN s priority to favor the high performance application, thereby achieving the high performance benefit of flash storage during congestion periods. ExpressLane enablement ExpressLane is a no-cost feature that is only available on Emulex LightPulse Gen 5 (LPe16xxx) FC adapters. Using OneCommand Manager, an administrator enables the ExpressLane feature by choosing a LUN to prioritize. The LUN can be any storage array connected to the SAN. This setting then instructs the host side drivers to change transmit queuing algorithms, giving a certain application paired with that LUN priority on network traffic, thus increasing the throughput from the host to the LUN by trading off performance of other traffic. ExpressLane does not increase the size of the pipe, but instead trades performance of other applications to give the most critical LUN higher throughput during peak congestion. In addition, ExpressLane can be configured with a QoS value that is carried in the header of every I/O frame of a prioritized LUN that then can be prioritized by switch QoS policy as the I/O moves through the fabric. Once the frame from the application is transmitted, the switch takes the prioritization bit, called the CS_CTL bit, from the frame and processes it through the fabric switch QoS to ensure the frame is not deprioritized on the switch layer. Emulex adapters are fully compliant with FC on the switch and will not break any compatibility using the priority bit. The CS_STL bit can also be used by storage arrays to prioritize processing at the final layer. As an environment scales out, the high priority traffic from any one server is mingling in the network with less important traffic from other servers. If the IT organization has SLAs with their customers, they will be motivated to ensure the delivery of traffic from this high priority application. 5 EMULEX WHITE PAPER Flash Storage Gets Priority with Emulex ExpressLane
Emulex ExpressLane can be deployed in three easy steps: Step 1 Enable ExpressLane option on port, as shown in Figure 3. Figure 3. Enable ExpressLane option on port. Step 2 Connect storage to achieve 16GFC line rate throughput, as shown in Figure 4 Note that this example uses SANBlaze and Medusa Labs Test Tool with the following: n SANBlaze - Six 16GFC target ports - Each port with one LUN n Medusa Labs MLTT - 64k 100% write I/O - 20 threads Figure 4. Connect storage to achieve 16GFC line rate throughput. 6 EMULEX WHITE PAPER Flash Storage Gets Priority with Emulex ExpressLane
Step 3 Prioritize a LUN, as shown in Figure 5 Figure 5. Prioritize a LUN. Following these three steps, it is easy to see the performance of ExpressLane, as shown by Figure 6, where throughput on the prioritized LUN more than doubles and I/O response time is reduced as well (compare this to Figure 4). The ExpressLane solution delivers the following: n End-to-end special handling of priority I/O n Separate queues in the driver and adapter n Use of prioritization flags to identify priority traffic to the fabric n Target priority identification n Initiator, Target and Fabric prioritization schemes are incrementally additive though all three are not necessary to realize full benefits Figure 6. Medusa Labs screenshot after ExpressLane is enabled. 7 EMULEX WHITE PAPER Flash Storage Gets Priority with Emulex ExpressLane
Conclusion Emulex ExpressLane delivers standards-based QoS and performance to meet SLAs for application-sensitive data that uses flash/cache storage. It alleviates congested networks with latency reduction and throughput improvements while maximizing ROI on flash/cache deployments. With its easy to manage on/off switch via OneCommand Manager, ExpressLane requires no changes to the existing infrastructure or additional cost. With ExpressLane, architects can now design their network to alleviate congestion to high priority storage devices using priority queuing to guarantee QoS across enterprise fabrics, mixed workloads and cloud applications. By allowing application-sensitive data to get higher priority for key apps with flash/cache investments, ExpressLane should be enabled for any deployment. World Headquarters 3333 Susan Street, Costa Mesa, CA 92626 +1 714 662 5600 Bangalore, India +91 80 40156789 Beijing, China +86 10 84400221 Dublin, Ireland +35 3 (0) 1 652 1700 Munich, Germany +49 (0) 89 97007 177 Paris, France +33 (0) 158 580 022 Tokyo, Japan +81 3 5325 3261 Singapore +65 6866 3768 Wokingham, United Kingdom +44 (0) 118 977 2929 Brazil +55 11 3443 7735 www.emulex.com 2014 Emulex, Inc. All rights reserved. This document refers to various companies and products by their trade names. In most cases, their respective companies claim these designations as trademarks or registered trademarks. This information is provided for reference only. Although this information is believed to be accurate and reliable at the time of publication, Emulex assumes no responsibility for errors or omissions. Emulex reserves the right to make changes or corrections without notice. This document is the property of Emulex and may not be duplicated without permission from the Company. 14-5147 7/14