Enterprise Strategy Group Getting to the bigger truth. White Paper Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era Apeiron introduces NVMe-based storage innovation designed to further extend the benefits of flash By Scott Sinclair, ESG Senior Analyst June 2016 This ESG White Paper was commissioned by Apeiron and is distributed under license from ESG.
White Paper: Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era 2 Contents Introduction... 3 Unlocking the Potential of Flash Storage... 4 Designed for Flash: The Potential of NVMe Technology... 4 Apeiron and Direct Scale-out Flash Storage... 5 The Bigger Truth... 6
Introduction White Paper: Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era 3 As part of a continual pursuit of competitive advantage, businesses are turning towards real-time analytics on much larger data sets. IT organizations are increasingly seeking insights from the mountains of digital content residing within their storage environments. In a recent ESG study, business intelligence/analytics was most often identified by respondent storage administrators as the workload most responsible for storage growth over the following 24 months. 1 However, mining digital content for this understanding, especially when seeking insights in real time, often requires far more performance than what traditional storage architectures can offer. A new generation of workloads (e.g., Splunk, Hadoop, and Spark) demands new levels of transactional performance to deliver the insights businesses require to compete in this era of real time big data analytics. The introduction of solid-state (or flash) storage has altered the performance paradigm of the data center. The removal of rotating media resulted in a significant performance improvement for the enterprise. In direct correlation with this performance boost, the business began to request access to much larger data sets, in real-time. Solid state devices, for the most part, were able to deliver. These benefits, while significant, remain limited. Essentially solid state has raised the bar, but still not to the level needed to truly leverage real-time access to massive data sets. The enterprise is restricted by the other components that still comprise the data path; the network, the applications, and the protocols were designed for rotating media. As such, the designs of these components simply did not anticipate the capabilities of solid-state devices, such as very low latency, parallel access, and significantly increased bandwidth. Accessing the full potential of solid-state storage requires not only the replacement of rotating media, but also the optimization of all other elements in the data path, including the solid-state protocol itself. The Non-Volatile Memory Express (NVMe) protocol has emerged as the industry s answer. Contemporary storage solutions that leverage NVMe technology, however, are often limited in their practicality when serving real-time big data production workloads. Often leveraging the PCIe infrastructure contained within the server and sometimes limited only to certain drive slots, existing NVMe-based solutions can be limited in scale and effectiveness. To be truly effective at leveraging the potential of this solution, the system must be able to pass the native NVMe commands to the drive. One new entrant to the storage industry, Apeiron Data Systems, seeks to meet the industry demand for highly scalable NVMe solutions. With its Direct Scale-out Flash Storage technology, Apeiron integrates the extreme performance and low latency of NVMe-based flash storage with the benefits of networked scale-out storage. These benefits include greater utilization, ease of management, and centralized protection. This new storage offering leverages NVMe over Ethernet (NoE) technology to deliver ultra-low latency storage networking, capable of scaling to thousands of NVMe drives and hundreds of application servers. As reviewed in an ESG Lab Spotlight, Apeiron performance is indistinguishable from internally connected drives (DAS). 2 Because Apeiron is passing native NVMe commands to the drives, any commercially available drive can be supported. The ability to deploy solid-state drives from multiple suppliers and tailor the deployment to the application s requirements offers considerable advantage. This includes next-generation 3D XPoint technology from Intel, a platform capable of providing sub 10 micro-second response times. 1 Source: ESG Research Report, 2015 Data Storage Market Trends, October 2015. All ESG research references and charts in this white paper have been taken from this research report. 2 Source: ESG Lab Spotlight, Apeiron Data Systems ADS1000: Native NVMe Networking Accelerates Real-time Workload Performance, April 2016.
White Paper: Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era 4 Unlocking the Potential of Flash Storage In addition to the findings on business intelligence and data analytics mentioned, ESG s recent research study investigating general storage industry trends provided insights on the impact of solid-state storage. As part of that study, IT storage leaders whose organizations were currently leveraging solid-state storage were asked to identify the benefits their organizations realized from the technology. Improved application performance, the most commonly identified response (57%), will not likely surprise anyone. The second, third, and fourth most-cited responses of improved resource utilization (51%), reduced operational expenses (45%), and improved total cost of ownership (TCO) (44%) may not be immediately intuitive (see Figure 1). These efficiency gains of better utilization and lower expenses were likely the result of the increases in storage performance shifting the bottleneck to another location in the data path. While the term bottleneck often has a negative connotation, it is an inherent aspect of all system design. Another way to think of a bottleneck is as the element that runs at the highest level of utilization, and restricts the utilization of the remaining components. By shifting the bottleneck, solid-state storage allows the remainder of the data path to achieve higher levels of utilization and therefore become more efficient. Simply put, reducing storage latencies enables the rest of the existing IT ecosystem, the applications, the servers, and the network to do more. The result is higher utilization, lower operational costs, and a greater opportunity to leverage the data. Figure 1. Benefits Realized as a Result of Deploying Solid-state Storage Which of the following benefits has your organization realized as the result of deploying solid-state storage technology? (Percent of respondents, N=181, multiple responses accepted) Improved application performance 57% Improved resource utilization 51% Reduced operational expenses Improved total cost of ownership (TCO) Reduced power consumption Improved SLAs Reduced/deferred hardware capital expenditures 45% 44% 43% 39% 37% None of the above 2% Designed for Flash: The Potential of NVMe Technology 0% 10% 20% 30% 40% 50% 60% Source: Enterprise Strategy Group, 2016 To understand where this market is going, you must first understand where it s been. Recently there have been three inflection points driving the demand for massively scalable NVMe solutions. A shift away from traditional, monolithic, controller-based SAN/NAS arrays towards scale out storage solutions has helped ease the management of larger storage pools, while providing an opportunity to scale performance. The second point supporting the rise of scalable NVMe solutions is an increase in applications and operating systems becoming more storage aware. Access to functions such as RAID protection, snaps/clones, and replication over distance no longer require a storage controller. When the application is
White Paper: Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era 5 capable of managing the storage, the need for a complex and latencyinducing controller layer is reduced. The third shift is the demand for realtime analytics on massive data sets. The proliferation of applications such NVMe Protocol Enhancements Increased number of operations as Splunk and Spark is driving storage and processing demand for larger and larger data sets. Analytics conducted over a larger timeframe have Efficient small block transactions more value to the business. The Internet of Things (IoT), for example, is Low power design only going to increase this demand, as mountains of machine-generated data become available for analysis. While traditional scale-out architectures offer capacity scale, contemporary solutions often cannot accommodate this type of growth while providing the performance necessary for real-time analysis. A new technology is required to ingest these massive data sets, while simultaneously providing the economics of storage networking. Unlike NVMe, legacy protocols, such as SAS and SATA, were not designed for solid-state storage. As such, many of the advantages simply cannot be realized by the application. Designs were optimized for parallel scaling and bandwidth rather than for efficient transactions and optimized latency. To address the deficiencies found in SAS and SATA protocols, NVMe offers a number of innovations designed to better support flash storage. For example, NVMe increases the number of concurrent I/O operations by supporting up to 64,000 I/O queues, with each I/O queue supporting up to 64,000 commands far exceeding the multiple command support of SATA, by comparison. For small block transactions with NVMe, all the information for a 4KB read is included in the command, eliminating the need for multiple operations between storage and server. Power consumption is reduced significantly with NVMe by reducing the number of processing cycles required per I/O. The protocol introduces the ability to transition to a lower power state after a pre-defined idle period without software intervention. Existing NVMe-based solutions, however, often leverage PCI-Express (PCIe). This technology helps NVMe maximize the efficiency of each transaction, while increasing the number of simultaneous transactions. However, PCIe introduces its own set of serious limitations. PCIe technology is predominantly locked within server architectures. Attempting to extend that outside of the server results in issues with fault isolation and network resiliency. In many cases, the use of NVMe-based solid-state drives is limited to only specific drive slots, limiting the ability to scale and process the massive data sets today s user is demanding. A new entrant into the NVMe-based storage market is Apeiron Data Systems. With its introduction of the ADS1000, the company is delivering the next step in the evolution of storage networking. With its Direct Scale-out Flash Storage, Apeiron is delivering the extreme performance promised by NVMe, with all of the economic advantages of a shared storage infrastructure. The system delivers a fully integrated NVMe storage and networking solution, capable of delivering native NVMe performance and scalability simply not found anywhere else in the industry. Apeiron and Direct Scale-out Flash Storage Direct Scale-out Flash is a marriage of the high performance and low-latency of direct-attached storage with the simplicity and scalability of scale-out networked storage. Apeiron s storage enclosure, the ADS1000, is a 2U enclosure with integrated storage and networking in a single device. The system leverages NVMe over Ethernet (NoE). Ensuring that the interconnect layer offers non-blocking performance, the solution integrates 32 40Gb Ethernet connections per 2U device, eliminating the need for any external switching such as fibre channel or InfiniBand. Dual 40Gb Ethernet HBAs provide the server side of the connection. Unlike other NVMe solutions based on PCIe, the ADS1000 is capable of scaling out to thousands of drives per domain. Each 2U enclosure has dedicated Inter-switch links to create a massive mesh of native NVMe storage. The resulting Direct Scale-out Flash solution offers a number of benefits to IT organizations: Industry leading performance, improved CPU utilization, and significant consolidation: Unlike scale-out solutions, Apeiron s Direct scale-out Flash enables 100% independent scaling of compute and storage. The ability to balance
White Paper: Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era 6 the storage and server environments results in tremendous CPU utilization improvements because workloads can be shared across many more drives than can be accommodated by captive storage architectures. Consolidation benefits are numerous; better CPU utilization means fewer servers (ESG s past audit of Apeiron testing proves a 70-150% improvement in CPU utilization, dependent upon which NVMe drive manufacturer is used), 3 and fully integrated switching eliminates the entire fabric layer typically associated with shared storage environments. Native NVMe means future-proof deployments: Because Apeiron has developed a native NVMe system, any commercially available drive can be implemented. Alternative systems typically require deploying a proprietary card or drive. This means the storage cannot be allocated based on application profile, but rather one size must fit all. Apeiron s solution boasts four suppliers already qualified, with two more in the process as of this publication (certified drives include: Intel, Toshiba, Samsung, and Micron). Even more important than choice is the ability to leverage the impressive NVMe capacity roadmap. By supporting multiple suppliers, Apeiron can bring this technology to market far in advance of the proprietary alternatives. For example, 6TB drives are expected to be qualified this summer, with 8TB close behind. There are performance advantages, too, as Apeiron is able to support Intel s 3D XPoint technology from day one. Scale for massive big data and analytical data sets: Access to real-time insights via business intelligence and advanced big data analytics workloads requires high-speed transactional access to data. Using NVMe is a significant step toward solving this challenge. Yet limiting that performance to only a few server drive slots effectively neutralizes the true potential of NVMe. The ADS1000 is capable of scaling to thousands of drives and supporting hundreds of servers. The resulting solution delivers the extreme performance critical to real-time and high-performance compute workloads in an architecture designed for scale-out production environments. Simultaneously, Apeiron s solution reduces capital and operational costs through the consolidation of the servers and elimination of external storage switching. Apeiron s ability to scale with multiple NVMe suppliers and profiles makes it possible to keep multiple years of data on NVMe SSDs, ready for real-time queries at petabyte-scale. Despite these benefits, some may question Apeiron s decision to move away from PCIe, asking whether Ethernet may hinder some of the performance potential. Those questions, however, would ultimately prove to be unfounded. In a recent ESG Lab Spotlight, ESG Lab previewed the solution and validated that the ADS1000 delivers performance equal to or better than direct-attached storage, while providing a robust storage network for NVMe. 4 As mentioned above, the limitations of PCIe hinder NVMe s ability to serve as primary storage for this new generation of workloads. With Direct Scale-out Flash storage, Apeiron delivers petabyte-scale NVMe storage capacity, leverages industry-standard technology, and offers choice in using any commercially available NVMe SSDs. The net result reduces lock-in, decreases the capital costs, and lowers operational costs, making Apeiron s Direct Scale-out Flash NVMe storage a compelling choice. The Bigger Truth Disruptive technologies, such as solid-state storage, are only as powerful as their surrounding ecosystems allow them to be. NVMe is the industry s next step to supporting the solid-state disruption and evolving the surrounding ecosystem that supports the technology. The success of new technologies, however, is very much dependent on their ability to integrate into existing processes and paradigms. Despite the benefits of NVMe, the real benefit of Apeiron s technology is its ability to deliver the performance of NVMe in a manner that aligns with the way IT organizations wish to deploy, protect, and service storage ecosystems. By leveraging Ethernet and relying on NVMe over Ethernet (NoE) connectivity, Apeiron provides the benefits of a solid-state-optimized data protocol over an existing, resilient, cost-effective, and, most importantly, familiar network infrastructure. 3 Source: ESG Lab Spotlight, Apeiron Data Systems ADS1000: Native NVMe Networking Accelerates Real-time Workload Performance, April 2016. 4 Source: ESG Lab Spotlight, Apeiron Data Systems ADS1000: Native NVMe Networking Accelerates Real-time Workload Performance, April 2016.
White Paper: Direct Scale-out Flash Storage: Data Path Evolution for the Flash Storage Era 7 Like all new technology players, Apeiron still has to prove itself. Yet integrating a new technology like NVMe in a familiar package such as Ethernet-based scale-out storage aligns well with the demands of modern IT organizations. Ultimately, Direct Scale-out Flash will better enable IT organizations to deploy, manage, and scale business intelligence for real-time insight through big data analytics workloads at petabyte-scale. All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at 508.482.0188. Enterprise Strategy Group is an IT analyst, research, validation, and strategy firm that provides actionable insight and intelligence to the global IT community. www.esg-global.com 2016 by The Enterprise contact@esg-global.com Strategy Group, Inc. All Rights Reserved. P. 508.482.0188