I D C T E C H N O L O G Y S P O T L I G H T

Transcription

1 I D C T E C H N O L O G Y S P O T L I G H T S o f tw a re - D e fined Storage: T h e N e x t - Generation Data Platform for the S o f tw a re - D e fined Datacenter July 2014 Adapted from IDC's Worldwide Software-Defined Storage Taxonomy, 2014 by Ashish Nadkarni, Laura DuBois, Eric Burgener, et al., IDC # Sponsored by CSC and EMC Enterprise IT is evolving into the software-defined datacenter as a result of cheaper commodity hardware, faster connectivity, and the continued rise of virtualization, Big Data, mobility, and social computing. Applications and infrastructure are no longer dependent on physical resources; rather, they can reside virtually anywhere in the corporate network or cloud. Rapidly following this trend is the move to software-defined storage (SDS), in which data storage also is less dependent on physical infrastructure and can be located where it is most needed. This Technology Spotlight describes the rise of software-defined storage as well as its definition, classification, and delivery. After a discussion of CSC's Storage-as-a-Service offerings, enabled by EMC technology, this paper also provides guidance for enterprises looking for next-generation software-defined storage solutions. Introduction: DAS Is Dead, Long Live DAS The evolution of enterprise IT has been well documented, with the benefits of virtualization and cloud computing leading the way in how enterprises are updating their IT infrastructure. Cheap processing and capacity and the relatively slower time to market for custom solutions have changed the focus of innovation from hardware to software. While initially found in the computing domain, software-based solutions are now entering the storage and networking domains. Thus, enterprise IT is entering the era of the software-defined datacenter, in which both applications and infrastructure are rapidly created in software and installed on cheap commodity hardware. IDC is seeing a rapid change in the storage side of enterprise IT. It wasn't long ago that all storage was physical. Then shared storage network architectures such as enterprise storage area networks (SANs) were an alternative for parallel SCSI connectors and direct-attached storage (DAS). They were the perfect replacement for IT infrastructures that were fragmented, inefficient, and error prone. With optical-based serial connectors and solutions utilizing newer protocols, IT infrastructure quickly became more efficient, nimble, and densely consolidated. Virtualization played a huge role in bringing everything together into a single homogeneous compute space. Shared disk become the mainstay of most IT environments. With newer workloads and continued innovation by Web 2.0 companies and in open source software, it seems that there is an increased push to return to the good old days of DAS. Only this time, the storage is a software-based combination of shared storage and DAS, resulting in a virtual resource that can physically reside anywhere in a network or cloud. Today's version of DAS differs in many ways. Hardware platforms, for example, are commodity in nature, and there is a higher degree of interoperability between platforms manufactured by different suppliers. The use of software-based server, storage, and network virtualization technologies makes IDC 1739

2 it easier to bind together disparate hardware elements into a single namespace. The growth of NoSQL databases helps local data organization and persistence of both structured and unstructured data. The speeds and feeds offered by today's interconnects are much better aligned with hardware and allow for greater sharing of data between hardware platforms. Driving the movement to software-defined storage is the success of storage area networks. These shared storage solutions have become more affordable and so versatile that they can handle nearly every workload in the IT datacenter today. Driving the union between software-based DAS and shared storage are the following key trends: Businesses are storing more data in their environments. These expanding data sets include not just structured data but also semistructured and unstructured data. Unlike archival data, which is basically cold data, these data sets need to be routinely analyzed and therefore need to have proximity to a compute layer. Along with the server and desktop virtualization of mission-critical applications, newer applications spanning Big Data, social, and mobile use cases are fast entering mainstream IT. Newer hybrid cloud based infrastructure-as-a-service offerings are being adopted across a broad range of industries to supplant their on-premises infrastructure. Newer hardware platforms are made of faster CPUs, memory, and disks and can be connected via ultrahigh-speed networks. The Internet has become fast too, allowing businesses to distribute their workforce geographically. On top of these trends are the 800-pound gorillas in IT: the rise of mobility, bring your own device (BYOD), and social media. Cloud and social media companies pioneered software-defined storage to drive down costs by developing and maintaining the entire stack on their own. It was an easy step from there to show other organizations the value of these solutions or to start offering software-based services through the cloud. BYOD and social media go hand in hand, causing the dramatic increase in distributed data that must be analyzed by enterprises for competitive or regulatory reasons. As a result, IDC expects SDS to become the de facto mechanism for designing, delivering, and consuming data. By 2015, commercial SDS revenue will exceed $1.8 billion. What Is Software-Defined Storage? IDC refers to SDS as delivery of storage services via a software stack that uses commodity hardware. For any solution to be classified as software-defined storage, it must satisfy the following requirements: The solution should not contain any proprietary hardware components such as custom ASICs, chipsets, memory components, or CPUs. The solution should be able to run on multiple (physical or virtual) hardware instances that are not factory configured by the supplier; buyers should be able to procure the platform as software and deploy it in a virtual environment or directly on physical hardware. The solution is a standalone or autonomous system that provides all essential storage services and data persistence functions without requiring additional hardware or software. SDS helps datacenters achieve two key objectives: The federation of disparate storage resources, which allows data to be placed on any physical or logical resource and moved across resources automatically and seamlessly The creation of a service-based infrastructure, which enables an abstraction layer and a unified presentation layer across disparate resources, providing the ability to procure, add, and provision components based on necessity with the ability to meter utilization IDC

3 Classifying and Delivering SDS IDC uses data organization as the starting point to classify types of software-defined storage. SDS solutions can organize data in the form of files, objects, and/or blocks. Furthermore, SDS solutions can leverage a variety of persistent data storage resources such as internal storage resources, external disk arrays, tape drives, and even higher-level services such as NoSQL databases, object storage, and cloud-based resources. SDS solutions should offer a full suite of data access interfaces, storage, and data management services. Software-defined solutions can be classified into three distinct categories: File-based layout, which employs a file system for organizing data (File systems can be unitary meaning they are confined to a single host or controller or they can be distributed meaning they are shared between multiple hosts/controllers.) Object-based layout, which refers to a system that employs a container-type mechanism for data organization rather than a hierarchical tree-based structure Block-based layout, in which data is organized in the form of blocks, with physical resources grouped in the form of disk or RAID groups and then chunked up into logical constructs such as plexes and volumes All three types of data organization employ a scaling mechanism that is either scale-up or scale-out in nature. In scale-up, clustering software can allow file servers or block-based nodes to be highly available when deployed with two or more nodes sharing a common pool of block storage. This type of data organization assumes a scale-up characteristic in which the capabilities of the controllers largely dictate scalability in terms of performance and capacity. Typically, capacity and performance cannot be scaled independent of each other. In scale-out, file servers are independent nodes that are tied together by a distributed file system, an object dispersal system, or a distributed target discovery scheme. Capacity and performance can be scaled independently. Geodispersal is a variant of scale-out in which nodes or clusters can be distributed geographically while ensuring data consistency and coherency. There are multiple ways to deliver SDS to the enterprise datacenter. Traditional storage delivery models were largely hardware based, and enterprises did not have the option to install the solution on a platform of their choosing. Today, suppliers are coming up with newer approaches: Physical SDS appliance delivery refers to the traditional way of delivering storage platforms. Suppliers that focus on physical appliances largely do so to ensure consistency in deployment quality and support and generally take ownership of the vertical integration. Virtual appliance (also referred to as virtual storage appliance) delivery is the delivery of the SDS platform as a virtual machine to the buyer. Embedded (platform) connector delivery is often employed by suppliers that want to embed their SDS platform into a hypervisor or the operating system running on a physical server or virtual machine. In the software-only model, the supplier makes the SDS platform available as software that can be downloaded or purchased. DIY-focused buyers then install it on a hardware platform or virtual server of their choosing. In cloud-based pay as you go, suppliers often host the SDS platform on a public cloud infrastructure and offer it as a service to buyers. This is becoming a popular delivery model as businesses leverage the cloud for on-demand scalability IDC 3

4 Compustorage clusters create a hybrid compute/storage environment. By using the SDS data organization in a creative manner, these platforms can allow workload localization on the storage node minimizing the need to move data to the compute layer. Compustorage clusters can be thought of as variants of converged infrastructure platforms for specific use cases such as analytics and virtualization. Considering CSC's Storage-as-a-Service Solutions, Enabled by EMC Technology CSC has been providing IT services and solutions for more than 50 years. This global company offers a broad range of cloud, mobility, cyber, datacenter, and connectivity solutions designed to increase IT infrastructure capacity, ROI, and performance. Included in CSC's offerings are complete datacenter environments that achieve ISO standards, high availability, and optimized IT infrastructure; compute services in which the company helps customers design, deploy, and maintain powerful compute infrastructure; enterprise connectivity services to help customers design and deploy WAN, LAN, and Internet connectivity; and a portfolio of fully managed data storage services that provide scalable solutions to meet the needs of information-intensive global enterprises. CSC manages over 220 petabytes of storage for customers, both on-premises and in 56 datacenters across the globe. CSC offers comprehensive Storage-as-a-Service solutions that can offer the scalability and cost efficiencies needed to store, recover, and archive massive amounts of distributed business-critical information. Based on EMC's ViPR storage platform, CSC's solutions are optimized for virtual, Big Data, mobile, and social environments, helping organizations evolve IT into a software-defined datacenter. As part of its Storage-as-a-Service, CSC offers three key solutions: Store is a fully managed utility-based service, providing a cost-effective solution for managing rapid data growth. Recover provides high levels of data protection by safeguarding business-critical information from events such as corruption, deletion, or malicious attacks. Archive, enabled by EMC ViPR, efficiently uses storage resources and improves storage management. CSC's solutions offer dynamic virtual storage pools, simplified provisioning, automated policy management, and simplified storage management and delivery, with expanded Big Data capabilities and resources for software-defined datacenters. Challenges But CSC does face some challenges. First, many enterprises have huge investments in traditional storage approaches. They have been tied to physical, scale-up growth for years and are slow to adopt new methods. CSC must use its enterprise experience to show the value of SDS and convince enterprises that SDS is the future. Competition, as always, is another significant challenge for CSC. Both IT services and storage product suppliers will be making the market more confusing for enterprises, and CSC needs to clearly differentiate itself and its services IDC

5 Conclusion and Essential Guidance IDC expects that SDS will become the de facto approach for designing next-generation storage platforms. Increasingly, users will look to software-defined platforms as the medium to store data in a cost-effective manner, especially as the data sets get bigger. As a result, the datacenters of tomorrow will look very different. Users will have a wide range of options for data storage, from the compute layer to disk storage mechanisms and from local open object interfaces to cloud-based interfaces. Initially, users will move their non-mission-critical and non-business-critical workloads to such platforms, but eventually, they will move all data to such platforms. In addition, SDS platforms will enable businesses to provision resources from a variety of locations but maintain a seamless presentation layer regardless of the device or location. IDC recommends that enterprises looking for next-generation SDS platforms consider the following: Platform openness: For an SDS platform to be successful, it has to support open standards and interfaces. Solution enablement: The value of the SDS platform lies in the number and diversity of solutions it enables. These solutions must handle geographically dispersed structured, unstructured, and semistructured data and computing. Scale: SDS platforms must match or exceed hardware-based platforms in performance. Only then will they appeal to a more diverse set of use cases and not be viewed just as platforms meant for archiving. Cloud capabilities: The more dispersal capabilities an SDS platform has, the better it is suited for cloud and cloudlike deployments in which data may be ingested and/or analyzed concurrently across multiple locations globally. Workload adjacency: In the era of Big Data and analytics, it is often necessary and efficient to make the analytical workloads run very close to where the data lives. This means that SDS platforms must accommodate compute workloads in such a way that the compute workloads have data affinity or localization. Object awareness: An SDS platform should provide adequate intelligence to handle "objects" of different types. In other words, it needs to understand what an object is, place the object on the appropriate storage tier, and maintain the proper level of metadata detail. Enterprises should keep an open mind when adopting newer software platform models that break the traditional concepts of compute, storage, and network components of the infrastructure. They should push their suppliers to offer service quality and service assurances using newer SDS platforms, which will further drive SDS into a bigger role in the datacenter. To the extent that CSC can address the challenges described in this paper, the company has an opportunity for success with its software-defined storage solutions and services IDC 5

6 A B O U T T H I S P U B L I C A T I ON This publication was produced by IDC Custom Solutions. The opinion, analysis, and research results presented herein are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor sponsorship is noted. IDC Custom Solutions makes IDC content available in a wide range of formats for distribution by various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee. C O P Y R I G H T A N D R E S T R I C T I O N S Any IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requires prior written approval from IDC. For permission requests, contact the IDC Custom Solutions information line at or gms@idc.com. Translation and/or localization of this document require an additional license from IDC. For more information on IDC, visit For more information on IDC Custom Solutions, visit Global Headquarters: 5 Speen Street Framingham, MA USA P F IDC