Software Defined Storage Overview and Concepts Chris Walker cawalker@us.ibm.com Consulting IT Specialist, IBM Systems Storage May 2015 2014 IBM Corporation
BIO IBM Systems - Storage Consulting IT Specialist 25+ years working in IT in an assortment of roles including: Systems Administrator, Programming, Software Engineering, Critical Problem Support, technical Sales, and most currently technical Pre-Sales for IBM System Storage. Tennis, Guitar, Shooting, going out to eat! I have lived in many different places but most recently have lived in the Boston area and moved to Northern Kentucky a little over 2 years back. I am married and have one daughter who recently graduated college and I just bought the farm! 2 2014 IBM Corporation
Agenda Presentation Topic Outline The New Storage Directions Software Defined Storage What is it? How can it be done? Why do it? An Example of SDS from the IBM portfolio Q&A 3 2014 IBM Corporation
The new era of Systems of Interaction transforms how companies engage, interact and transact with their customers Cloud, Analytics, Mobile, and Social Systems of Interaction Traditional IT 20% of new data 80% of new data Mobile/Social Exploding Data Volumes (Big Data) Diverse Data Types (Big Data) Increasing Value of Information (Analytics) Mobile and Social Engagement We need new agile, scalable, and more cost effective ways to store, protect, integrate, and manage data, a business most valued resource Block Data = Structured Data (Data Bases, VMs, core business applications) 4 2014 IBM Corporation File and Object Data = Unstructured Data (Text, Video, Pictures, Office Docs, Device data, Social and Mobile)
What is Software defined Storage? Software-defined storage is any storage software stack that can be installed on any commodity (x86 hardware, hypervisors, or cloud) and/or off-the-shelf computing hardware and used to offer a full suite of storage services and federation between the underlying persistent data placement resources to enable data mobility of its tenants between these resources Key attributes of SDS Runs on commodity hardware: no special hardware or components needed Offers a full suite of storage services: equivalent to traditional hardware systems Federates multiple persistent storage resources: internal disk, cloud, other external storage systems or cloud/object platforms Leverages: Shared nothing architecture (or in some cases a partial or fully shared architecture) and Scale-up or scale-out model Broadly grouped into three buckets: File-based, Object-based and Block-based 5 2014 IBM Corporation
Control Plane vs. Data Plane Control Plane Network of Servers with Disk and Flash (Back-End Could be anywhere running on almost anything) Software Defined Storage Traditional Appliance Integrated device Data Plane 6 2014 IBM Corporation
Three Ways to Get there from here Stay with Specialized Hardware on the back-end (Data Plane) Provide Data Services (Control Plane) tied to specialized hardware Commoditize the back-end (Data Plane) Decouple the Data Services (Control Plane) from specialized hardware; run on commodity servers SW Data Services On Commoditize the the back-end (Data Plane) AND provide Specialized hardware solutions Best of both suited to the SW Data Services task On Provides transition path to And/Or commodity back-end 7 2014 IBM Corporation
Software Define Storage Control and Data Planes Concepts Data Workload Optimized and Transaction Integrity Agility and Rapid Scale Integration and API Services Storage Virtualization SDS Control Plane (SOFTWARE) Policy Automation Analytics and Optimization Backup and Copy Managemen t Storage Efficiency Storage Interface (GUI) Provision Open Common Control Plane to Manage Storage Services for application Workloads Service Classes Self- Service Charge back SDS Data Plane (HARDWARE) Life Cycle Monitor and Manage Scalable Storage Performance and Capacity Virtual Storage Storage Systems 8 2014 IBM Corporation
Software Define Storage Control and Data Planes How we deploy today - Typically Workload Optimized and Transaction Integrity Agility and Rapid Scale Integration and API Services Storage Virtualization SDS Control Plane (SOFTWARE) Policy Automation Analytics and Optimization Backup and Copy Managemen t Storage Efficiency Open Common Control Plane to Manage Storage Services for application Workloads Provision Service Classes Self- Service Charge back Life Cycle Tight Coupling How we deploy today SDS Data Plane (HARDWARE) Scalable Storage Performance and Capacity IBM, EMC, NetApp, Hitachi, HP Specialized, fit to task Hardware Devices Storage Arrays Storage Systems File, Object Block 9 2014 IBM Corporation
Software Define Storage Control and Data Planes Now and Beyond Workload Optimized and Transaction Integrity Agility and Rapid Scale Integration and API Services Storage Virtualization SDS Control Plane (SOFTWARE) Policy Automation Analytics and Optimization Backup and Copy Services Storage Efficiency Open Common Control Plane to Manage Storage Services for the Workloads Provision Service Classes Self- Service Charge back Life Cycle Decoupling of HW and SW SDS Data Plane (HARDWARE) Scalable Storage Performance and Capacity Customer picks their Deployment Model: Integrated Appliance (SW+HW), SaaS Cloud, Software Only on their own hardware platform Tape Libraries, SSDs, Flash HW appliances, Disks File, Block, Object 10 2014 IBM Corporation
Software Defined Storage How can we do this? Most Storage arrays are: Built on Linux, Unix or Windows Kernels - potentially offering a common code base Most Run on Intel x86 offering a common hardware architecture. Are based on Server technology at the core While delivering advanced data services, most storage products are essentially based on 2 or more servers under the covers. What are the components of a server? CPU and associated architecture Memory Disk Bays (SSD, Hard Drives) PCI BUS- I/O HBAs Fibre Channel, NICs, SAS Other PCIe cards as needed Support for Common off-the Shelf (COTS) Servers 11 2014 IBM Corporation
IBM s Software Defined Storage How can we do this? OK, so storage services are essentially delivered from a server platform. How does one certify all the possible server platforms out there?? Virtualization\Hypervisor if the server platform is VMware ESX or Linux KVM certified it can be certified for use. No hardware compatibility lists or expensive testing labs Virtualization vendors have done the interop work already Test the components in the server software deployed atop a virtual machine should perform basic component checks Does it have the required interfaces? NICs, HBAs Does it have the min amount of memory, drives, CPU cores, etc Is it highly available? i.e. redundancy in components and paths If it passes the tests, it could run the data services software stack Caution UPS, Flushing of cache to non-volatile devices Customer responsible for UPS, consistent power 12 2014 IBM Corporation
Software Define Storage Federating data and data management What is driving the need for this? Business Pressures New Applications Data Growth Hybrid Cloud Free data from constraints of hardware and realign with new business processes and applications LOWER Costs 13 2014 IBM Corporation
Software Defined Storage Why offer this? Customer Choice Data without Borders Flexibilty of deployment options no physical or locational lock-in. Rapid deployment and scaling Standardization on HW - Flat Data Center Repurpose standard servers to storage services permanently or as needed Reduced CAPEX OPEX options Mobility and Manageability between deployments whether on premise or off, on integrated appliances or your own standardized servers, or choice of cloud Excellent for QA, DEV, TEST and even DR Flat Data Center Economics Device Independent Storage Services 14 2014 IBM Corporation
IBM s Software Defined Storage Customer Choice using IBM XIV example Deployment Options IBM Integrated Appliance pre-built and tested storage arrays and appliances from IBM. Hardware and Software in an integrated solution. XiV Gen3 IBM HW and Software Bundled XiV Gen3 Software Only Solution IBM sells storage services as software to deploy on any Virtual machine (ESX and later KVM) certified server platform that the customer chooses. SaaS offer in IBM s SoftLayer Cloud or other Cloud storage services delivered from the cloud. Installed on Standard Servers deployed in a public cloud. OPEX model. SLA driven. IBM SW deployed on customer s choice of server HW wherever it may be. as software IBM XiV data/storage services served from SoftLayer, Amazon, Azure Clouds Lenovo, HP, Dell, or other servers 15 2014 IBM Corporation All storage is manageable and interoperable between the different deployments models
IBM s Software Defined Storage What might this look like? Customer Choice Data without Borders Remote Branch Office Storage Services on Standard Servers On Premise Same Functionality 100% Interoperable Location independent storage services Allows for Hybrid Cloud Disaster Recovery CLOUD Storage Services (SaaS) On Public Cloud 16 2014 IBM Corporation IT Data Center Core (HQ) Integrated Storage Appliance (HW + SW) On Premise
SDS Advantage Deployed on standard server hardware from a variety of vendors Flexible deployment on cloud, as an integrated appliance, and as software on client choice of server hardware Complements and transforms existing infrastructure to support next generation applications and deployment models Built on and exploiting robust proven virtualization technology and Cloud APIs Customer wins with lower costs with CAPEX and OPEX deployment options, data mobility. and agility to respond to business demands. 17 2014 IBM Corporation
Q&A 18 2014 IBM Corporation
IBM Ranked #1 Software Defined Storage Software Defined Storage (SDS) Market Share, Worldwide based on revenue for Q2 2014. In new Software Defined Storage Platforms, IBM has #1 share rank and 40% market share The #2 ranked supplier has 9% share This IDC study provides IDC's assessment of software-defined storage platforms, an approach for storage solutions that leverages commodity components to create a highly scalable storage platform all in software The fact the IBM offers world class Highly Avaialble Hardware to couple with our SDS makes it all the better of a solution. Award based on our Storwize Technology and Spectrum Scale(Elastic Storage) IDC Quarterly Storage Software Qview, - 2014 Q2 19 2014 IBM Corporation
Spectrum Storage Announcement IBM s new Billion Dollar Investments - Feb 17 th Press Release https://www-03.ibm.com/press/us/en/pressrelease/46093.wss 20 2014 IBM Corporation
What is Spectrum Storage? - More than a rebranding Breadth of software storage solutions working in concert to provide and manage data services like those of our current integrated offerings Leverages and integrates with existing Storage Investments and solutions that have been time tested and proven - not white board concepts Can be deployed in different models: Software Only, SaaS Cloud, an Integrated solution, or a combination*. IBM Spectrum Storage is broken into 2 basic Macro Collections of Solutions: Storage Management integrating the management, protection, and long term storage of data IBM Spectrum Control - control and monitoring of data infrastructure IBM Spectrum Protect - data backup and recovery IBM Spectrum Archive - long term cost effective storage of inactive data, especially of larger file types Infrastructure Optimization and Data Storage Services IBM Spectrum Virtualize - virtualizing data resources and gaining storage efficiencies IBM Spectrum Accelerate - enterprise class scalable storage cloud environments deployed in hours vs. days or weeks IBM Spectrum Scale - high performance, highly scalable unstructured data platform for file, object, big data, and analytics 21 2014 IBM Corporation * Not all solutions at this time can be delivered in all deployment models.
IBM Spectrum Storage IBM Product Value Proposition Family of Storage Management and Optimization Software Control Protect Archive Virtualize Accelerate Scale Customer Choice Integrated IBM Hardware Solution Appliance Private, Public or Hybrid Cloud SaaS The path to Data without Borders.. First Spectrum Storage Q1 2015 Offering. Although Spectrum Scale has been offered for some time under GPFS name. 22 2014 IBM Corporation
IBM Storage Portfolio 2015 Q1- IBM Spectrum Storage Family mapping/relationships Integrated Systems Elastic Storage File / Object Storage ESS Storage Server Platform Integrated Systems VersaStack Storage Management Software Built-in Innovation Software Defined Storage Easy Tier Real-time Compression IBM Active Cloud Engine TM For Enterprise Workloads DS8000 For Midrange & Entry Workloads DCS3xxx Storage Systems XIV XIV as SW SVC FlashSystem V840/V9000 Storwize Midrange family V7000, V7000 Unified, V5000, V3700 (New Code) SONAS Tivoli Virtual Storage Center (VSC) SmartCloud Storage Access Tivoli Storage FlashCopy Manager Deduplication Linear Tape File System (LTFS) 23 2014 IBM Corporation Spectrum Control For Data Protection and Retention ProtecTIER Dedup Systems Tape drives Tape Libraries For illustration purposes showing relationships Spectrum Protect Spectrum Archive Spectrum Virtualize Spectrum Accelerate Enterprise Tape Virtualization Spectrum Scale Tivoli Storage Manager
IBM Storage IBM Spectrum Storage Solutions Naming Targeted Roll-Out March GA Offering Marketing usage Orderable by this name Name in offering Spectrum Scale (Elastic Storage) 1Q15 2Q15 2Q15 Spectrum Virtualize (SVC) 1Q15 2Q15 4Q15 Spectrum Accelerate (XiV as Software) 1Q15 1Q15 1Q15 Spectrum Control (VSC) Spectrum Control Basic Spectrum Control Storage Insights 1Q15 1Q15 / 2Q15 4Q15 Spectrum Archive (LTFS) 1Q15 2Q15 4Q15 Spectrum Protect (TSM) 1Q15 TBD TBD Other products are available to be sold and ordered under the usual product names. 24
Internet of Things - Storage Related Announcement IBM s new 3 Billion Dollar Investment - Mar 31 st Press release http://www- 03.ibm.com/press/us/en/pressrelease/46453.wss 25 2014 IBM Corporation
Deep Dive into The IBM Spectrum Storage Announce Replay will be available within three business days after the conclusion of the event at the following (corrected URLS): IBMers: http://lt.be.ibm.com/stg/ltu48751 IBM Business Partners: http://www.ibm.com/services/weblectures/dlv/partnerworld/ltu48751 26 2014 IBM Corporation
APR 3rd Partner Monthly calls start for Spectrum Storage IBM Spectrum Scale Monthly Partner C (Scheduled for March 6th) To ensure you have the correct time zone, please select YOUR local time zone when registering by clicking on the "show in my time zone" link. Audience - IBM and Business Partners Only Title: IBM Spectrum Scale Monthly Partner Call Date: APR 3, 2015 Time: 10 a.m. New York, 3 p.m. London, 4 p.m. Paris, 15:00:00 GMT Duration: 1.5 hours Abstract: Monthly Spectrum Scale partner calls provide a high value forum for partners to interact with leading speakers from IBM Platform to enable them to find business, win business, and put their strongest foot forward when leveraging Spectrum Scale for solution sales. Register for this session now: https://www1.gotomeeting.com/register/603324209 After registering you will receive an email confirming your registration with information you need to join the Webinar. 27 2014 IBM Corporation
Q&A 28 2014 IBM Corporation
Extras 29 2014 IBM Corporation
New name for the IBM Software Defined Storage family Family Name: IBM Spectrum Storage Name Usage: There is NO acronym and the name must be fully spelled out every time. You should use IBM Spectrum Scale the first time and then you can use 'Spectrum Scale' after. Do not use Spectrum alone or Scale alone. Do not abbreviate to SS or ISS. Repeat the full name (with IBM) from time to time. IBM Spectrum Scale is a member of the IBM Spectrum Storage family. You can and should say that at some point in documents. There is no shortening of 'IBM Spectrum Storage' 30 2014 IBM Corporation
Spectrum Control = VSC Extend Storage Economic Benefits to Operations Overview Storage virtualization platform and storage management solution for cloud-based and software-defined storage. Virtual Storage Center helps organizations transition to new workloads and updated storage infrastructures. Key Capabilities A single console for managing all types of data on disk, flash, file and object storage systems. Simplified visual administration tools including an advanced web-based user interface, VMware vcentertm plug-in and Cognos Business Intelligence with predesigned reports. Analytics-driven tiered storage optimization that automatically moves data to the most cost-effective tier. Benefits First to Market volume-level, cross-platform automated storage tiering, reduces users cost of storage up to 50% 1 Cloud and Software Defined Storage Service catalog, automated provisioning, optimized utilization of pools of storage Intelligent Performance Management with application and department views of the storage infrastructure what we find is an excellent example of an integrated storage management environment that can centrally manage a heterogeneous storage environment, that can automate a wide variety of storage management tasks (including virtualization and provisioning), and that enables storage to easily be integrated into a cloud computing environment. - Clabby Analytics 1. Based on IBM s own experience using storage analytics 31 31 2014 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Coming in 2Q15: Storage Insights Overview SoftLayer cloud-based storage and data reporting provides key insights into usage without need for major software deployment IBM Storage Analytics Key Capabilities Identify and categorize storage assets View performance history for application workloads Reclaim unused / wasted storage space Optimize data placement within the infrastructure Benefits Helps reduce storage and administrative costs Simplified management and deployment Provides deep understanding of storage and data usage Enables capacity planning and performance management Delivered As a Service, over the Cloud achieve all this without a complex, time consuming deployment of software into your existing environment IBM Storage Insights is available now as public beta Visit https://www-304.ibm.com/software/support/trial/cst/forms/nomination.wss?id=5993 32 32 2014 32 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Spectrum Protect = TSM Transformed Economics for Data Protection Overview Comprehensive data protection and recovery solution for virtual, physical and cloud data. Provides backup, snapshot, archive, recovery, space management, bare machine recovery and disaster recovery capabilities. Key Capabilities Protects virtual, physical, and cloud data with one solution Reduces backup and recovery infrastructure costs by up to 38 percent 1 Delivers greater visualization and administrator productivity Simplifies backups by consolidating administration tasks Benefits Application-aware and VM-aware data protection for any size organization Simplified administration Built-in efficiency features: Deduplication, incremental forever backup Integrated multi-site replication and disaster recovery What s different now is that we ll no longer experience data explosions. Tivoli Storage Manager allows us to keep pace with and manage data growth seamlessly and in ways we ve never been able to before. - Piedmont Healthcare 1. Average of individual customer Analysis Engine Reports from Butterfly Software (PDF, 1.40MB), May 2013, n=450. The savings include cumulative 36-month hardware, hardware maintenance and electrical power savings. Excludes one-time TSM migration cost. 33 33 2014 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Spectrum Virtualize = SVC Transformed Economics of Traditional SAN Overview Highly scalable storage virtualization system providing common functionality, management, and mobility across heterogeneous storage types Key Capabilities Pools storage from multiple systems Compresses data with high performance with Real-time Compression for extraordinary efficiency Enables non-disruptive movement of data among storage systems Supports ultra high availability multi-site configurations Benefits Improves storage utilization up to 100% Supports up to 5x as much data in the same physical space Simplifies management of heterogeneous storage systems Enables rapid deployment of new storage technologies for greater ROI Improves application availability with virtually zero storage-related downtime Virtual Volumes SAN SAN Volume Controller Common Copy Services Storage Pool Heterogeneous Storage Systems 34 34 2014 34 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Spectrum Accelerate = XIV as Software Redefining Economics of Storage Systems Overview Software based on IBM XIV storage that extends XIV enterprise capabilities, such as tuning-free consistent performance, to new delivery models needed by next-generation application environments Key Capabilities Enterprise cloud storage in minutes, leveraging commodity hardware Hotspot-free performance and QoS without any manual or background tuning needed Advanced remote replication, role-based security, and multi-tenancy Deploy on premise or on the cloud Hyper-scale management of dozens of PB Best in class VMware and OpenStack integration Benefits Enterprise-class function and data availability Maximized business agility and flexibility Simplified acquisition and rapid deployment Minimized administration and training costs Exceptional ease of use 35 35 2014 35 IBM Corporation For IBM and IBM Business Partner education only.
Spectrum Scale = Elastic Storage Unleash New Storage Economics on a Global Scale Overview Data management supporting high performance, scale-out file and object storage for multiple use cases (Technical Computing, Big Data, Cloud) up to Yottabytes of data Key Capabilities Single global namespace (enabling single file system across multiple sites) with high performance access scales from departmental to global, multisite needs Automated tiering, data lifecycle management from Flash (6x acceleration) to Tape (10x savings) Enterprise ready: data security (encryption), availability, reliability, large scale proven Open: POSIX compliant, integrated with OpenStack and Hadoop (37% faster) Benefits Improves performance by removing data-related bottlenecks Enhances collaboration across geographies Lowers cost by eliminating duplicate data Reduces cost per performance by placing data on most applicable storage (for example, flash to tape) Enables sharing with a single file system for multiple applications 36 36 2014 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Spectrum Archive = Linear Tape File System New Storage Economics for Active Archive Overview Network attached unstructured data storage with native tape support using Linear Tape File System, delivers the best mix of performance and lowest cost storage Key Capabilities Seamless virtualization of storage tiers Policy based placement of data Single universal namespace for all file data Security and protection of assets Open, non-proprietary, cross platform interchange Integrated functionality with Spectrum Scale Benefits Use data stored on low cost tape as easily as disk Optimize data placement for cost and performance Enable data file sharing without proprietary software Access data anywhere, any place, any time, in any format Scalable, low cost 37 37 2014 37 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Disclaimer IBM s statements regarding its plans, directions and intent are subject to change or withdraw without notice at IBM s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release and timing of any future features or functionality described for our products remains at our sole discretion. 38 38 2014 IBM Corporation For IBM and IBM Business Partner education only. Not intended for distribution to clients or other third parties
Spectrum Storage - ordering, certifications, and rewards work in progress SPECTRUM FAMILY Former Name Ordering / Pricing Process HW Cert Required SW Authorization / Certification Requirement Storage Performance Rewards Rebates IBM Spectrum Control Virtual Storage Center (VSC) AAS & PPA S2 Tivoli Storage Manager SVP Group NO IBM Spectrum Protect Tivoli Storage Manager (TSM) PPA Only NA Open, no cert needed NO IBM Spectrum Archive Linear Tape File Services (LTFS) AAS Only S1 & S3* *EE Only NA YES IBM Spectrum Virtualize San Volume Controller (SVC) AAS & PPA S2 Tivoli Storage Manager SVP Group YES IBM Spectrum Accelerate XIV as SW Only Purple PPA Only S3 Tivoli Storage Manager SVP Group YES IBM Spectrum Scale General Parallel File System (GPFS) AAS & PPA S1 Open, no cert needed YES 39 2014 IBM Corporation
Software Define Storage Control and Data Planes Current Offerings in the Portfolio Workload Optimized and Transaction Integrity Agility and Rapid Scale Integration and API Services Storage Virtualization IBM SDS Control Plane (SOFTWARE) Policy Automation Analytics and Optimization Backup and Copy Managemen t Storage Cloud Open Common Control Plane to Manage Storage Services for application Workloads Provision Service Classes Self- Service Charge back Life Cycle Tight Coupling IBM SDS Data Plane (HARDWARE) How we deploy today ELASTIC STORAGE GPFS Storage Server x86 Elastic Storage Server Power V7000 Unified Scalable Storage Performance and Capacity Storwize Family Storage Virtualization SAN Volume Controller V3700, V5000, V7000 260+ Arrays as External Virtualized Storage XiV Cloud Ready Multi-Tenant Hyper- Scalable DS8 FlashSystem High End 840/900 Mainframe, V840/V9000 Specialty ProtecTier File, Object Block 40 2014 IBM Corporation
IBM s Software Defined Storage Customer Choice using IBM XIV example Deployment Options IBM Integrated Appliance pre-built and tested storage arrays and appliances from IBM. Hardware and Software in an integrated solution. Current portfolio. DS8, XiV Gen3, Gen4, SVC, V9000, V7000, V5000, ProtecTIER, Elastic Storage Server, Tape Libraries Software Only Solution IBM sells our storage services as software to deploy on any Virtual machine (ESX and later KVM) certified server platform that the customer chooses. Elastic Storage, XiV as Software(Spectrum Accelerate), more to come IBM HW and Software Bundled XiV Gen3 IBM SW deployed on customer s choice of server HW wherever it may be. as software Lenovo, HP, Dell, or other servers SaaS offer in IBM s SoftLayer Cloud or other storage services delivered from the cloud. OPEX model. SLA driven. Elastic Storage, XiV as Software(Spectrum Accelerate), more to come 41 2014 IBM Corporation All storage is manageable and interoperable between the different deployments models IBM XiV data/storage services served from SoftLayer.