IBM ELASTIC STORAGE SEAN LEE

Transcription

1 IBM ELASTIC STORAGE SEAN LEE Solution Architect Platform Computing Division IBM Greater China Group

2 Agenda Challenges in Data Management What is IBM Elastic Storage Key Features Elastic Storage Server Elastic Storage on SoftLayer Cloud Elastic Storage Use Cases and Success Stories Q&A

3 General Challenges in Data Management Performance High and growing per file, per data set, per file system and parallel I/O requirements How to achieve performance goals cost effectively? Cost challenges Capital expenditures on custom hardware appliances and non-transferrable software features Operational costs, especially with data islands Technology A variety of workloads make consolidation difficult Growing media density and RAID6 technology limits Compliance and Security Compliance (HIPAA, NSF, financial regulations, warranties...) Security (encryption, secure deletion) Difficult to meet archiving and backup SLAs

4 Data Growth and Budgets Zettabytes 1000 X Exabytes 1000 X Petabytes 1000 X Data doubles approximately every 2 years Elastic Storage closes this gap 3 ways 1) Easy to manage at scale 2) Data lifecycle automation 3) Commodity hardware Terabytes 1000 X Storage budgets increase only 1-5% Gigabytes

5 IT Infrastructure Silos Big Data Compute Network Storage HPC Compute Network Storage File Sharing Compute Network Storage Cloud Compute Network Storage Rigid and manual assignment of redundant IT resources Low utilization Needless data movement and copying

6 Cloud-Specific Challenges - End of Outdated Generalist Approach MB/s Today, deployments can only scale to 100TB using NFS NFS performance will not scale for larger deployments Delayed data synchronization to/from the cloud is reducing processing efficiency MB/s NFS servers in the cloud quickly reach a scalability and performance limit

7 New Generation & Traditional Apps Have Important Differences New Generation Applications Traditional Applications C CC V1 V2 V3 V4 V5 V5.... Vn Elastic Storage Systems of Engagement Insights and engagement Rapid pace and massive data scale Global Elasticity Storage Virtualization Systems of Record Transactions and processes Controlled data growth Efficiency through virtualization Integrated Storage Management and Data Protection

8 What is IBM Elastic Storage and How it Helps Software defined storage solution for traditional and new era applications All delivered through software, based upon IBM General Parallel File System Delivered as software, ready-to-use appliances and cloud-based SaaS (LAN) Elastic Storage Extreme scale and performance High resource and operational efficiency Flash Fast Disk Slow Disk Tape Achieve greater IT agility Helps you transform your business and extract more value from your investments

9 Elastic Storage provides parallel, block-level access to single namespace Clients use data, Network Storage Devices (NSDs) serve shared data All NSD servers export to all clients in active-active mode Elastic Storage stripes files across NSD servers and NSDs in units of file- system block-size NSD client communicates with all the servers File-system load spread evenly across all the servers and storage. No hotspots Easy to scale file-system capacity and performance while keeping the architecture balanced IBM's Network Shared Disk (NSD) protocol File stored in blocks Client does real-time parallel I/O to all the servers and storage disks

10 Elastic Storage Server Delivers Extreme Data Integrity 2- and 3-fault-tolerant erasure codes End-to-end checksum Protection against lost writes Fastest rebuild times using Declustered RAID Breakthrough Performance Declustered RAID reduces app load during rebuilds Up to 3x lower overhead to applications Built-in SSDs and NVRAM for write performance Faster than alternatives today and tomorrow! Lowers TCO 3 Years Maintenance and Support General Purpose Servers Off-the-shelf SBODs Standardized in-band SES management Standard Linux Modular Upgrades Extreme Performance & Scalability * Prevents "Bit Rot" * Fast Rebuilds * Leading Price/Performance * Includes All Original GPFS Features

11 Rebuild Test 3 rd disk failure - start of critical rebuild critical rebuild finished, continue normal rebuild 1 st disk failure no rebuild Norm Rebuild 4 Minutes 16 seconds crit rebuild norm rebuild Rebuild of a Critical Failure in minutes instead of hours and days!

12 Elastic Storage Use Cases Technical Computing Big Data & Analytics Cloud Enterprise storage on standard hardware File NFS GPFS POSIX Elastic Storage Hadoop Connector Single Name Space Data Lake Cinder Swift (Block) (Object) Linear capacity & performance scale out Elastic Storage

13 Enterprise On-premise File Sharing Storage Embeddable, Secure File Sync and Share to PREVENT INFORMATION LEAKAGE Limited popular user devices to deliver user data USE Allow users to assess file anywhere, anytime. Desktop Clients Windows, Mac and Linux Mobile Clients ios, Android, Web Top 3 Business pain-points for File Sharing Lose control, privacy and security of your files in public cloud MANAGE Allow enterprise to have full control of their sensitive data Suffer transaction storm from data growth within same system (no hotspots) PROTECT & INNOVATE Provide highly scalable onpremises application with high reliability and performance 44% of surveyed organizations report that employees use Dropbox without permission. Osterman Research Single Name Space High Flexibility High Security Elastic Storage Server Highly Scalable & Efficient

14 Elastic Storage on Cloud Client and ISV Applications Platform Platform LSF Symphony (SaaS) (SaaS) Elastic Storage on Cloud* Workload & Data Platform LSF Platform Symphony Elastic Storage, based on GPFS SoftLayer, an IBM Company bare metal infrastructure 24X7 CloudOps Support Platform Computing Cloud Service Ready to use, high performance cluster in the cloud On-premise infrastructure Local Infrastructure Platform LSF or Platform Symphony cluster or grid Elastic Storage delivered as a service, bringing high performance, scalable storage and integrated data governance for managing vast amounts of data and billions of files to the cloud Deployed on dedicated bare metal resources at a named data center for optimal I/O performance & security Optimized for technical computing & analytics workloads Installed, integrated & administered by skilled Cloud Ops team

15 Non-Shared Storage Paradigm in the Cloud Elastic Storage servers and storage are isolated inside each organization's private VLAN no sharing for maximum security Compute Nodes + ES clients NSD protocol Elastic Storage Servers LSF Master & Platform Application Center Server Private VLAN Replication Elastic Storage Cluster Elastic Storage on Cloud is a fully integrated solution that includes server and client licenses, installation, support & maintenance of the Elastic Storage environment

16 Success Story: Elastic Storage in Telecommunications Major communications service provider (CSP) in China Challenge: CDR/XDR ingest & analytics Quick and reliable data loading Fast query/analysis Platform scalability for future growth Solution IBM Elastic Storage, Platform Symphony (for Map Reduce ("Hadoop") workload management) and IBM Linux on Power Results IBM's solution was faster than the competitor's while using 50% fewer servers Data loading: 400% faster Scalability and query speed execution: 300% more users and >200% faster

17 Success story: Elastic Storage in Financial Sector 100x performance improvement with IBM Platform Computing (Symphony) and IBM Elastic Storage Dr. Greg Lavender CTO Cloud Architecture and Infrastructure Engineering Video available on Youtube Start 35:44, quote at 38:10

18 Success story: Elastic Storage for Hybrid Clouds Automotive manufacturer in Europe Quickly handle periodic increases in data-intensive computing on SoftLayer cloud IBM Platform workload management solutions dispatches data and compute jobs to cloud-based InfiniBandconnected cluster Secure access via VPN and non-shared bare metal servers Platform Computing Cloud Service Complete hybrid cloud solution On-premise infrastructure with high performance LSF workload management Burst workloads from onpremise to the cloud to quickly address demand Non-shared, encrypted storage to protect IP Bare metal servers, InfiniBand and Platform LSF for maximum performance

19 Elastic Storage Data Plane Vision Single scale-out data plane for the entire data center Unifies VM images, analytics, block devices, objects, and files Single name space no matter where data resides Data in best location, on the best tier (performance & cost), at the right time All in software Single Name Space GPFS POSIX NFS CIFS iscsi OpenStack Nova Swift Cinder Manila Glance VMWare VADP VAAI SRM vsphere Hadoop GPFS Active File Manager GPFS Cloud Storage GPFS SSD Fast Disk GNR Tape Policies for Tiering, Data Distribution, Migration to Tape and Cloud

20 Elastic Storage Summary Realize extreme scale and performance through linear, building block, scale-out Increase resource and operational efficiency by pooling redundant isolated resources and optimizing utilization Achieve greater IT agility by being able to quickly react, provision and redeploy resources in response to new requirements Save acquisitions costs by using standard servers and storage Lower data management costs through policy driven automation and tiered storage management

21 Q&A THANK YOU!