SCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS

Transcription

1 Sean Lee Solution Architect, SDI, IBM Systems SCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS

2 Agenda Converging Technology Forces New Generation Applications Data Management Challenges for IoT IBM Spectrum Scale and Software Defined Infrastructure Solutions Q&A 2

3 Converging Technology Forces Data The basis for competitive advantage Cloud The growth engine for business Engagement Changing expectations, fueled by mobile and social 3

4 New Generation of Applications New Generation Applications Traditional Applications C C V1 V2 V3 V4 V5 V5.... Vn Spectrum Scale Systems of Engagement Insights and engagement Rapid pace and massive data scale Global elasticity Storage Virtualization Systems of Record Transactions and processes Controlled data growth Efficiency through virtualization Integrated Storage Management and Data Protection 4

5 Top Data Management Issues from Customer Perspective Managing Storage Growth Forecasting / Reporting Managing Costs Backup Administration Managing Complexity Performance Problems Archiving / Archive Mgmt Storage Provisioning 0% 10% 20% 30% 40% 50% 60% 70% Source: InfoPro Survey 5

6 Data Processes in IoT Explore and Store Ingest - handle massive amounts of incoming data Process - provide required performance to analytics apps Store - save valuable data, discard useless data Manage and Secure Manage - make data available at the right location, to the right person, at the right time Protect - encrypt, securely delete, backup valuable data Manage & Protect 6

7 Meet New "Data Silo": Data Pool Big Data Compute HPC Compute File Sharing Compute Network Cloud Compute Network Network Network Storage Storage Storage Storage Rigid and manual assignment of redundant IT resources Low utilization Needless data movement and copying 7

8 Traditional File Storage Filer 1 Filer 2 8 8

9 More traditional file storage Filer 1 Filer 2 Filer 3 Filer 4 Filer 5 Filer 6 Filer 7 Filer 8 9 9

10 Solution: IBM Spectrum Scale Filer 1 Filer 2 Filer 3 Filer 4 Global Namespace with automated storage tiering 10 10

11 Parallel, Scale-Out File Sharing and Data Management Solution Global Namespace with automated storage tiering 11 11

12 IBM Spectrum Scale Helps You Handle Data Lakes Spectrum Scale has a maximum file system size of one million yottabytes Designed to scale 2 63 (~9 quintillion) files per file system 1 to 16,384 nodes in a single cluster Spectrum Scale enables virtually limitless performance and capacity Here s where an admin usually hits the data management wall FUTURE PROOF Multiples of bytes Metric YB yottabyte ZB zettabyte EB exabyte PB petabyte TB terabyte GB gigabyte MB megabyte IPv6 support 12

13 IBM Spectrum Scale from IBM Spectrum Storage Family 13

14 What is IBM Spectrum Scale? Software Defined Storage solution for traditional and new era applications All delivered through software, based upon IBM General Parallel File System Delivered as software, readyto-use appliances and cloudbased SaaS Spectrum Scale (software) IBM SPECTRUM SCALE Elastic Storage (SaaS) Elastic Storage Server (appliance) 14 Icons: Visual Pharm, Laura Reen

15 Software Defined Storage for Internet of Things High Output Instruments High Volume Devices compute nodes Computation & Analytics Instruments 400 GB/s GPFS GPFS Sensors 20 PB Single Name Space GPFS Data Ingest IBM Spectrum Scale 15 Images Videos Local Data Access Tape Library Low Latency Global Data Access Auto-tiering and Migration Right Data Right Place Right Time Right Performance Right Cost

16 Deployment with Couplets, Tiers and Tape Archive Spectrum Scale Clients (GPFS, NFS, CIFS, HTTP) Multiple PB's in single name space: G:\ (Windows) /mnt/data (Linux, UNIX) Client-side NETWORK Ethernet/InfiniBand Storage-side NETWORK Spectrum Scale Servers and Storage SAS NL-SAS NL-SAS TAPE SSD NL-SAS NL-SAS Building Block #1 Building Block #2 Building Block #3 Building Block #4 Single Global Namespace (One File System) up to hundreds of PB's 16

17 How Spectrum Scale Helps with Data Management at Scale Common Management Network performance monitoring Drop in replacement for Hadoop File System Native Encryption And Secure Erase Policy based data migration Global Active File Management High speed scanning engine Other cloud Advanced Mirroring and Caching Services Commodity Hardware Cloud Ready Non-disruptive data migration Flash Acceleration File Placement Optimization 17

18 Complementary Solutions: Aspera High Speed File Transfer Software solution Moves your data to/from Spectrum Scale at maximum speed, regardless of file size, transfer distance and network conditions FASP - highly efficient bulk data transport technology Aspera is the industry standard for the transport and management of large data files produced by life sciences. Sifei He, Cloud Product Director, BGI* 18 * Source:

19 Complementary Solutions: Big Storage Active Archive Integrates with local and cloud storage What is it? Enterprise-class cloud based storage archive service for clients who need to store large amounts of data (TBs) and easily retrieve it on demand. Low cost, long term tape-based archive for massive amounts of data Who should use it? Long term archives from TB+ to PB+ size How to use it? Supports both File and Object storage via standard POSIX interface and standard Swift API (HTTP(S)) Spectrum Scale Big Storage Active Archive Spectrum Scale Other service providers Private Cloud 19

20 Success Story: Spectrum Scale in Telecommunications Major communications service provider (CSP) in China Challenge: CDR/XDR ingest & analytics Quick and reliable data loading Fast query/analysis Platform scalability for future growth Solution IBM Spectrum Scale, Platform Symphony (for Map Reduce ("Hadoop") workload management) and IBM Linux on Power Results IBM's solution was faster than the competitor's while using 50% fewer servers Data loading: 400% faster Scalability and query speed execution: 300% more users and >200% faster 20

21 Summary Data management challenges are increasing due to the growth of IoT and new systems of engagement Internet of Things requires agility, the ability to quickly ingest, process and protect data Overcome Data Management for IoT challenges by using IBM Spectrum Scale and other IBM Software Defined solutions 21

22 Q&A IBM's Internet of Things Mission Helping upstream innovators who engineer the next generation of connected devices and those downstream who operate those devices, deliver new business outcomes through better innovation, operation, and customer engagement IoT 22