SUSE Enterprise Storage Highly Scalable Software Defined Storage. Māris Smilga

Transcription

1 SUSE Enterprise Storage Highly Scalable Software Defined Storage āris Smilga

2 Storage Today Traditional Storage Arrays of disks with RAID for redundancy SANs based on Fibre Channel connectivity Total System purchase Control integrated with capacity Hardware and software bundled System costs amortized against capacity Gartner "agic Quadrant for General-Purpose Disk Arrays," 21 arch 2013, G

3 Storage Tomorrow Software Defined Storage Separate control from capacity Enable self-service provisioning Commoditization Use standard servers for storage control Off the shelf drives Example: 2 TB 7200 rpm enterprise drive NetApp certified - $699 Seagate - $199 Pervasive Flash Performance gets cheaper No need for 15K rpm hard drives 3

4 Enterprise Data Capacity Utilization 1-3% 15-20% Tier 0 Ultra High Performance Tier 1 High-value, OLTP, Revenue Generating 20-25% Tier 2 Backup/Recovery, Reference Data, Bulk Data 50-60% of Enterprise Data Tier 3 Object, Archive, Compliance Archive, Long-term Retention Source: Horison Information Strategies - Fred oore 5

5 SUSE Enterprise Storage

6 SUSE Storage Product Positioning High-end Disk Array id-range Array Fully Featured NAS Device id-range NAS Entry-level Disk Array JBOD Storage SUSE Enterprise Storage 7

7 SUSE Storage Feature Overview Scalability Upper Limit: EBs, Incremental expansion No Single Point of Failure 100% Software Defined Storage Self managing, Self healing 8

8 Traditional Storage vs. SUSE Storage Traditional Storage Ceph Proprietary hardware Proprietary software Life-cycle enforced by vendor Hard scale limit Commodity hardware Open Source software Hardware Exabyte scale $$$$ $$ 9

9 APP APP Host / V Client RADOSGW RBD CEPH FS Librados A library to directly access RADOS (C,C++, Java, Python, Ruby and PHP) A REST Gateway compatible with Amazon S3 and OpenStack Swift Object based distributed block device with support for Linux kernel and Qemu/KV POSIX compliant distributed file system with support for Linux RADOS Reliable Autonomous Distributed Object Store comprised of self-healing, self-managing intelligent storage nodes 10

10 The Components of the Ceph cluster OSD OSD OSD OSD FS FS FS FS Object Storage Daemon (OSD) Responsible for storing objects on a local file system and providing access to them over the network Disk Disk Disk Disk File System XFS, Btrfs, Ext4 Disk Local SATA or SAS disk 11

11 The Components of the Ceph cluster A Ceph onitor aintains maps of the cluster state, incl: monitor map, OSD map, Placement Group (PG) map, CRUSH map. Disk Local SATA or SAS disk 12

12 Accessing the RADOS Cluster

13 Voilà, a Small RADOS Cluster 14

14 Application Access Application librados Socket 15

15 Application Access Application librados Socket Application REST radosgw librados Socket 16

16 RADOS Block Device Linux Host krbd librados 17

17 RADOS Block Device Disk images are striped across (parts of) the cluster Supports Snapshot and rollback COW cloning Thin provisioning 19

18 CRUSH: Data Placement

19 Several Ingredients Basic Idea Coarse grained partitioning of storage supports policy based mapping (don't put all copies of my data in one rack) Topology map and Rules allow clients to compute the exact location of any storage object Three conceptual components Pools Placement groups CRUSH: deterministic decentralized placement algorithm 24

20 Self-Healing

21 Self-Healing swimmingpool/rubberduck 38.b0b onitors detect a dead OSD 28

22 Self-Healing swimmingpool/rubberduck 38.b0b onitors allocate other OSDs to PG and update CRUSH map 29

23 Self-Healing swimmingpool/rubberduck 38.b0b onitors initiate recovery 30

24 Self-Healing swimmingpool/rubberduck 38.b0b Future writes update the new replica 31

25 Redundancy Levels RADOS offers standard 1:N replication Cheap in terms of compute power Expensive in terms of disk space and write bandwidth In addition, RADOS offers Erasure Coding Essentially, you compute parity data over disks that allows you to recover data from a lost disk A bit like RAID 5, which uses XOR for 1 block of parity for 2 blocks of data Except that erasure code algorithms are more space efficient and have better recovery properties Reasonably cheap in the read case Very expensive in write and recovery case Best suited for read-only/cold data (eg archiving) 32

26 Summary SAVINGS: Total cost of ownership Reduced CAPEX expenditures Reduced OPEX expenditures Ease of management FLEXIBILITY: Adaptability to evolving business needs Reduced dependency upon proprietary Locked In storage CONFIDENCE: Reliability and availability Leverage SUSE world-class support and services 33

27 Questions? Thank you. 34

28 For more information please visit our website: Thank you. 35

29

30 Unpublished Work of SUSE. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.

31 Backup Slides