Software Defined Storage No More Limits
|
|
- Matthew Hudson
- 8 years ago
- Views:
Transcription
1 RED HAT ARCHITECT WARSZAWA, 25 WRZEŚNIA 2015 Software Defined Storage No More Limits Wojciech Furmankiewicz Senior Solution Architect Red Hat CEE wojtek@redhat.com
2 WHAT HAPPENES IN AN INTERNET MINUTE
3 THE FORECAST By 2020 over 15 ZB of data will be stored 1.5 ZB are stored today
4 STORAGE MARKET GROWTH FORECAST Software-Defined Storage is leading a shift in the global storage industry, with far-reaching effects. By 2016, server-based storage solutions will lower storage hardware costs by 50% or more. SDS-P MARKET SIZE BY SEGMENT $1,349B $1,195B Block Storage File Storage Object Storage Hyperconverged $1,029B $859B Gartner: IT Leaders Can Benefit From Disruptive Innovation in the Storage Industry $706B By 2020, between 70-80% of unstructured data will be held on lower-cost storage managed by SDS environments. $592B $457B Innovation Insight: Separating Hype From Hope for Software-Defined Storage By 2019, 70% of existing storage array products will also be available as software only versions Innovation Insight: Separating Hype From Hope for Software-Defined Storage Source: IDC Market size is projected to increase approximately 20% year-over-year between 2015 and 2019
5 A COUPLE OF STORAGE FACTS >50% Average Storage Capacity CAGR (Compound Annual Growth Rate) >30% Storage share of Total-IT Budget 90:10% Unstructured vs. Structured Data 8 ct Avg. Legacy Storage Price for 1 GB/mon (*) 3 ct Avg. Cloud Storage Price for 1 GB/mon 1-2 ct Avg. Open SW-defined Price 1 GB/mon (*) (*) HW + SW + maintenance for 3 yr. depreciation period
6 THE PROBLEM Growth of data IT Storage Budget Existing systems don t scale Increasing cost and complexity Need to invest in new platforms ahead of time
7 THE SOLUTION PAST: SCALE UP FUTURE: SCALE OUT
8 SCALE-OUT ARCHITECTURE Scale out performance, capacity and availability SERVER (CPU, MEM, IO) STORAGE (CAPACITY, PERFORMANCE) Scale up capacity 1TB GLOBAL NAMESPACE / SINGLE CONSISTENT STORAGE SYSTEM... Single consistent storage system Global namespace Aggregates CPU, memory, network capacity Deploys on RHEL-supported servers and directly connected storage Scale out linearly Scale out performance and capacity as needed
9 SOLUTIONS FOR PETABYTE-SCALE OPERATORS Optimized for large-scale deployments Enhanced for flexibility and performance Version 1.3 of Red Hat Ceph Storage is the first major release since joining the Red Hat Storage product portfolio, and incorporates feedback from customers who have deployed in production at large scale. Version 3.1 of Red Hat Gluster Storage contains many new features and capabilities aimed to bolster data protection, performance, security, and client compatibility. New capabilities include: Areas of improvement: Erasure coding Robustness at scale Tiering Performance tuning Bit Rot detection Operational efficiency NVSv4 client support
10 GROWING INNOVATION COMMUNITIES 78 AUTHORS/mo Contributions from Intel, SanDisk, SUSE,and DTAG. Presenting Ceph Days in cities around the world and quarterly virtual Ceph Developer Summit events COMMITS/mo 258 POSTERS/mo 41 AUTHORS/mo Over 11M downloads in the last 12 months 259 COMMITS/mo Increased development velocity, authorship, and discussion has resulted in rapid feature expansion. 166 POSTERS/mo
11 RED HAT GLUSTER STORAGE
12 GLUSTER: SCALE-OUT ARCHITECTURE
13 GLUSTER: OVERVIEW RED HAT GLUSTER STORAGE POOL VIRTUAL PHYSICAL /brick ADMIN Gluster Node RED HAT STORAGE CLI & WEB /brick /brick SSH HTTPS /brick Gluster Node /brick /brick NFS USERS CIFS /brick FUSE /brick Native Client OpenStack Swift Gluster Node /brick
14 GLUSTER: NODES BRICKS AND VOLUMES A VOLUME IS SOME NUMBER OF BRICKS, CLUSTERED AND EXPORTED WITH GLUSTER Volumes have administrator assigned names (= export names) A brick is a member of only one volume A global namespace can have a mix of replicated and distributed volumes Data in different volumes physically exists on different bricks Volumes can be sub-mounted on clients using NFS, CIFS and/or Glusterfs clients THE DIRECTORY STRUCTURE OF THE VOLUME EXISTS ON EVERY Volume 2 /archives Storage Node Storage Node Storage Node /export1 /export2 /export3 /export1 /export2 /export3 /export4 /export5 /export1 /export2 /export3 /export4 3 bricks 5 bricks 4 bricks Volume 1 /shares
15 GLUSTER: DISTRIBUTED VOLUME
16 GLUSTER: REPLICATED VOLUME
17 GLUSTER: GEO-REPLICATION
18 GLUSTER: LINEAR PERFORMANCE - READ read transfer rate read transfer rate -- normalized 1000 normalized transfer rate (MB/s/server) MB/s servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl= servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl=2 16
19 GLUSTER: LINEAR PERFORMANCE - WRITE transfer rate transfer rate (normalized) MB/s/server MB/s servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl= servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl=
20 BIT ROT DETECTION Detection of silent data corruption Bit rot detection is a mechanism that detects data corruption resulting from silent hardware failures, leading to deterioration in performance and integrity. ADMIN Red Hat Gluster Storage 3.1 provides a mechanism to scan data periodically and detect bit-rot. Using the SHA256 algorithm, checksums are computed when files are accessed and compared against previously stored values. If they do not match, an error is logged for the storage admin.!!! X 0 0
21 SECURITY Scalable and secure NFSv4 client support Using NFS-Ganesha, an NFS server implementation, Red Hat Gluster Storage 3.1 provides client access with simplified failover and failback in the case of a node or network failure. Supporting both NFSv3 and NFSv4 clients, NFSGanesha introduces ACLs for additional security, Kerberos authentication, and dynamic export management. CLIENT CLIENT CLIENT CLIENT NFS NFS NFS NFS NFS-GANESHA NFS-GANESHA STORAGE CLUSTER
22 ROADMAP: RED HAT GLUSTER STORAGE Gluster 3.7, RHEL 6, 7 CORE At-rest encryption PERF MGMT MGMT CORE FILE SEC Device Management Geo-Replication, Snapshots Dashboard FILE SELinux SSL encryption (in-flight) SMB 3 (advanced features) Multi-channel Dynamic provisioning of resources Gluster 3.8, RHEL 6, 7 Compression Deduplication Highly scalable control plane Next-gen replication/distribution Inode quotas Faster Self-heal Controlled Rebalance Active/Active NFSv4 SMB 3 (basic subset) SEC CORE FUTURE (v4.0 and beyond) MGMT Erasure Coding Tiering Bit Rot Detection Snap Schedule V3.2 (H1-2016) FILE TODAY (v3.1) pnfs QoS Client side caching New UI Gluster REST API Gluster 4, RHEL 7
23 DETAIL: RED HAT GLUSTER STORAGE 3.1 New support in the console for snapshotting and geo-replication features. Tiering New features to allow creation of a tier of fast media (SSDs, Flash) that accompanies slower media, supporting policy-based movement of data between tiers and enhancing create/read/write performance for many small file workloads. Bit rot detection Ability to detect silent data corruption in files via signing and scrubbing, enabling long term retention and archival of data without fear of bit rot. Snapshot scheduling Ability to schedule periodic execution of snapshots easily, without the complexity of custom automation scripts. Backup hooks Features that enable incremental, efficient backup of volumes using standard commercial backup tools, providing time-savings over full-volume backups. Erasure coding Introduction of erasure coded volumes (dispersed volumes) that provide cost-effective durability and increase usable capacity when compared to standard RAID and replication. CORE CORE MGMT Snapshots, Geo-replication CORE Support in the console for discovery, format, and creation of bricks based on recommended best practices; an improved dashboard that shows vital statistics of pools. CORE Device management, dashboard CORE MGMT These features were introduced in the most recent release of Red Hat Gluster Storage, and are now supported by Red Hat.
24 DETAIL: RED HAT GLUSTER STORAGE 3.1 PERF Small file Optimizations to enhance small file performance, especially with small file create and write operations. PERF Rebalance Optimizations that result in enhanced rebalance speed at large scale. SECURITY SELinux enforcing mode Introduction of the ability to operate with SELinux in enforcing mode, increasing security across an entire deployment. PROTOCOL NFSv4 (multi-headed) Support for data access via clustered, active-active NFSv4 endpoints, based on the NFS-Ganesha project. PROTOCOL These features were introduced in the most recent release of Red Hat Gluster Storage, and are now supported by Red Hat. SMB 3 (subset of features) Enhancements to SMB 3 protocol negotiation, copy-data offload, and support for in-flight data encryption [Sayan: what is copy-data offload?]
25 RED HAT CEPH STORAGE
26 HISTORY OF CEPH 10 years in making Open Source Project Starts at UCSC OpenStack Integration Mainline Linux Kernel MAY 2012 Launch of Inktank Production Ready Ceph SEPT 2012 Xen Integration CloudStack Integration RHEL-OSP Certification FEB 2014 OCT 2013 Inktank Ceph Enterprise Launch APR 2014 Inktank Acquired by Red Hat
27 TRADITIONAL STORAGE VS. CEPH TRADITIONAL ENTERPRISE STORAGE Single Purpose Multi-Purpose, Unified Hardware Distributed Software Single Vendor Lock-in Open Hard Scale Limit Exabyte Scale
28 ARCHITECTUAL COMPONENTS APP RGW A web services gateway for object storage, compatible with S3 and Swift HOST/VM RBD A reliable, fully-distributed block device with cloud platform integration CLIENT CEPHFS A distributed file system with POSIX semantics and scale-out metadata management LIBRADOS A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP) RADOS A software-based, reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes and lightweight monitors
29 CEPH UNIFIED STORAGE OBJECT STORAGE BLOCK STORAGE FILE SYSTEM S3 & Swift Snapshots POSIX Multi-tenant Clones Linux Kernel Keystone OpenStack CIFS/NFS Geo-Replication Linux Kernel HDFS Erasure Coding Tiering Distributed Metadata
30 CEPH ARCHITECTURE S3/SWIFT HOST/HYPERVISOR OBJECT STORAGE iscsi CIFS/NFS BLOCK STORAGE MONITORS SDK FILE SYSTEM OBJECT STORAGE DAEMONS (OSD) NODE NODE NODE NODE NODE NODE NODE NODE NODE
31 RADOS COMPONENTS OSDs: 10s to 10000s in a cluster One per disk (or one per SSD, RAID group ) Serve stored objects to clients Intelligently peer for replication & recovery M Monitors: Maintain cluster membership and state Provide consensus for distributed decision-making Small, odd number These do not serve stored objects to clients
32 OBJECT STORAGE DAEMONS M OSD btrfs xfs ext4 OSD OSD OSD M FS FS FS FS DISK DISK DISK DISK M
33 A METADATA SERVER? M 1 M APPLICATION 2 M
34 CALCULATED PLACEMENT A-G M H-N M APPLICATION F O-T M U-Z
35 EVEN BETTER: CRUSH OBJECT RADOS CLUSTER
36 CRUSH DYNAMIC DATA PLACEMENT CRUSH: Pseudo-random placement algorithm Fast calculation, no lookup Repeatable, deterministic Statistically uniform distribution Stable mapping Limited data migration on change Rule-based configuration Infrastructure topology aware Adjustable replication Weighting
37 FOCUSED SET OF USE CASES ANALYTICS CLOUD INFRASTRUCTURE RICH MEDIA AND ARCHIVAL SYNC AND SHARE ENTERPRISE VIRTUALIZATION Big Data analytics with Hadoop Machine data analytics with Splunk Virtual machine storage with OpenStack Object storage for tenant applications Cost-effective storage for media streaming Active archives File sync and share with owncloud Storage for conventional virtualization with RHEV
38 VIRTUAL MACHINE STORAGE OpenStack Keystone API Swift API Cinder API The most widely deployed 1 technology for OpenStack storage Glance API Nova API Hypervisor Ceph Oobject Gateway (RADOS GW) Ceph Block Device (RBD) Red Hat Ceph Storage FEATURES Full integration with Nova, Cinder and Glance Single storage for images and ephemeral and persistent volumes Copy-on-write provisioning Swift-compatible object storage gateway Full integration with Red Hat Enterprise Linux OpenStack Platform 1 BENEFITS Provides both volume storage and object storage for tenant applications Reduces provisioning time for new virtual machines No data transfer of images between storage and compute nodes required Unified installation experience with Red Hat Enterprise Linux OpenStack Platform
39 OPENSTACK USER SURVEY DEV / QA PROOF OF CONCEPT PRODUCTION
40 STORAGE FOR TENANT APPLICATIONS OpenStack Unstructured data storage for distributed, cloud-native applications APP APP APP APP S3 API S3 API S3 API S3 API Ceph Object Gateway (RADOS GW) Ceph Object Gateway (RADOS GW) Red Hat Ceph Storage FEATURES Compatibility with S3 and Swift APIs Fully-configurable replicated or erasure coded storage backends Cache tiering pools Multi-site failover BENEFITS Supports broad ecosystem of tools and applications built for the S3 API Provides a modern hot/warm/cold storage topology that offers cost-efficient performance Advanced durability at optimal price
41 RICH MEDIA Unstructured image, video, and audio content Massively-scalable, flexible, and cost-effective storage for image, video, and audio content Red Hat Gluster Storage FEATURES Support for multi-petabyte storage clusters on commodity hardware Erasure coding and replication for capacity-optimized or performance-optimized pools Support for standard file & object protocols Snapshots and replication capabilities for high availability and disaster recovery Red Hat Ceph Storage BENEFITS Provides massive and linear scalability in on-premise or cloud environments Offers robust data protection with an optimal blend of price & performance Standard protocols allow access to broadcast content anywhere, on any device Cost-effective, high performance storage for on-demand rich media content
42 ACTIVE ARCHIVES Unstructured file data Open source, capacity-optimized archival storage on commodity hardware Red Hat Gluster Storage FEATURES Cache tiering to enable "temperature"-based storage Erasure coding to support archive and cold storage use cases Support for industry-standard file and object access protocols Unstructured object data Volume backups Red Hat Ceph Storage BENEFITS Store data based on its access frequency Store data on premise or in a public or hybrid cloud Achieve durability while reducing raw capacity requirements and limiting cost Deploy on industry-standard hardware
43 CEPH WITH RHEV Red Hat Enterprise Virtualization 3.6 RHEV Hypervisor RHEV Hypervisor RHEL Node CEPH BLOCK DEVICE (RBD) Red Hat Ceph Storage (RADOS) Copyright 2014 Red Hat, Inc. Private and Confidential RHEL Node
44 ARCHIVE/COLD STORAGE APPLICATION CACHE POOL (SSD, REPLICATED) BACKING POOL (ERASURE CODED) CEPH STORAGE CLUSTER Copyright 2014 Red Hat, Inc. Private and Confidential
45 RED HAT CEPH STORAGE ROADMAP OSD w/ssd optimization More robust rebalancing Improved repair process Local and pyramid erasure c. CORE Improved read IOPS Faster booting from clones BLOCK S3 object versioning Bucket sharing OBJECT v1.3.z (Q3/Q4 2015) Foreman/puppet installer CLI :: Calamari API parity Multi-user and multi-cluster MGMT v1.3 (June 2015) v2.0 and Beyond (2016) MGMT CORE LTTNG Tracepoints BLOCK Swift Storage Policies SELinux Puppet Modules (Tech Preview) Ceph Hammer SELinux OBJECT Performance Consistency Guided Repair New Backing Store (Tech Preview) MGMT OBJECT BLOCK CORE Ceph Hammer iscsi Mirroring NFS Active/Active multi-site New UI Alerts Ceph Jewel
46 DETAIL: RED HAT CEPH STORAGE 1.3 CORE OSD with SSD optimization Performance improvements for both read and write operations, especially applicable for configurations including all-flash cache tiers. CORE More robust rebalancing Improved rebalancing that prioritizes repair of degraded data over rebalancing of sub-optimally-placed data; optimized data placement and improved utilization reporting and management that delivers better distribution of data. CORE Local/pyramid erasure codes Inclusion of locally-stored parity bit (within a rack or data-center) that reduces network bandwidth required to repair degraded data. MGMT Foreman/puppet installer Support for deployment of new Ceph clusters using Foreman and provided Puppet modules. MGMT CLI :: Calamari API parity Improvements to the Calamari API and command-line interface that enable administrators to perform the same set of operations through each. MGMT These features were introduced in the most recent release of Red Hat Ceph Storage, and are now supported by Red Hat. Multi-user and multi-cluster Support in the calamari interface for multiple administrator accounts and multiple deployed clusters.
47 DETAIL: RED HAT CEPH STORAGE 1.3 BLOCK Improved read IOPS Introduction of allocation hints, which reduce file system fragmentation over time and ensure IOPS performance throughout the life of a block volume. BLOCK Faster booting from clones Addition of copy-on-read functionality to improve initial and subsequent write performance for cloned volumes. OBJECT S3 object versioning New versioning of objects that help users avoid unintended overwrites/ deletions and allow them to archive objects and retrieve previous versions. OBJECT Bucket sharding Sharding of buckets in the Ceph Object Gateway to improve metadata operations on those with a large number of objects. OBJECT These features were introduced in the most recent release of Red Hat Ceph Storage, and are now supported by Red Hat. New RGW implementation A new implementation of the Ceph Object Gateway that uses a Civetweb-based embedded web server to simplify installation and upgrades
48 DETAIL: RED HAT CEPH STORAGE TUFNELL CORE Performance Consistency More intelligent scrubbing policies and improved peering logic to reduce impact of common operations on overall cluster performance. CORE Guided Repair More information about objects will be provided to help administrators perform repair operations on corrupted data. CORE New Backing Store (Tech Preview) New backend for OSDs to provide performance benefits on existing and modern drives (SSD, K/V). MGMT New UI A new user interface with improved sorting and visibility of critical data. MGMT These projects are currently active in the Ceph development community. They may be available and supported by Red Hat once they reach the necessary level of maturity. Alerting Introduction of altering features that notify administrations of critical issues via or SMS.
49 DETAIL: RED HAT CEPH STORAGE TUFNELL BLOCK iscsi Introduction of a highly-available iscsi interface for the Ceph Block Device, allowing integration with legacy systems BLOCK Mirroring Capabilities for managing virtual block devices in multiple regions, maintaining consistency through automated mirroring of incremental changes OBJECT NFS Access to objects stored in the Ceph Object Gateway via standard Network File System (NFS) endpoints, providing storage for legacy systems and applications OBJECT These projects are currently active in the Ceph development community. They may be available and supported by Red Hat once they reach the necessary level of maturity. Active/Active Multi-Site Support for deployment of the Ceph Object Gateway across multiple sites in an active/active configuration (in addition to the currently-available active/passive configuration)
50 VISION: UNIFIED STORAGE MANAGER WEB CONSOLE API COMMAND LINE A browser interface designed for managing distributed storage A full API for automation and integration with outside systems A robust, scriptable command-line interface for expert operators PROVISION INSTALL CONFIGURE TUNE MONITOR Full lifecycle management for distributed, software-defined data services OBJECT STORE VIRTUAL BLOCK DEVICE Storage Cluster on commodity hardware DISTRIBUTED FILE SYSTEM
51 THE RED HAT STORAGE MISSION To offer a unified, open software-defined storage portfolio that delivers a range of data services for next generation workloads thereby accelerating the transition to modern IT infrastructures
52 RED HAT ARCHITECT WARSZAWA, 25 WRZEŚNIA 2015 Dziękuję! Wojciech Furmankiewicz Senior Solution Architect Red Hat CEE wojtek@redhat.com
53
54 RED HAT ARCHITECT WARSZAWA, 25 WRZEŚNIA 2015 Software Defined Storage No More Limits Wojciech Furmankiewicz Senior Solution Architect Red Hat CEE wojtek@redhat.com
55 WHAT HAPPENES IN AN INTERNET MINUTE
56 THE FORECAST By 2020 over 15 ZB of data will be stored 1.5 ZB are stored today 3
57 STORAGE MARKET GROWTH FORECAST Software-Defined Storage is leading a shift in the global storage industry, with far-reaching effects. By 2016, server-based storage solutions will lower storage hardware costs by 50% or more. SDS-P MARKET SIZE BY SEGMENT $1,349B $1,195B Block Storage File Storage Object Storage Hyperconverged $1,029B $859B Gartner: IT Leaders Can Benefit From Disruptive Innovation in the Storage Industry $706B By 2020, between 70-80% of unstructured data will be held on lower-cost storage managed by SDS environments. $592B $457B Innovation Insight: Separating Hype From Hope for Software-Defined Storage By 2019, 70% of existing storage array products will also be available as software only versions Innovation Insight: Separating Hype From Hope for Software-Defined Storage Source: IDC Market size is projected to increase approximately 20% year-over-year between 2015 and 2019
58 A COUPLE OF STORAGE FACTS >50% Average Storage Capacity CAGR (Compound Annual Growth Rate) >30% Storage share of Total-IT Budget 90:10% Unstructured vs. Structured Data 8 ct Avg. Legacy Storage Price for 1 GB/mon (*) 3 ct Avg. Cloud Storage Price for 1 GB/mon 1-2 ct Avg. Open SW-defined Price 1 GB/mon (*) (*) HW + SW + maintenance for 3 yr. depreciation period
59 THE PROBLEM Growth of data IT Storage Budget Existing systems don t scale Increasing cost and complexity Need to invest in new platforms ahead of time 6
60 THE SOLUTION PAST: SCALE UP FUTURE: SCALE OUT 7
61 SCALE-OUT ARCHITECTURE Scale out performance, capacity and availability SERVER (CPU, MEM, IO) STORAGE (CAPACITY, PERFORMANCE) Scale up capacity 1TB GLOBAL NAMESPACE / SINGLE CONSISTENT STORAGE SYSTEM... Single consistent storage system Global namespace Aggregates CPU, memory, network capacity Deploys on RHEL-supported servers and directly connected storage Scale out linearly Scale out performance and capacity as needed
62 SOLUTIONS FOR PETABYTE-SCALE OPERATORS Optimized for large-scale deployments Enhanced for flexibility and performance Version 1.3 of Red Hat Ceph Storage is the first major release since joining the Red Hat Storage product portfolio, and incorporates feedback from customers who have deployed in production at large scale. Version 3.1 of Red Hat Gluster Storage contains many new features and capabilities aimed to bolster data protection, performance, security, and client compatibility. New capabilities include: Areas of improvement: Erasure coding Robustness at scale Tiering Performance tuning Bit Rot detection Operational efficiency NVSv4 client support
63 GROWING INNOVATION COMMUNITIES 78 AUTHORS/mo Contributions from Intel, SanDisk, SUSE,and DTAG. Presenting Ceph Days in cities around the world and quarterly virtual Ceph Developer Summit events COMMITS/mo 258 POSTERS/mo 41 AUTHORS/mo Over 11M downloads in the last 12 months 259 COMMITS/mo Increased development velocity, authorship, and discussion has resulted in rapid feature expansion. 166 POSTERS/mo
64 RED HAT GLUSTER STORAGE
65 GLUSTER: SCALE-OUT ARCHITECTURE
66 GLUSTER: OVERVIEW RED HAT GLUSTER STORAGE POOL VIRTUAL PHYSICAL /brick ADMIN Gluster Node RED HAT STORAGE CLI & WEB /brick /brick SSH HTTPS /brick Gluster Node /brick /brick NFS CIFS FUSE USERS Native Client OpenStack Swift /brick Gluster Node /brick /brick
67 GLUSTER: NODES BRICKS AND VOLUMES A VOLUME IS SOME NUMBER OF BRICKS, CLUSTERED AND EXPORTED WITH GLUSTER Volumes have administrator assigned names (= export names) A brick is a member of only one volume A global namespace can have a mix of replicated and distributed volumes Data in different volumes physically exists on different bricks Volumes can be sub-mounted on clients using NFS, CIFS and/or Glusterfs clients THE DIRECTORY STRUCTURE OF THE VOLUME EXISTS ON EVERY Volume 2 /archives Storage Node Storage Node Storage Node /export1 /export2 /export3 /export1 /export2 /export3 /export4 /export5 /export1 /export2 /export3 /export4 3 bricks 5 bricks 4 bricks Volume 1 /shares
68 GLUSTER: DISTRIBUTED VOLUME
69 GLUSTER: REPLICATED VOLUME
70 GLUSTER: GEO-REPLICATION
71 GLUSTER: LINEAR PERFORMANCE - READ read transfer rate read transfer rate -- normalized 1000 normalized transfer rate (MB/s/server) MB/s servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl= servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl=2 16
72 GLUSTER: LINEAR PERFORMANCE - WRITE transfer rate transfer rate (normalized) MB/s/server MB/s servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl= servers glusterfs, repl=1 gluster-nfs, repl=1 glusterfs, repl=2 gluster-nfs, repl=
73 BIT ROT DETECTION Detection of silent data corruption Bit rot detection is a mechanism that detects data corruption resulting from silent hardware failures, leading to deterioration in performance and integrity. ADMIN Red Hat Gluster Storage 3.1 provides a mechanism to scan data periodically and detect bit-rot. Using the SHA256 algorithm, checksums are computed when files are accessed and compared against previously stored values. If they do not match, an error is logged for the storage admin.!!! X 0 0
74 SECURITY Scalable and secure NFSv4 client support Using NFS-Ganesha, an NFS server implementation, Red Hat Gluster Storage 3.1 provides client access with simplified failover and failback in the case of a node or network failure. Supporting both NFSv3 and NFSv4 clients, NFSGanesha introduces ACLs for additional security, Kerberos authentication, and dynamic export management. CLIENT CLIENT CLIENT CLIENT NFS NFS NFS NFS NFS-GANESHA NFS-GANESHA STORAGE CLUSTER
75 ROADMAP: RED HAT GLUSTER STORAGE TODAY (v3.1) Gluster 3.7, RHEL 6, 7 At-rest encryption Dynamic provisioning of resources CORE SMB 3 (advanced features) Multi-channel Gluster 3.8, RHEL 6, 7 Compression Deduplication Highly scalable control plane Next-gen replication/distribution FILE PERF SEC Device Management Geo-Replication, Snapshots Dashboard MGMT CORE FILE SELinux SSL encryption (in-flight) FILE Active/Active NFSv4 SMB 3 (basic subset) Inode quotas Faster Self-heal Controlled Rebalance MGMT SEC CORE FUTURE (v4.0 and beyond) MGMT Erasure Coding Tiering Bit Rot Detection Snap Schedule V3.2 (H1-2016) pnfs QoS Client side caching New UI Gluster REST API Gluster 4, RHEL 7
76 DETAIL: RED HAT GLUSTER STORAGE 3.1 MGMT Snapshots, Geo-replication New support in the console for snapshotting and geo-replication features. CORE Tiering New features to allow creation of a tier of fast media (SSDs, Flash) that accompanies slower media, supporting policy-based movement of data between tiers and enhancing create/read/write performance for many small file workloads. CORE Bit rot detection Ability to detect silent data corruption in files via signing and scrubbing, enabling long term retention and archival of data without fear of bit rot. CORE Support in the console for discovery, format, and creation of bricks based on recommended best practices; an improved dashboard that shows vital statistics of pools. Snapshot scheduling Ability to schedule periodic execution of snapshots easily, without the complexity of custom automation scripts. CORE Device management, dashboard Backup hooks Features that enable incremental, efficient backup of volumes using standard commercial backup tools, providing time-savings over full-volume backups. CORE MGMT These features were introduced in the most recent release of Red Hat Gluster Storage, and are now supported by Red Hat. Erasure coding Introduction of erasure coded volumes (dispersed volumes) that provide cost-effective durability and increase usable capacity when compared to standard RAID and replication.
77 DETAIL: RED HAT GLUSTER STORAGE 3.1 PERF Small file Optimizations to enhance small file performance, especially with small file create and write operations. PERF Rebalance Optimizations that result in enhanced rebalance speed at large scale. SECURITY SELinux enforcing mode Introduction of the ability to operate with SELinux in enforcing mode, increasing security across an entire deployment. PROTOCOL NFSv4 (multi-headed) Support for data access via clustered, active-active NFSv4 endpoints, based on the NFS-Ganesha project. PROTOCOL These features were introduced in the most recent release of Red Hat Gluster Storage, and are now supported by Red Hat. SMB 3 (subset of features) Enhancements to SMB 3 protocol negotiation, copy-data offload, and support for in-flight data encryption [Sayan: what is copy-data offload?]
78 RED HAT CEPH STORAGE
79 HISTORY OF CEPH 10 years in making Open Source Project Starts at UCSC OpenStack Integration Mainline Linux Kernel MAY 2012 Launch of Inktank Production Ready Ceph SEPT 2012 Xen Integration CloudStack Integration RHEL-OSP Certification FEB 2014 OCT 2013 Inktank Ceph Enterprise Launch APR 2014 Inktank Acquired by Red Hat We can start with a quick how we got here. 26
80 TRADITIONAL STORAGE VS. CEPH TRADITIONAL ENTERPRISE STORAGE Single Purpose Multi-Purpose, Unified Hardware Distributed Software Single Vendor Lock-in Open Hard Scale Limit Exabyte Scale 27
81 ARCHITECTUAL COMPONENTS APP RGW A web services gateway for object storage, compatible with S3 and Swift HOST/VM RBD A reliable, fully-distributed block device with cloud platform integration CLIENT CEPHFS A distributed file system with POSIX semantics and scale-out metadata management LIBRADOS A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP) RADOS A software-based, reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes and lightweight monitors 28
82 CEPH UNIFIED STORAGE OBJECT STORAGE BLOCK STORAGE FILE SYSTEM S3 & Swift Snapshots POSIX Multi-tenant Clones Linux Kernel Keystone OpenStack CIFS/NFS Geo-Replication Linux Kernel HDFS Erasure Coding Tiering Distributed Metadata 29
83 CEPH ARCHITECTURE S3/SWIFT HOST/HYPERVISOR OBJECT STORAGE iscsi CIFS/NFS BLOCK STORAGE MONITORS SDK FILE SYSTEM OBJECT STORAGE DAEMONS (OSD) NODE NODE NODE NODE NODE NODE NODE NODE NODE 30
84 RADOS COMPONENTS OSDs: 10s to 10000s in a cluster One per disk (or one per SSD, RAID group ) Serve stored objects to clients Intelligently peer for replication & recovery M Monitors: Maintain cluster membership and state Provide consensus for distributed decision-making Small, odd number These do not serve stored objects to clients
85 OBJECT STORAGE DAEMONS M OSD btrfs xfs ext4 OSD OSD OSD M FS FS FS FS DISK DISK DISK DISK M
86 A METADATA SERVER? M 1 M APPLICATION 2 M
87 CALCULATED PLACEMENT A-G M H-N M APPLICATION F O-T M U-Z
88 EVEN BETTER: CRUSH OBJECT RADOS CLUSTER
89 CRUSH DYNAMIC DATA PLACEMENT CRUSH: Pseudo-random placement algorithm Fast calculation, no lookup Repeatable, deterministic Statistically uniform distribution Stable mapping Limited data migration on change Rule-based configuration Infrastructure topology aware Adjustable replication Weighting
90 FOCUSED SET OF USE CASES ANALYTICS CLOUD INFRASTRUCTURE RICH MEDIA AND ARCHIVAL SYNC AND SHARE ENTERPRISE VIRTUALIZATION Big Data analytics with Hadoop Machine data analytics with Splunk Virtual machine storage with OpenStack Object storage for tenant applications Cost-effective storage for media streaming Active archives File sync and share with owncloud Storage for conventional virtualization with RHEV
91 VIRTUAL MACHINE STORAGE OpenStack Keystone API Swift API Cinder API The most widely deployed 1 technology for OpenStack storage Glance API Nova API Hypervisor Ceph Oobject Gateway (RADOS GW) Ceph Block Device (RBD) Red Hat Ceph Storage FEATURES Full integration with Nova, Cinder and Glance Single storage for images and ephemeral and persistent volumes Copy-on-write provisioning Swift-compatible object storage gateway Full integration with Red Hat Enterprise Linux OpenStack Platform BENEFITS Provides both volume storage and object storage for tenant applications Reduces provisioning time for new virtual machines No data transfer of images between storage and compute nodes required Unified installation experience with Red Hat Enterprise Linux OpenStack Platform Meeting Notes (12/9/14 13:23) ----Nigel: reduce to just Ceph and block? Nurendra: include old technology on graph? 38
92 OPENSTACK USER SURVEY DEV / QA PROOF OF CONCEPT PRODUCTION The community agrees 39
93 STORAGE FOR TENANT APPLICATIONS OpenStack Unstructured data storage for distributed, cloud-native applications APP APP APP APP S3 API S3 API S3 API S3 API Ceph Object Gateway (RADOS GW) Ceph Object Gateway (RADOS GW) Red Hat Ceph Storage FEATURES Compatibility with S3 and Swift APIs Fully-configurable replicated or erasure coded storage backends Cache tiering pools Multi-site failover BENEFITS Supports broad ecosystem of tools and applications built for the S3 API Provides a modern hot/warm/cold storage topology that offers cost-efficient performance Advanced durability at optimal price Meeting Notes (12/9/14 13:23) ----Nigel: reduce to just Ceph and block? Nurendra: include old technology on graph? 40
94 RICH MEDIA Unstructured image, video, and audio content Massively-scalable, flexible, and cost-effective storage for image, video, and audio content Red Hat Gluster Storage FEATURES Support for multi-petabyte storage clusters on commodity hardware Erasure coding and replication for capacity-optimized or performance-optimized pools Support for standard file & object protocols Snapshots and replication capabilities for high availability and disaster recovery Red Hat Ceph Storage BENEFITS Provides massive and linear scalability in on-premise or cloud environments Offers robust data protection with an optimal blend of price & performance Standard protocols allow access to broadcast content anywhere, on any device Cost-effective, high performance storage for on-demand rich media content Meeting Notes (12/9/14 13:23) ----Nigel: reduce to just Ceph and block? Nurendra: include old technology on graph? 41
95 ACTIVE ARCHIVES Unstructured file data Open source, capacity-optimized archival storage on commodity hardware Red Hat Gluster Storage FEATURES Cache tiering to enable "temperature"-based storage Erasure coding to support archive and cold storage use cases Support for industry-standard file and object access protocols Unstructured object data Volume backups Red Hat Ceph Storage BENEFITS Store data based on its access frequency Store data on premise or in a public or hybrid cloud Achieve durability while reducing raw capacity requirements and limiting cost Deploy on industry-standard hardware Meeting Notes (12/9/14 13:23) ----Nigel: reduce to just Ceph and block? Nurendra: include old technology on graph? 42
96 CEPH WITH RHEV Red Hat Enterprise Virtualization 3.6 RHEV Hypervisor RHEV Hypervisor RHEL Node RHEL Node CEPH BLOCK DEVICE (RBD) Red Hat Ceph Storage (RADOS) Copyright 2014 Red Hat, Inc. Private and Confidential 43
97 ARCHIVE/COLD STORAGE APPLICATION CACHE POOL (SSD, REPLICATED) BACKING POOL (ERASURE CODED) CEPH STORAGE CLUSTER Copyright 2014 Red Hat, Inc. Private and Confidential 44
98 RED HAT CEPH STORAGE ROADMAP v1.3.z (Q3/Q4 2015) OSD w/ssd optimization More robust rebalancing Improved repair process Local and pyramid erasure c. Improved read IOPS Faster booting from clones S3 object versioning Bucket sharing OBJECT v1.3 (June 2015) v2.0 and Beyond (2016) Foreman/puppet installer CLI :: Calamari API parity Multi-user and multi-cluster CORE LTTNG Tracepoints BLOCK SELinux Swift Storage Policies SELinux OBJECT Puppet Modules (Tech Preview) MGMT Ceph Hammer MGMT MGMT Performance Consistency Guided Repair New Backing Store (Tech Preview) CORE BLOCK OBJECT BLOCK CORE Ceph Hammer iscsi Mirroring NFS Active/Active multi-site New UI Alerts Ceph Jewel
99 DETAIL: RED HAT CEPH STORAGE 1.3 CORE OSD with SSD optimization Performance improvements for both read and write operations, especially applicable for configurations including all-flash cache tiers. CORE More robust rebalancing Improved rebalancing that prioritizes repair of degraded data over rebalancing of sub-optimally-placed data; optimized data placement and improved utilization reporting and management that delivers better distribution of data. CORE Local/pyramid erasure codes Inclusion of locally-stored parity bit (within a rack or data-center) that reduces network bandwidth required to repair degraded data. MGMT Foreman/puppet installer Support for deployment of new Ceph clusters using Foreman and provided Puppet modules. MGMT CLI :: Calamari API parity Improvements to the Calamari API and command-line interface that enable administrators to perform the same set of operations through each. MGMT These features were introduced in the most recent release of Red Hat Ceph Storage, and are now supported by Red Hat. Multi-user and multi-cluster Support in the calamari interface for multiple administrator accounts and multiple deployed clusters.
100 DETAIL: RED HAT CEPH STORAGE 1.3 BLOCK Improved read IOPS Introduction of allocation hints, which reduce file system fragmentation over time and ensure IOPS performance throughout the life of a block volume. BLOCK Faster booting from clones Addition of copy-on-read functionality to improve initial and subsequent write performance for cloned volumes. OBJECT S3 object versioning New versioning of objects that help users avoid unintended overwrites/ deletions and allow them to archive objects and retrieve previous versions. OBJECT Bucket sharding Sharding of buckets in the Ceph Object Gateway to improve metadata operations on those with a large number of objects. OBJECT These features were introduced in the most recent release of Red Hat Ceph Storage, and are now supported by Red Hat. New RGW implementation A new implementation of the Ceph Object Gateway that uses a Civetweb-based embedded web server to simplify installation and upgrades
101 DETAIL: RED HAT CEPH STORAGE TUFNELL CORE Performance Consistency More intelligent scrubbing policies and improved peering logic to reduce impact of common operations on overall cluster performance. CORE Guided Repair More information about objects will be provided to help administrators perform repair operations on corrupted data. CORE New Backing Store (Tech Preview) New backend for OSDs to provide performance benefits on existing and modern drives (SSD, K/V). MGMT New UI A new user interface with improved sorting and visibility of critical data. MGMT These projects are currently active in the Ceph development community. They may be available and supported by Red Hat once they reach the necessary level of maturity. Alerting Introduction of altering features that notify administrations of critical issues via or SMS.
102 DETAIL: RED HAT CEPH STORAGE TUFNELL BLOCK iscsi Introduction of a highly-available iscsi interface for the Ceph Block Device, allowing integration with legacy systems BLOCK Mirroring Capabilities for managing virtual block devices in multiple regions, maintaining consistency through automated mirroring of incremental changes OBJECT NFS Access to objects stored in the Ceph Object Gateway via standard Network File System (NFS) endpoints, providing storage for legacy systems and applications OBJECT These projects are currently active in the Ceph development community. They may be available and supported by Red Hat once they reach the necessary level of maturity. Active/Active Multi-Site Support for deployment of the Ceph Object Gateway across multiple sites in an active/active configuration (in addition to the currently-available active/passive configuration)
103 VISION: UNIFIED STORAGE MANAGER WEB CONSOLE API COMMAND LINE A browser interface designed for managing distributed storage A full API for automation and integration with outside systems A robust, scriptable command-line interface for expert operators PROVISION INSTALL CONFIGURE TUNE MONITOR Full lifecycle management for distributed, software-defined data services OBJECT STORE VIRTUAL BLOCK DEVICE Storage Cluster on commodity hardware DISTRIBUTED FILE SYSTEM
104 THE RED HAT STORAGE MISSION To offer a unified, open software-defined storage portfolio that delivers a range of data services for next generation workloads thereby accelerating the transition to modern IT infrastructures
105 RED HAT ARCHITECT WARSZAWA, 25 WRZEŚNIA 2015 Dziękuję! Wojciech Furmankiewicz Senior Solution Architect Red Hat CEE wojtek@redhat.com
106
Software Defined Storage No More Limits
RED HAT ARCHITECT SEMINARS @ PRAGUE, OCTOBER 2nd 2015 Software Defined Storage No More Limits Wojciech Furmankiewicz Senior Solution Architect Red Hat CEE wfurmank@redhat.com WHAT HAPPENES IN AN INTERNET
More informationTHE FUTURE OF STORAGE IS SOFTWARE DEFINED. Jasper Geraerts Business Manager Storage Benelux/Red Hat
THE FUTURE OF STORAGE IS SOFTWARE DEFINED Jasper Geraerts Business Manager Storage Benelux/Red Hat THE FUTURE OF STORAGE Traditional Storage Complex proprietary silos Open, Software-Defined Storage Standardized,
More informationRED HAT STORAGE PORTFOLIO OVERVIEW
RED HAT STORAGE PORTFOLIO OVERVIEW Andrew Hatfield Practice Lead Cloud Storage and Big Data MILCIS November 2015 THE RED HAT STORAGE MISSION To offer a unified, open software-defined storage portfolio
More informationProduct Spotlight. A Look at the Future of Storage. Featuring SUSE Enterprise Storage. Where IT perceptions are reality
Where IT perceptions are reality Product Spotlight A Look at the Future of Storage Featuring SUSE Enterprise Storage Document # SPOTLIGHT2013001 v5, January 2015 Copyright 2015 IT Brand Pulse. All rights
More informationSUSE Enterprise Storage Highly Scalable Software Defined Storage. Gábor Nyers Sales Engineer @SUSE gnyers@suse.com
SUSE Enterprise Storage Highly Scalable Software Defined Storage Gábor Nyers Sales Engineer @SUSE gnyers@suse.com Setting the Stage Enterprise Data Capacity Utilization 1-3% 15-20% 20-25% Tier 0 Ultra
More informationWHITE PAPER. Software Defined Storage Hydrates the Cloud
WHITE PAPER Software Defined Storage Hydrates the Cloud Table of Contents Overview... 2 NexentaStor (Block & File Storage)... 4 Software Defined Data Centers (SDDC)... 5 OpenStack... 5 CloudStack... 6
More informationRED HAT STORAGE SERVER TECHNICAL OVERVIEW
RED HAT STORAGE SERVER TECHNICAL OVERVIEW Ingo Börnig Solution Architect, Red Hat 24.10.2013 NEW STORAGE REQUIREMENTS FOR THE MODERN HYBRID DATACENTER DESIGNED FOR THE NEW DATA LANDSCAPE PETABYTE SCALE
More informationSUSE Enterprise Storage Highly Scalable Software Defined Storage. Māris Smilga
SUSE Enterprise Storage Highly Scalable Software Defined Storage āris Smilga Storage Today Traditional Storage Arrays of disks with RAID for redundancy SANs based on Fibre Channel connectivity Total System
More informationSep 23, 2014. OSBCONF 2014 Cloud backup with Bareos
Sep 23, 2014 OSBCONF 2014 Cloud backup with Bareos OSBCONF 23/09/2014 Content: Who am I Quick overview of Cloud solutions Bareos and Backup/Restore using Cloud Storage Bareos and Backup/Restore of Cloud
More informationClodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage
Clodoaldo Barrera Chief Technical Strategist IBM System Storage Making a successful transition to Software Defined Storage Open Server Summit Santa Clara Nov 2014 Data at the core of everything Data is
More informationBuilding Storage as a Service with OpenStack. Greg Elkinbard Senior Technical Director
Building Storage as a Service with OpenStack Greg Elkinbard Senior Technical Director MIRANTIS 2012 PAGE 1 About the Presenter Greg Elkinbard Senior Technical Director at Mirantis Builds on demand IaaS
More informationDistributed File System Choices: Red Hat Storage, GFS2 & pnfs
Distributed File System Choices: Red Hat Storage, GFS2 & pnfs Ric Wheeler Architect & Senior Manager, Red Hat June 27, 2012 Overview Distributed file system basics Red Hat distributed file systems Performance
More informationWith Red Hat Enterprise Virtualization, you can: Take advantage of existing people skills and investments
RED HAT ENTERPRISE VIRTUALIZATION DATASHEET RED HAT ENTERPRISE VIRTUALIZATION AT A GLANCE Provides a complete end-toend enterprise virtualization solution for servers and desktop Provides an on-ramp to
More informationVMware Software-Defined Storage Vision
VMware Software-Defined Storage Vision Lee Dilworth (@leedilworth) Principal Systems Engineer 2014 VMware Inc. All rights reserved. The Software-Defined Data Center Expand virtual compute to all applications
More informationOPTIMIZING PRIMARY STORAGE WHITE PAPER FILE ARCHIVING SOLUTIONS FROM QSTAR AND CLOUDIAN
OPTIMIZING PRIMARY STORAGE WHITE PAPER FILE ARCHIVING SOLUTIONS FROM QSTAR AND CLOUDIAN CONTENTS EXECUTIVE SUMMARY The Challenges of Data Growth SOLUTION OVERVIEW 3 SOLUTION COMPONENTS 4 Cloudian HyperStore
More informationWhitepaper. NexentaConnect for VMware Virtual SAN. Full Featured File services for Virtual SAN
Whitepaper NexentaConnect for VMware Virtual SAN Full Featured File services for Virtual SAN Table of Contents Introduction... 1 Next Generation Storage and Compute... 1 VMware Virtual SAN... 2 Highlights
More informationAnalyzing Big Data with Splunk A Cost Effective Storage Architecture and Solution
Analyzing Big Data with Splunk A Cost Effective Storage Architecture and Solution Jonathan Halstuch, COO, RackTop Systems JHalstuch@racktopsystems.com Big Data Invasion We hear so much on Big Data and
More informationENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE
ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE Hadoop Storage-as-a-Service ABSTRACT This White Paper illustrates how EMC Elastic Cloud Storage (ECS ) can be used to streamline the Hadoop data analytics
More informationAddressing Storage Management Challenges using Open Source SDS Controller
Addressing Storage Management Challenges using Open Source SDS Controller Anjaneya Reddy Chagam, Intel Chief SDS Architect, Data Center Group Shayne Huddleston, Oregon State University Infrastructure Architect,
More informationMaxta Storage Platform Enterprise Storage Re-defined
Maxta Storage Platform Enterprise Storage Re-defined WHITE PAPER Software-Defined Data Center The Software-Defined Data Center (SDDC) is a unified data center platform that delivers converged computing,
More informationDreamObjects. Cloud Object Storage Powered by Ceph. Monday, November 5, 12
DreamObjects Cloud Object Storage Powered by Ceph This slide is all about me, me, me. Ross Turk Community Manager, Ceph VP Community, Inktank ross@inktank.com @rossturk inktank.com ceph.com 2 DreamHost
More informationGetting performance & scalability on standard platforms, the Object vs Block storage debate. Copyright 2013 MPSTOR LTD. All rights reserved.
Getting performance & scalability on standard platforms, the Object vs Block storage debate 1 December Webinar Session Getting performance & scalability on standard platforms, the Object vs Block storage
More informationRed Hat Storage Server
Red Hat Storage Server Marcel Hergaarden Solution Architect, Red Hat marcel.hergaarden@redhat.com May 23, 2013 Unstoppable, OpenSource Software-based Storage Solution The Foundation for the Modern Hybrid
More informationProduct Overview. Marc Skinner Principal Solutions Architect Red Hat RED HAT ENTERPRISE LINUX OPENSTACK PLATFORM
Product Overview Marc Skinner Principal Solutions Architect Red Hat Seismic Shift in Enterprise IT Driven by IT Consumerization EMERGING EXISTING Applications with predictable usage models Open source
More informationovirt and Gluster Hyperconvergence
ovirt and Gluster Hyperconvergence January 2015 Federico Simoncelli Principal Software Engineer Red Hat ovirt and GlusterFS Hyperconvergence, Jan 2015 1 Agenda ovirt Architecture and Software-defined Data
More informationVMware. 2014 VMware Inc. All rights reserved.
VMware 2014 VMware Inc. All rights reserved. 1 Today s Datacenter Not Yet A Private Cloud Hypervisor Compute challenge largely resolved Hardware abstracted and pooled Dynamic provisioning Self-service
More informationNew Features in PSP2 for SANsymphony -V10 Software-defined Storage Platform and DataCore Virtual SAN
New Features in PSP2 for SANsymphony -V10 Software-defined Storage Platform and DataCore Virtual SAN Updated: May 19, 2015 Contents Introduction... 1 Cloud Integration... 1 OpenStack Support... 1 Expanded
More informationDevelop a process for applying updates to systems, including verifying properties of the update. Create File Systems
RH413 Manage Software Updates Develop a process for applying updates to systems, including verifying properties of the update. Create File Systems Allocate an advanced file system layout, and use file
More informationSMART SCALE YOUR STORAGE - Object "Forever Live" Storage - Roberto Castelli EVP Sales & Marketing BCLOUD
SMART SCALE YOUR STORAGE - Object "Forever Live" Storage - Roberto Castelli EVP Sales & Marketing BCLOUD 1 BCLOUD at a Glance 4 years constantly growing + 3PBs protected and distributed every day from
More informationNetapp @ 10th TF-Storage Meeting
Netapp @ 10th TF-Storage Meeting Wojciech Janusz, Netapp Poland Bogusz Błaszkiewicz, Netapp Poland Ljubljana, 2012.02.20 Agenda Data Ontap Cluster-Mode pnfs E-Series NetApp Confidential - Internal Use
More informationYour Journey to the Cloud with Red Hat
Your Journey to the Cloud with Red Hat Open Source Summit Warszawa 24 marca 2015 Wojciech Furmankiewicz Senior Solution Architect Red Hat CEE wojtek@redhat.com Agenda Cloud Management Platform Evolution,
More informationLinux Powered Storage:
Linux Powered Storage: Building a Storage Server with Linux Architect & Senior Manager rwheeler@redhat.com June 6, 2012 1 Linux Based Systems are Everywhere Used as the base for commercial appliances Enterprise
More informationThe path to the cloud training
The path to the cloud training Guy Carmin RHCE, RHCI, RHCVA, RHCSA Solution Architect IGC, Red Hat May 2015 Roei Goldenberg RHCE Linux Consultant and Cloud expert, Matrix I.T. Challenges in Enterprise
More informationTesting of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari
Testing of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari 1 Agenda Introduction on the objective of the test activities
More informationIntroduction to NetApp Infinite Volume
Technical Report Introduction to NetApp Infinite Volume Sandra Moulton, Reena Gupta, NetApp April 2013 TR-4037 Summary This document provides an overview of NetApp Infinite Volume, a new innovation in
More informationSYNNEFO: A COMPLETE CLOUD PLATFORM OVER GOOGLE GANETI WITH OPENSTACK APIs VANGELIS KOUKIS, TECH LEAD, SYNNEFO
SYNNEFO: A COMPLETE CLOUD PLATFORM OVER GOOGLE GANETI WITH OPENSTACK APIs VANGELIS KOUKIS, TECH LEAD, SYNNEFO 1 Synnefo cloud platform An all-in-one cloud solution Written from scratch in Python Manages
More informationStorPool Distributed Storage Software Technical Overview
StorPool Distributed Storage Software Technical Overview StorPool 2015 Page 1 of 8 StorPool Overview StorPool is distributed storage software. It pools the attached storage (hard disks or SSDs) of standard
More informationEMC IRODS RESOURCE DRIVERS
EMC IRODS RESOURCE DRIVERS PATRICK COMBES: PRINCIPAL SOLUTION ARCHITECT, LIFE SCIENCES 1 QUICK AGENDA Intro to Isilon (~2 hours) Isilon resource driver Intro to ECS (~1.5 hours) ECS Resource driver Possibilities
More informationStorReduce Technical White Paper Cloud-based Data Deduplication
StorReduce Technical White Paper Cloud-based Data Deduplication See also at storreduce.com/docs StorReduce Quick Start Guide StorReduce FAQ StorReduce Solution Brief, and StorReduce Blog at storreduce.com/blog
More informationBlueArc unified network storage systems 7th TF-Storage Meeting. Scale Bigger, Store Smarter, Accelerate Everything
BlueArc unified network storage systems 7th TF-Storage Meeting Scale Bigger, Store Smarter, Accelerate Everything BlueArc s Heritage Private Company, founded in 1998 Headquarters in San Jose, CA Highest
More informationAmazon Cloud Storage Options
Amazon Cloud Storage Options Table of Contents 1. Overview of AWS Storage Options 02 2. Why you should use the AWS Storage 02 3. How to get Data into the AWS.03 4. Types of AWS Storage Options.03 5. Object
More information東 海 大 學 資 訊 工 程 研 究 所 碩 士 論 文
東 海 大 學 資 訊 工 程 研 究 所 碩 士 論 文 指 導 教 授 : 楊 朝 棟 博 士 以 異 質 儲 存 技 術 實 作 一 個 軟 體 定 義 儲 存 服 務 Implementation of a Software-Defined Storage Service with Heterogeneous Storage Technologies 研 究 生 : 連 威 翔 中 華 民
More informationScale out NAS on the outside, Object storage on the inside
Exablox Company and Business Update Douglas Brockett, CEO Launched April 2013 (Founded 2010) OneBlox - scale-out storage for the enterprise - converged storage for primary and backup / archival data OneSystem
More informationEMC SOLUTION FOR SPLUNK
EMC SOLUTION FOR SPLUNK Splunk validation using all-flash EMC XtremIO and EMC Isilon scale-out NAS ABSTRACT This white paper provides details on the validation of functionality and performance of Splunk
More informationHigh Performance Server SAN using Micron M500DC SSDs and Sanbolic Software
High Performance Server SAN using Micron M500DC SSDs and Sanbolic Software White Paper Overview The Micron M500DC SSD was designed after months of close work with major data center service providers and
More informationBig data Devices Apps
Trends driving IT pressures Cloud Big data Devices Apps Devices: Info Workers Will Erase Boundary Between enterprise And Consumer Technologies. Forrester Research. August 30, 2012 Apps: Gartner: Predicts
More informationVM Image Hosting Using the Fujitsu* Eternus CD10000 System with Ceph* Storage Software
Intel Solutions Reference Architecture VM Image Hosting Using the Fujitsu* Eternus CD10000 System with Ceph* Storage Software Intel Xeon Processor E5-2600 v3 Product Family SRA Section: Audience and Purpose
More informationBusiness-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000
Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000 Clear the way for new business opportunities. Unlock the power of data. Overcoming storage limitations Unpredictable data growth
More informationMaxDeploy Hyper- Converged Reference Architecture Solution Brief
MaxDeploy Hyper- Converged Reference Architecture Solution Brief MaxDeploy Reference Architecture solutions are configured and tested for support with Maxta software- defined storage and with industry
More informationMPSTOR Technology Overview
MPSTOR Technology Overview Delivering operational efficiency to the datacenter through software defined storage enabling data centers generate new higher margin revenue streams via a single programmable
More informationIntroduction to Red Hat Storage. January, 2012
Introduction to Red Hat Storage January, 2012 1 Today s Speakers 2 Heather Wellington Tom Trainer Storage Program Marketing Manager Storage Product Marketing Manager Red Hat Acquisition of Gluster What
More informationThe path to the cloud training
The path to the cloud training Guy Carmin Roei Goldenberg RHCE, RHCI, RHCVA, RHCSA Solution Architect IGC, Red Hat RHCE Linux Consultant and Cloud expert, Matrix May 2015 I.T. Challenges in Enterprise
More informationArif Goelmhd Goelammohamed Solutions Architect. @agoelammohamed. Hyperconverged Infrastructure: The How-To and Why Now?
Arif Goelmhd Goelammohamed Solutions Architect @agoelammohamed Hyperconverged Infrastructure: The How-To and Why Now? Agenda: 1. SimpliVity Overview 2. The Problem 3. The Solution 4. Demo Simplify IT with
More informationObject-based Storage in Big Data and Analytics. Ashish Nadkarni Research Director Storage IDC
Object-based Storage in Big Data and Analytics Ashish Nadkarni Research Director Storage IDC IDC s definition of Big Data and Analytics (BDA) Mix of data, talent, technology, processes, and services that
More informationKaminario K2 All-Flash Array
Kaminario K2 All-Flash Array The Kaminario K2 all-flash storage array delivers predictable performance, cost, scale, resiliency and simplicity so organizations can handle ever-changing and unforeseen business
More informationProduct Brochure. Hedvig Distributed Storage Platform Modern Storage for Modern Business. Elastic. Accelerate data to value. Simple.
Product Brochure Elastic Scales to petabytes of data Start with as few as two nodes and scale to thousands. Add capacity if and when needed. Embrace the economics of commodity x86 infrastructure to build
More informationHP OpenStack & Automation
HP OpenStack & Automation Where we are heading Thomas Goh Cloud Computing Cloud Computing Cloud computing is a model for enabling ubiquitous network access to a shared pool of configurable computing resources.
More informationIBM ELASTIC STORAGE SEAN LEE
IBM ELASTIC STORAGE SEAN LEE Solution Architect Platform Computing Division IBM Greater China Group Agenda Challenges in Data Management What is IBM Elastic Storage Key Features Elastic Storage Server
More informationM710 - Max 960 Drive, 8Gb/16Gb FC, Max 48 ports, Max 192GB Cache Memory
SFD6 NEC *Gideon Senderov NEC $1.4B/yr in R & D Over 55 years in servers and storage (1958) SDN, Servers, Storage, Software M-Series and HYDRAstor *Chauncey Schwartz MX10-Series New models are M110, M310,
More informationBuilding Storage Service in a Private Cloud
Building Storage Service in a Private Cloud Sateesh Potturu & Deepak Vasudevan Wipro Technologies Abstract Storage in a private cloud is the storage that sits within a particular enterprise security domain
More informationEMC VNX-F ALL FLASH ARRAY
EMC VNX-F ALL FLASH ARRAY Purpose-built for price, density & speed ESSENTIALS Incredible scale & density with up to 172 TB usable flash capacity in 6U @ 28.63 TB/U Consistent high performance up to 400K
More informationEnd to end application delivery & Citrix XenServer 5. John Glendenning Vice President Server Virtualization, EMEA
End to end application delivery & Citrix XenServer 5 John Glendenning Vice President Server Virtualization, EMEA Businesses Run on Applications Users Apps 2 Users and Apps are Moving Further Apart Consolidation
More informationAt-Scale Data Centers & Demand for New Architectures
Allen Samuels At-Scale Data Centers & Demand for New Architectures Software Architect, Software and Systems Solutions August 12, 2015 1 Forward-Looking Statements During our meeting today we may make forward-looking
More informationRed Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep Dive Dustin L. Black, RHCA Sr. Technical Account Manager Red Hat Global Support Services ** This session will include a live demo from 6-7pm ** Dustin L. Black,
More informationSCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS
Sean Lee Solution Architect, SDI, IBM Systems SCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS Agenda Converging Technology Forces New Generation Applications Data Management Challenges
More informationMit Soft- & Hardware zum Erfolg. Giuseppe Paletta
Mit Soft- & Hardware zum Erfolg IT-Transformation VCE Converged and Hyperconverged Infrastructure VCE VxRack EMC VSPEX Blue IT-Transformation IT has changed dramatically in last past years The requirements
More informationWHITE PAPER Improving Storage Efficiencies with Data Deduplication and Compression
WHITE PAPER Improving Storage Efficiencies with Data Deduplication and Compression Sponsored by: Oracle Steven Scully May 2010 Benjamin Woo IDC OPINION Global Headquarters: 5 Speen Street Framingham, MA
More informationEnterprise Storage Solution for Hyper-V Private Cloud and VDI Deployments using Sanbolic s Melio Cloud Software Suite April 2011
Enterprise Storage Solution for Hyper-V Private Cloud and VDI Deployments using Sanbolic s Melio Cloud Software Suite April 2011 Executive Summary Large enterprise Hyper-V deployments with a large number
More informationSMB in the Cloud David Disseldorp
SMB in the Cloud David Disseldorp Samba Team / SUSE ddiss@suse.de Agenda Cloud storage Common types Interfaces Applications Cloud file servers Microsoft Azure File Service Demonstration Amazon Elastic
More informationIntroduction to Gluster. Versions 3.0.x
Introduction to Gluster Versions 3.0.x Table of Contents Table of Contents... 2 Overview... 3 Gluster File System... 3 Gluster Storage Platform... 3 No metadata with the Elastic Hash Algorithm... 4 A Gluster
More informationowncloud Architecture Overview
owncloud Architecture Overview owncloud, Inc. 57 Bedford Street, Suite 102 Lexington, MA 02420 United States phone: +1 (877) 394-2030 www.owncloud.com/contact owncloud GmbH Schloßäckerstraße 26a 90443
More informationBest Practices for Increasing Ceph Performance with SSD
Best Practices for Increasing Ceph Performance with SSD Jian Zhang Jian.zhang@intel.com Jiangang Duan Jiangang.duan@intel.com Agenda Introduction Filestore performance on All Flash Array KeyValueStore
More informationSOFTWARE-DEFINED STORAGE IN ACTION
SOFTWARE-DEFINED STORAGE IN ACTION Anne Tarantino VP Sales- Velocity Tech Solutions Copyright 2014 DataCore Software Corp. All Rights Reserved. Copyright 2014 DataCore Software Corp. All Rights Reserved.
More informationSOFTWARE DEFINED STORAGE IN ACTION
SOFTWARE DEFINED STORAGE IN ACTION Copyright 2014 DataCore Software Corp. All Rights Reserved. Copyright 2014 DataCore Software Corp. All Rights Reserved. 1 Storage is Growing At an Unsustainable Rate
More informationStorage Architectures for Big Data in the Cloud
Storage Architectures for Big Data in the Cloud Sam Fineberg HP Storage CT Office/ May 2013 Overview Introduction What is big data? Big Data I/O Hadoop/HDFS SAN Distributed FS Cloud Summary Research Areas
More informationA virtual SAN for distributed multi-site environments
Data sheet A virtual SAN for distributed multi-site environments What is StorMagic SvSAN? StorMagic SvSAN is a software storage solution that enables enterprises to eliminate downtime of business critical
More informationDiagram 1: Islands of storage across a digital broadcast workflow
XOR MEDIA CLOUD AQUA Big Data and Traditional Storage The era of big data imposes new challenges on the storage technology industry. As companies accumulate massive amounts of data from video, sound, database,
More informationAccelerate Cloud Initiatives with Cisco UCS and Ubuntu OpenStack
White Paper Accelerate Cloud Initiatives with Cisco UCS and Ubuntu OpenStack What You Will Learn This document is intended for IT decision makers. It describes how the combination of configurations running
More informationRED HAT INFRASTRUCTURE AS A SERVICE OVERVIEW AND ROADMAP. Andrew Cathrow Red Hat, Inc. Wednesday, June 12, 2013
RED HAT INFRASTRUCTURE AS A SERVICE OVERVIEW AND ROADMAP Andrew Cathrow Red Hat, Inc. Wednesday, June 12, 2013 SERVICE MODELS / WORKLOADS TRADITIONAL WORKLOADS Stateful VMs: Application defined in VM Application
More informationThe Design and Implementation of the Zetta Storage Service. October 27, 2009
The Design and Implementation of the Zetta Storage Service October 27, 2009 Zetta s Mission Simplify Enterprise Storage Zetta delivers enterprise-grade storage as a service for IT professionals needing
More informationGetting Started Hacking on OpenNebula
LinuxTag 2013 Berlin, Germany, May 22nd Getting Started Hacking on OpenNebula Carlos Martín Project Engineer Acknowledgments The research leading to these results has received funding from Comunidad de
More informationOpen Source, Scale-out clustered NAS using nfs-ganesha and GlusterFS
Open Source, Scale-out clustered NAS using nfs-ganesha and GlusterFS Anand Subramanian Senior Principal Engineer, Red Hat anands@redhat.com Agenda Introduction GlusterFS NFSv4 nfs-ganesha Nfs-ganesha Architecture
More informationEMC ISILON OneFS OPERATING SYSTEM Powering scale-out storage for the new world of Big Data in the enterprise
EMC ISILON OneFS OPERATING SYSTEM Powering scale-out storage for the new world of Big Data in the enterprise ESSENTIALS Easy-to-use, single volume, single file system architecture Highly scalable with
More informationNEXT GENERATION EMC: LEAD YOUR STORAGE TRANSFORMATION. Copyright 2013 EMC Corporation. All rights reserved.
NEXT GENERATION EMC: LEAD YOUR STORAGE TRANSFORMATION 1 The Business Drivers Increase Revenue INCREASE AGILITY Lower Operational Costs Reduce Risk 2 CLOUD TRANSFORMS IT Lower Operational Costs 3 Disruptive
More informationSwiftStack Filesystem Gateway Architecture
WHITEPAPER SwiftStack Filesystem Gateway Architecture March 2015 by Amanda Plimpton Executive Summary SwiftStack s Filesystem Gateway expands the functionality of an organization s SwiftStack deployment
More informationEMC DATA DOMAIN OPERATING SYSTEM
ESSENTIALS HIGH-SPEED, SCALABLE DEDUPLICATION Up to 58.7 TB/hr performance Reduces protection storage requirements by 10 to 30x CPU-centric scalability DATA INVULNERABILITY ARCHITECTURE Inline write/read
More informationEvolution from Big Data to Smart Data
Evolution from Big Data to Smart Data Information is Exploding 120 HOURS VIDEO UPLOADED TO YOUTUBE 50,000 APPS DOWNLOADED 204 MILLION E-MAILS EVERY MINUTE EVERY DAY Intel Corporation 2015 The Data is Changing
More informationIntroduction to Highly Available NFS Server on scale out storage systems based on GlusterFS
Introduction to Highly Available NFS Server on scale out storage systems based on GlusterFS Soumya Koduri Red Hat Meghana Madhusudhan Red Hat AGENDA What is GlusterFS? Integration with NFS Ganesha Clustered
More informationUnderstanding Enterprise NAS
Anjan Dave, Principal Storage Engineer LSI Corporation Author: Anjan Dave, Principal Storage Engineer, LSI Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA
More informationOpenStack Manila File Storage Bob Callaway, PhD Cloud Solutions Group,
OpenStack Manila File Storage Bob Callaway, PhD Cloud Solutions Group, Agenda Project Overview API Overview Architecture Discussion Driver Details Project Status & Upcoming Features Q & A 2 Manila: Project
More informationTHE EMC ISILON STORY. Big Data In The Enterprise. Copyright 2012 EMC Corporation. All rights reserved.
THE EMC ISILON STORY Big Data In The Enterprise 2012 1 Big Data In The Enterprise Isilon Overview Isilon Technology Summary 2 What is Big Data? 3 The Big Data Challenge File Shares 90 and Archives 80 Bioinformatics
More informationowncloud Architecture Overview
owncloud Architecture Overview Time to get control back Employees are using cloud-based services to share sensitive company data with vendors, customers, partners and each other. They are syncing data
More informationAssignment # 1 (Cloud Computing Security)
Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual
More informationBoas Betzler. Planet. Globally Distributed IaaS Platform Examples AWS and SoftLayer. November 9, 2015. 20014 IBM Corporation
Boas Betzler Cloud IBM Distinguished Computing Engineer for a Smarter Planet Globally Distributed IaaS Platform Examples AWS and SoftLayer November 9, 2015 20014 IBM Corporation Building Data Centers The
More informationA Virtual Filer for VMware s Virtual SAN A Maginatics and VMware Joint Partner Brief
A Virtual Filer for VMware s Virtual SAN A Maginatics and VMware Joint Partner Brief With the massive growth of unstructured data in today s enterprise environments, storage IT administrators are constantly
More informationVMware VSAN och Virtual Volumer
VMware VSAN och Virtual Volumer Claes Bäckström RTS Här läggs leverantörsloggor om det finns VMware VSAN 6.0 Här läggs leverantörsloggor om det finns VMware Virtual SAN 6.0 Radically Simple Hypervisor-Converged
More informationStorage solutions for a. infrastructure. Giacinto DONVITO INFN-Bari. Workshop on Cloud Services for File Synchronisation and Sharing
Storage solutions for a productionlevel cloud infrastructure Giacinto DONVITO INFN-Bari Synchronisation and Sharing 1 Outline Use cases Technologies evaluated Implementation (hw and sw) Problems and optimization
More informationEnterprise IT is complex. Today, IT infrastructure spans the physical, the virtual and applications, and crosses public, private and hybrid clouds.
ENTERPRISE MONITORING & LIFECYCLE MANAGEMENT Unify IT Operations Enterprise IT is complex. Today, IT infrastructure spans the physical, the virtual and applications, and crosses public, private and hybrid
More informationNFS Ganesha and Clustered NAS on Distributed Storage System, GlusterFS. Soumya Koduri Meghana Madhusudhan Red Hat
NFS Ganesha and Clustered NAS on Distributed Storage System, GlusterFS Soumya Koduri Meghana Madhusudhan Red Hat AGENDA NFS( Ganesha) Distributed storage system GlusterFS Integration Clustered NFS Future
More informationRED HAT ENTERPRISE VIRTUALIZATION FOR SERVERS: COMPETITIVE FEATURES
RED HAT ENTERPRISE VIRTUALIZATION FOR SERVERS: COMPETITIVE FEATURES RED HAT ENTERPRISE VIRTUALIZATION FOR SERVERS Server virtualization offers tremendous benefits for enterprise IT organizations server
More information