Cloudian The Storage Evolution to the Cloud.. Cloudian Inc. Pre Sales Engineering

Transcription

1 Cloudian The Storage Evolution to the Cloud.. Cloudian Inc. Pre Sales Engineering

2 Agenda Industry Trends Cloud Storage Evolu4on of Storage Architectures Storage Connec4vity redefined S3 Cloud Storage Use Cases

3 Machine Data Builds 35 Zettabytes in Enterprise Data 2020 DATA VOLUME Machine Generated Data Zettabyte Exabyte Logs, Files Multi- Structured Block & File Perpetual Streaming 1.8 Zettabytes in Enterprise Data Interactions 2011 Petabyte Human Files Terabyte Transactional Data Mainframe PC Internet Mobile Machine Zettabyte = 1,000,000,000,000,000,000,000 Bytes Graph based on IDC and UC Berkeley Data Growth Estimates, Source: IDC & CosmoBC.com:

4 Market Trends Data Explosion & Shrinking Budgets Workloads, IT Budgets and Trends Source: 2014 Company, Gartner, IDC and Wall Street research.

5 Data Classifica>on Structured data Structured data typically refers to data in a database (Oracle, MSFT SQL Server, IBM DB2, SAP Sybase ) Data that resides in a fixed field within a record or file Includes data in a rela4onal database or spreadsheet Easily stored and analyzed Data types include numeric, currency, alphabe4c, name, data, address Unstructured data Generically refers to everything else and typically refers to data in file systems Describes any corporate data not in a database Can include s, presenta4ons, word processing documents, video, audio, photos, webpages Requires use of seman4c search in order to locate informa4on

6 A Cloud Storage plaaorm offers OBJECT OBJECT OBJECT OBJECT OBJECT OBJECT OBJECT Low cost Designed for PB scale Separa4on from performance 4ers A flat namespace of billions of objects Geo distribu4on, protec4on, & access Analy4cs based on Smart Data User meta data as a first-class en4ty Storage as a revenue source Mul4-tenant access & metering So_ware Defined via RESTful API Amazon S3, Swi_ and CDMI API OBJECT OBJECT 6

7 Cloud Storage Changes the Game.. L.A. BOSTON LONDON L.A. BOSTON LONDON Storage Islands Disparate Systems Mul4ple Administra4on Points Mul4ple Filesystem s need to be managed and replicated Cloud /Object Storage Pool Single System Across Loca4ons Access to One Storage System without provisioning Web Based Administra4on and delivery 18/11/15 Cloudian Confiden4al 7

8 Storage needs are evolving, squeezing the tradi>onal model Scale Up / Tera Byte Flash host/array Tradi>onal SAN/NAS Scalability / Big Data Object Storage Scale Out / Peta Byte Limited by Controllers Transac4on Volume Unlimited by Scale High Price / Proprietary Price / GB Commodity / Low Price High Performance / Low Latency Latency Higher Latency / Archive Local / Locking Access Distributed Limited / Ac>ve-Readonly Robustness / DR Cloudian Confiden4al Durable / Ac>ve-Ac>ve 8

9 Op>mal Storage Architecture Workload Centric Backup & Archive Big Data Analytics Storage As a Service File Web Content Collaboration Storage Tier 1 Storage Scale out All Flash Array Op4mised for $/IOPs to support performance workloads Tier 2 Storage Scale out Cloud/object Storage Op4mised for $/GBs to support capacity workloads 9

10 S3 Simple Storage Service Amazon launched S3, in the US in March 2006/Europe in November 2007 S3 is storage for the Internet. It s a simple storage service that offers so_ware developers a Programmable API using web service interfaces (HTTP). which offers; Highly-scalable Reliable Low-latency Commodity level costs It is designed to be highly flexible: Store any type and amount of data that you want; read the same piece of data a million 4mes or only for emergency disaster recovery *2000 ISVs support for S3 today 10

11 S3 Func>onality A modern storage protocol combining Connec4vity Access control Data management Repor4ng & No4fica4ons Advanced Func4onality Tiering. that can be used across the internet. Encryp4on Billing Monitoring No4fica4ons Replica4on Time limited access Standard Storage Protocols S3 API 11

12 S3 Concepts - Objects S3 is a simple key, value store designed to store as many objects as you want. You store these objects in one or more buckets. An object consists of the following: Key The name that you assign to an object. You use the object key to retrieve the object. Version ID Within a bucket, a key and version ID uniquely iden4fy an object. The version ID is a string that is generated when an object is added to a bucket. Value The content that you are storing. An object value can be any sequence of bytes. Metadata A set of name-value pairs with which you can store informa4on regarding the object. You can assign metadata, referred to as user-defined metadata, to objects. S3 also assigns systemmetadata to these objects, which it uses for managing objects. Access Control Informa>on You can control access to the objects you store with S3, both resource-based access controls, such as ACL s and bucket policies, and user-based access control is supported. 12

13 What is an object? X-Ray image Image file stored as jpg/4f.. File system meta data would include; Image name Date/4me created Image owner Extended meta data for object could include; Pa4ent Name Pa4ent Age Scan details Why scan taken? X-Ray Machine used Machine operator How incident occurred. 13

14 S3 Concepts - Buckets S3 buckets are a virtual storage resource which acts as a data repository (similar to a LUN or file system). Objects are stored in buckets Bucket names are globally unique, even in a mul4-region setup. The account that creates a bucket is the owner and can assign permissions as appropriate S3 supports both virtual-hosted style and path-style URLs to access a bucket. In a virtual-hosted style URL, the bucket name is part of the domain name in the URL. For example: hap://bucket.s3.cloudian.com In a path-style URL, the bucket name is not part of the domain (unless you use a region-specific endpoint). For example: hap://s3.cloudian.com/bucket 14

15 S3 Concepts - Keys Each S3 object has data, a key, and metadata. The Object key (or key name) uniquely iden4fies the object in a bucket. When you create an object, you specify the key name, which is a sequence of Unicode characters whose UTF-8 encoding is at most 1024 bytes long. The following are examples of valid object key names: 4my-organiza4on my.great_photos-2014/jan/myvaca4on.jpg videos/2014/birthday/video1.wmv The S3 data model is a flat structure: you create a bucket, and the bucket stores objects. There is no hierarchy of sub-buckets or subfolders; however, you can infer logical hierarchy using keyname prefixes and delimiters. S3 applica4ons supports a concept of folders. Suppose the bucket (companybucket) has four objects with the following object keys: Development/Projects1.xls Finance/statement1.pdf Private/taxdocument.pdf s3-dg.pdf The console uses the keyname prefixes (Development/, Finance/, and Private/) and delimiter ('/') to present a folder structure as shown: 15

16 What it takes to be 100% S3 Compa>ble Compa4bility View API View Basic Opera4ons Moderately Complex Opera4ons Advance Complex Opera4ons

17 S3 vs.. 17

18 Cloud Storage Use Cases Backup and Archive Enterprise backup & store Backup is the classic applica4on for object storage Tape replacement Object storage is much more reliable than tape Object storage is much faster than tape Automated offsite, mul4-site storage Restore anywhere from anywhere Integra4on with con4nuous data replica4on Enterprise long term archiving More cost effec4ve than typical disk solu4on Far greater reliability than tape Overcome corrup4on risks Greater reliability and far lower cost than duplica4ng on tradi4onal storage 18

19 Cloud Storage Use Cases File Distribu>on and Sharing Enterprise file sync and share Any file from any device from anywhere File access, backup, synchroniza4on Enterprise security and control Distributed users no geographic boundaries Remote office file storage Lower cost remote office file shares Central administra4on and control Remote office backup to central data center Content sharing and collabora4on 19

20 Cloud Storage Use Cases Private Cloud Storage So_ware defined storage runs on commodity hardware Integra4on with Amazon, OpenStack and CloudStack Storage for Virtual Machines Templates, Snapshots and backups Storage infrastructure as a Service 20

21 Cloud Storage Use Cases Analy>cs Consumer Activity (Events, GPS, WiFi) Device Tracking and Logs INTERNET OF THINGS Social Media Result of Analysis Benefits ü Faster time-to-decision Event processing plaaorm BIG DATA Fast Efficient Beaer business decisions ü Analyze more allows for efficient bulk data analysis in place Analytics ü No redundant storage of data ü HyperStore scales out with your data adding nodes for I/O ü Take advantage of multi-core CPUs makes sense for MapReduce Cloudian HyperStore ü Can feed smarter data to subsequent analytic systems 21

22 HyperStore 5.2 Smart Data Storage Technology Partner Ecosystem Smart Data Applica>ons High Value (do more) Backup Media and Content Distribu>on File Services Collabora>on Analy>cs Hybrid Cloud Storage Private, On-Premises Storage: Soiware or Appliances Public Cloud Low Cost (with less) 1 Cent per GB a Month QoS Data Center Management Commodity Servers 22

23 THANK YOU Questions? Cloud Storage for Everyone CLOUDIAN HYPERSTORE USE CASES JAN 2015