Storage Virtualisation in the Cloud

Transcription

1 PRESENTATION TITLE GOES HERE Storage Virtualisation in the Cloud Bob Plumridge SNIA Europe Chair HDS

2 Overview What is Storage Virtualisation? Cloud Storage Virtualisation Use Cases CDMI CDMI and CIMI 2

3 What is Storage Virtualisation? Virtualisation presents a familiar interface to existing software, but changes the implementation behind that interface Example: x86 architecture Virtual machines look like a dedicated CPU Example: SCSI disk drive Virtualized volumes spanning multiple disks in a RAID device Virtualisation can be on Block, File or Object interfaces Implication is that additional services are being provided behind that interface that add value to the data being stored 3

4 What Is Cloud Storage Used For? Elastic demand for web based media (video, ebooks, audio) Backup to the cloud Restore, Recovery, Seed the backup with hard drive Sync of files to the cloud and multiple devices Internet Drive secondary storage Archive/Preservation in the cloud Including Compliance, Retention and ediscovery Storage for Cloud Computing Support for legacy storage interfaces key Self Service Storage Administration 4

5 Data Storage Interface Evolution Born Digital data is being generated by an ever diversifying set of devices 75% of data being generated by individuals (IDC) Amount of Data doubling every two years 5 Billion Mobile Phones Tablets on the rise Storage space on devices will always be limited (although continues to grow exponentially) Device Apps are accessing content/storage via the Internet Cloud Storage largely driven by device based use cases Traditional data storage interfaces (filesystems) are not up to this task! 5

6 Where are your users putting their data? Source: Amazon With corporate users bringing their own devices (BYOD), where are they putting files? Devices have limited storage capacity They are saving them in the storage cloud! Corporate data is already in the public cloud and unmanaged/protected Source: Oxygen Cloud 6

7 Distilled Requirements Need a Global Namespace for data location FILE handle -> URL + Unique Object ID globally Need support for Rich Metadata associated with the data Plus Metadata Query and Index based Search Need HTTP access browser support Data Services need to be transparent to the user, but extensible and powerful Driven by Metadata Support for large objects, many small objects Support for active preservation operations Driven by preservation Metadata 7

8 Build Your Own Cloud Corporations already offer their own (instant messaging, etc.) services to their employees Cloud Storage is the next type of service offering for employee devices Enterprise Dropbox implementations Essentially a private cloud with access over the Internet from corporate and other Apps Data is retained in house, protected, available, secured and compliant 8

9 Post-Filesystem Data Storage The SNIA has produced the Cloud Data Management Interface (CDMI) standard for post-filesystem Data Storage i.e. Public and Private/Hybrid Storage Clouds Specifically designed to meet the requirements (for the foreseeable future) of contemporary data usage and management Superset of features currently being offered, but shrink to fit Innovative use of Metadata to express Data Requirements Requirements are then met by the implemented Data Services CDMI can be (and is being) extended compatibly To accommodate new types of data requirements, use cases 9

10 How do you find data? Most popular data access application today? ! Data is saved chronologically, searchable via metadata attachments come with rich metadata Subject, Sender, Recipient, Date, Thread, Priority Future Data Storage Interfaces will leverage this concept for general file/object storage Not only for locating data, but for managing it as well CDMI provides a standard Data Storage Interface with rich metadata query and global namespace 10

11 Where is it being used? Commercial Cloud Storage vendors started announcing implementations in April 2011 Increasing Adoption by escience and Academic community Especially in Europe (OpenNebula open source cloud, others) Quarterly Plugfests with growing participation > September plugfest in Europe and North America CDMI is now an International Standard ISO/IEC 17826:2012 CDMI has a reference implementation (Java under BSD) Downloaded by hundreds of developers worldwide 11

12 CDMI Overview Data Portability Standard Move Data (and most importantly Metadata) from cloud to cloud Advanced Cloud Services Data System Metadata allows cloud vendors to up-sell! Specialized storage clouds for specific use cases Logging, Security, Audit Trails Extensible to accommodate rapid innovation in cloud market Proposed Extensions: CIMI/OVF, Versioning, Jobs Shrink-to-fit Only implement what makes sense for your cloud Profiles: Simple Storage (i.e. S3), Simple Storage Management (NAS) 12

13 CDMI Overview CDMI Basic flow: 13

14 Model for the Interface The resources which are accessed through the RESTful interface 14

15 Additional Challenges Lack of uniform semantics and standard interfaces Interoperability between public cloud providers Managing data format changes over time Authenticity verification Compliance and Governance Risk Management & Litigation 15

16 A new class of data migration challenges? Cloud A Cloud B Data over WAN via vendor specific API s 16

17 Cloud storage is not going away IDC. Worldwide Storage in the Cloud Forecast: The expanding role of Public Cloud Storage Services 17

18 Cloud Peering 18

19 Separate Infrastructure Implementations Cloud Storage for Cloud Computing uses virtualised interfaces Virtual machines use virtualised storage in the infrastructure cloud

20 Proposal If there are sufficient members that see a situation where we need separate implementations, we need to allow for the storage connections information to be exposed. Add the connectivity and credential information to the APIs but make it optional to implement (in the common infrastructure case). Make it as simple as possible

21 For More information SNIA Technical Work Groups: & Large Cloud Storage Community SNIA Cloud Blog Site: 21