An Overview of Data Storage on the Cloud

Transcription

1 An Overview of Data Storage on the Cloud Pragya Jain #1, Anita Goel #2, S.C. Gupta # #1 Department of Computer Science, University of Delhi, India 1 prag_2648@yahoo.co.in #2 Department of Computer Science, Dyal Singh College, University of Delhi, India 2 agoel@dsc.du.ac.in # National Informatics Centre, NIC, Delhi, India scgupta@nic.in Abstract Cloud provides highly scalable, self-managed, affordable and fast system for of data. It supports multiple users at the same time in the same infrastructure. Data stored on the cloud is accessible anytime from anywhere via Internet. Usually, the data preferred to be stored on the cloud includes the data not used in daily routine and which requires a large capacity. In this paper, we present an overview of the data stored in the cloud. We discuss the features supported by the cloud for data. Further, we present the ways in which different cloud providers implement the features of cloud. For this, we discuss three data cloud Amazon S, EMC Atmos and Window Azure Blob Storage. We also define the types of data that are preferred for cloud. I. INTRODUCTION User can store and access their data on the cloud from anywhere via Internet. The data stored on the cloud is maintained by the cloud provider who is also responsible for security of the data. Storage of data on the cloud works on the basis of pay-as-you-go model. Cloud [] providers maintain their own geographically distributed data centres in which they use virtualized pools of in order to store user s data and give on lease to the user according to their data requirement. Data stored on the cloud can be deployed publically or privately. Cloud is automatically scalable and less expensive than traditional computer methods if large amount of data is to be stored. It supports multiple users simultaneously in the same infrastructure system in an isolated and secure manner. Cloud supports web service APIs to access the data from anywhere via internet. Data stored on the cloud is self-managed by the cloud. Usually, the data which is not used in daily routine is stored on the cloud such as backup and archiving data and data for the data warehousing and analysis. Cloud is also used to store large objects such as photo, video etc. Some is also required to run the application software on the cloud. The remaining paper is organised as follows: Section II describes deployment of data on the cloud. Section III discusses the traditional of data and the cloud. Section IV discusses general features of cloud. Section V defines the techniques used by different cloud providers to implement these features. Section V describes the types of data that are preferred for on the cloud. II. DATA DEPLOYMENT MODELS ON THE CLOUD There are primarily four most common types of data deployment models on the cloud - public cloud, private cloud, hybrid cloud and community cloud. We discuss data on them in the following subsections. A. Public Cloud Storage Data on the public cloud is offered by a third party vendor. Users can store and access data on the public cloud anytime from anywhere. The user is relieved from the task of investing in physical devices for storing the data. Some cloud providers also provide some services to the users free of charge. The amount of capacity can be extended depending upon the user s need. In public cloud, same infrastructure is shared among multiple users keeping their data secured and isolated from other users. Thus, this model is a better option for data in cases where data sharing is required. Google and Amazon are examples of public cloud providers. B. Private Cloud Storage Data stored on the private cloud is dedicated to a single organization. Usually, the organisation owns the cloud and users belonging only to the organisation, can access the stored data. Private cloud remains behind the organization s firewall. As a result, data is highly secured. C. Hybrid Cloud Storage Hybrid cloud is a combination of public and private cloud. Organizations store their sensitive data on private cloud and avail its security benefits. At the same time, organisations store their non-sensitive and publicly accessible data on public cloud and take costefficiency benefits. Hybrid cloud also helps to handle conditions like cloud-bursting which refers to a scenario where load spike cannot be handled by private cloud. In such cases, private cloud keeps critical and sensitive data and sends non-sensitive data of the organization to the public cloud. The hybrid cloud migrate the workload between private and public cloud without any inconvenience to the user. Microsoft Azure and Force.com are examples of this model. Since hybrid cloud combines two different - one may be on-premise of the organization and other 18

2 somewhere off-premise. In hybrid cloud, there are some technical problems to be faced in joining the two different s and moving the data between them. D. Community Cloud Storage Community cloud is shared by several organizations having same policies, e.g., military, government departments etc. Data stored is managed by the organization itself or by a third party. It helps to reduce cost as compared to private cloud, as shared by a large group. Google s GovCloud is an example of community cloud. III. TRADITIONAL STORAGE SYSTEMS AND CLOUD STORAGE Different computer technologies provide access to the user at different levels of computer data. The data in a computer is generally stored at three levels - (i) Application software, (ii) File system logic, (iii) Physical. Application software level provides interface to the user for storing and accessing data in the form of files. File system logic level defines how the data is stored, retrieved and updated at the physical and also manage free available space. Different techniques, such as, replication, data reduction, etc. are implemented at this level. At physical level, raw block physical is provided, over which the users can implement their own file system logic to store and manage data. A. Traditional Computer Storage System In traditional computer system, the three commonly used technologies are discussed below. Direct-Attached Storage (DAS) is attached directly to a server or workstation. It provides access to the locally at the application software level via different protocols such as SCSI, ATA, and SATA etc. DAS systems are easy to implement and have low maintenance cost. Since capacity is limited to the server, it is difficult to expand the capacity when needed. DAS systems are mainly used for local file sharing environments with a single server or a few servers. Network-Attached Storage (NAS) provides access to the user at file system logic level. A dedicated file server with optimized file system logic is connected to a computer network, providing access to a heterogeneous group of users using some network file sharing protocols such as NFS, CIFS etc. NAS system is useful for small and medium organizations looking for a way to achieve fast data access for multiple clients at file system logic level. Storage Area Network (SAN) provides access to consolidated, block level data via a dedicated network. Users can build their own file system over the block level data provided by SAN. SAN structure allows the user to store large amount of data regardless of the location using protocols such as iscsi, fabric channel etc. B. Cloud Storage Cloud provides data at all three levels of computer data i.e. at application software level, file system logic level and physical level. Cloud delivers the on demand in highly scalable and multitenant way. Cloud uses different APIs such as web service APIs (REST, SOAP); file based APIs, and also traditional protocols (iscsi, NFS, CIFS etc.); to provide access to the data stored on the cloud. Cloud providers provide as a service via Internet, on demand, on the basis of pay-per-use policy. Unlike traditional computer technologies, cloud [1] [2] provides an appearance of infinite capacity available on demand. Hence, users may not plan far ahead for data provisioning. Also, cloud users, need not make an upfront commitment for - the user can start with small and increase it when needed. In cloud service, users pay for the used on a short term basis (typically on a per day basis). Thus, users can take advantage of more capacity when the load increases, without paying any capital cost for provisioning for the unusual conditions such as a load surge and also release the and letting them go when they are no longer useful. IV. FEATURES OF CLOUD STORAGE The cloud providers provide different features for the data stored on the cloud. In this section, we discuss the features supported by most of the cloud providers. A. Manageability User s data on the cloud is stored and managed at the cloud provider s site. Different cloud providers provide different management interfaces to facilitate user to manage the data according to their needs. For example, the user can determine the location at which to store the data or the number of copies of the data maintained at different geographically distributed locations. However, some policies of cloud are automatically implemented on the data when the data is stored on the cloud. Policies defined on the data may vary for different providers B. Access Methods Traditionally, data is accessed using file based protocols such as NFS, CIFS etc. and block based protocols such as iscsi. Internet is a necessary component to access cloud. As the Internet is highly latent, traditional access methods do not work over the Internet. In order to reduce latency and improve performance, data stored on the cloud can be accessed via web service APIs such as REST APIs and SOAP APIs. Web service APIs work on client-server architecture. The APIs use HTTP protocol to send and receive messages between client and server. With the help of web service APIs, user can access cloud anytime from anywhere via Internet. Since data stored on the cloud can also be accessed on a private network or LAN, traditional access methods are also supported for accessing the data. As a result, cloud also supports traditional access methods; such as, file based protocols (NFS, 19

3 CIFS etc.) and block-based protocols (iscsi etc.), in addition to the web service APIs. C. Multi-tenancy Cloud supports multiple users simultaneously and share same infrastructure among them in a secure and isolated manner. Multi-tenancy is implemented by cloud service providers by creating virtual instances of the same infrastructure and by providing virtual instance to the user. D. Scalability Cloud is a scalable system. If there is a load surge, cloud can scale horizontally or vertically by automatically adding new capacity to accommodate the growth. It also has the capability of increasing the bandwidth to the when transfer of data in/out of the cloud is high and also maintains the geographic distribution of data stored. E. Availability When a user has data on the cloud, then cloud provider is responsible to make the data available to the user on request. In case of any error or system failure, such as, site failure or system crash, it is difficult for the cloud provider to provide the data to the user in a reliable way. Different cloud providers use different techniques such as replication, data with erasure coding etc. to increase the user s data availability. Replication technique copies the data at various geographically distributed locations. If a site or copy of the data becomes unavailable due to any error, the data can be made available to the user from some other location. Erasure coding technique divides the data into fragments and adds some coding fragments with it. Fragments are placed in independent fault domains on geographically distributed sites. Original data can be recovered by a subset of total number of fragments. Replication technique has 100% overhead, but easy to implement. Erasure coding has less overhead but complex to implement. F. Efficiency In cloud multiple copies of data are kept. It increases the data access performance and read performance. There is a need to store the data efficiently so that more data can be stored in the available space. Techniques like data compression, deduplication etc. are used by cloud providers to achieve efficiency. V. CLOUD STORAGE PROVIDERS Different cloud providers use different techniques to implement specific features of storing the data on the cloud. Here, we discuss about three data clouds EMC Atmos, Amazon S and Windows Azure Blob. These data clouds provide services to store unstructured data. Their respective vendors - EMC, Amazon and Window Azure also provide other services to support different types of data. A. EMC Atmos EMC Atmos [10] [14] is a cloud platform provided by EMC Corporation. It stores and protects global unstructured data in the form of objects. It distributes the objects in an active paradigm not in hierarchical approach which is common for the typical file system based structures. Data can be accessed via web service based APIs, e.g., REST, SOAP APIs. Atmos also supports traditional file based APIs such as NFS, CIFS, and IFS to access the data within the firewall of Atmos. Data stored on Atmos is self-managed by policy manager. Policy manager treats different types of objects differently and defines policies for each object according to the type of the object. Policies are automatically implemented on the object. Majority of services runs on every node in every rack at every site of its data centre. Hence, there is no single site of failure. If one of the services, a node, an entire rack or site goes down, redundant services facilitate normal operation. EMC Atmos implements secure multi-tenancy. System is configured in one or two tenants and leverages multiple subtenants underneath each tenant who equate to specific customers, divisions or departments, all with different business rules. User ID at the subtenant level identifies an application, or an individual user. EMC Atmos is scalable. To handle load surge or a need for more capacity, two ways are used (1) by adding new cabinets with nodes and disk enclosures, and, (2) by adding nodes and disk enclosures to the existing cabinets. The new capacity is added to up and running cloud automatically and immediately. EMC Atmos supports GeoMirror and GeoParity techniques to protect data from any type of loss and to achieve availability. GeoMirror technique supports two forms of replication (i) Synchronous replication, and (ii) Asynchronous replication. In synchronous replica, an acknowledgement is produced after a successful write. In case of asynchronous replica, no acknowledgement is required. Policy manager defines number of synchronous and asynchronous replica for the object according to the need, and the location where the replica should be stored. GeoParity technique splits the object into m data fragments and k coding fragments placed in independent fault domains. When an object is requested by the client, Atmos decodes the fragment back into the original object using only a minimal subset of m+k fragments. Policy manager also defines services such as compression, deduplication, spinning, striping etc. on the stored data to increase efficiency. B. Amazon S Amazon S [7] [8] is a cloud service provided by Amazon Web Services (AWS). It provides a structure to store and retrieve unstructured data over the web. It stores data in the form of objects. Each object is stored in a bucket. Hence it uses two level of hierarchy to store the data. Data is accessed using REST and SOAP interfaces as well as Bit Torrent Protocol. Users manage their buckets and objects with the help of AWS Management Console [15].The console is a web 20

4 interface which helps the user to easily create bucket, select the region to store buckets, upload files and control access to buckets and objects. Amazon S is highly scalable in nature. It can scale in terms of, request rate and users to support an unlimited number of web application. To achieve availability, it replicates the objects on multiple devices and provides 99.99% availability. S also supports versioning of objects so that user can recover last updated work in case of any failure. It provides a facility of Reduced Redundancy Storage (RRS), which allows users to reduce their cost by storing non-critical and reproducible data at lower level of redundancy than the standard one. C. Windows Azure Blob Storage Window azure blob [9] [1] store unstructured data which can be accessed from anywhere through web using RESTAPI. Azure blob stores the data in the form of two types of blobs Block blob and Page blob, to store the data efficiently. Block blob is used to store streaming data up to 200GB. Page blob stores frequently modified data up to 1TB. Users have to create a account to store their data in Blob Storage. Within the account, a user can create different containers to store data. A account can have size up to 100TB.A user may have more than one account. Azure Blob is auto-scalable. The size can be scaled up to 100TB, transactions up to 5000/ blob/ sec, bandwidth up to GB/sec for a account. To achieve availability, Azure blob keeps three copies of a blob in a data centre in three different nodes across three different fault domains synchronously and also replicates the data asynchronously on another data centre within the same region but at least 500 miles away. This technique is called Geo-replication. Also, Azure blob maintains three copies of the data on another data centre in three different nodes in three different fault domains. Window Azure provides Window Azure management portal for managing the cloud services. The portal allows the user to create, update and deploy blobs and select the region for storing the data. Management Portal gives an option to the user to allow geo-replication of data. TABLE I FEATURES IMPLEMENTED BY DIFFERENT CLOUD STORAGE PROVIDERS S. no 1 2 Featur es Manage ability Access method EMC Atmos Policy manager REST &SOAP API and NFS, CIFS& IFS protocol Multitenancy Tenant(1-2)- >subtenant(multi ple)->user ID Amazon S AWS manageme nt console REST & SOAP API, Bit-Torrent protocol Bucket- >object Azure blob Window Azure Manageme nt Portal REST API Storage account- >container- >blob Scalabil ity Availab ility Efficien cy Add new cabinet/ Add new node & disk enclosure in existing cabinet GeoMirror, GeoParity technique Data compression, deduplication, spinning, striping Scale unlimited capacity Redundant data on multiple devices Reduced redundancy Up to 100 TB for a account Georeplication Block blob and page blob VI. TYPES OF DATA STORED ON THE CLOUD The backup data, archive data etc. [11] are generally preferred to be stored on the cloud. Here, we discuss the different types of data that are stored on the cloud. A. Backup data The increase in enterprise data has resulted in a need of low cost data backup solution. Traditionally, NAS or SAN based private systems and tapes are used for data backup, where, the enterprise manages the backup. Cloud provides an advantage of securely storing the data at some offsite server location. It helps to make the data available even if the user s premise or the local disk is struck with disaster. If backup of large amount of data is to be taken, cloud provides a cost-effective solution. The user need not invest in purchasing physical device only for backup. Cloud helps to access backup data from anywhere via Internet. Most of the cloud providers replicate the data at different locations to improve disaster recovery. B. Archiving data The enterprises need to store some of the data for archive purposes. Usually, archive data is not kept in the infrastructure which is used for daily purposes. Most of the enterprises create a separate archive infrastructure. If cloud is used to store archive data, it preserves the enterprise resource and time to store and manage the data. Cloud is a scalable system and capacity can be increased when required, for storing archiving data. Most of the cloud providers use different data compression techniques so that more data can be stored in less space. C. Data Warehousing And Analysis Data warehouse is a database used for analysing the enterprise s data to help the decision makers to take strategic business decisions. Data warehouse keeps day to day operational data and also the historical data for analysis. Data stored in the data warehouse is used for computing detailed statistics and analysing underlying trends within the data. Cloud makes the organizations free from the task of setting up a data warehouse system. There is no need for the organizations to hire the expertise for managing the data warehouse. Cloud can scale automatically according to the requirement of capacity. As a result, there is no need of over-provisioning and helps to reduce cost. As cloud 21

5 stores the data off-premises, it helps the organizations to recover the data in case of system crash or downtime. D. Content Distribution Web applications and some digital content are accessed by users via Internet. The performance may get affected; as increase in Internet traffic may increase the data transfer latency. Cloud is a good option for storing the data which is required by multiple users, distributed geographically. Cloud replicates such data at the different sites and reduces latency in data transfer. Cloud also provides scalability benefits so that appropriate number of replicas can be distributed over the different cloud sites. E. Storage Of Large Objects Large objects such as photo, video and text documents can be stored on the cloud. As cloud is a scalable system, capacity can be added based on the requirement. Cloud uses different data compression techniques to store the data more efficiently. Cloud stores the objects off-premises, so the user can recover the data in case of any disaster at the premises. Also, user can access their photo and video stored on the cloud from anywhere at any time via Internet. Cloud promises secure and reliable for large objects. F. Databases In The Cloud Cloud provides a cost-effective solution to keep large databases of the enterprise. With the help of cloud, enterprises are made free from the task of building large databases. There is no expertise needed for updating and modifying the database. Cloud automatically manages all the functionality of the database. As data is replicated to multiple data centres, data remains highly available. Any hardware or software failure is automatically handled by the cloud. Due to multi-tenancy, multiple users can access and query the same database stored on the cloud at the same time from different locations via Internet in an isolated and secure manner. Cloud can scale automatically based on the need for database capacity. REFERENCES [1] Armbrust, Fox, Griffith, Joseph, et al. 2009, Above the clouds: 28,EECS Department, U.C. Berkeley, Feb 2009 [2] Armbrust, Fox, Griffith, Joseph, et al.2010, A View of Cloud Computing, Communication of the acm, vol. 5, no. 4, April 2010 [] Mark Carlson, SNIA- Cloud Storage Standards Overview and Research Ideas Brainstorm, CMU SDI Lecture, November 2009 [4] Tim M. Jones. Anatomy of a cloud infrastructure, Models, features, and internals. Nov [5] Scale out with EMC Atmos. Intel Cloud Builders Guide: cloud design and deployment on Intel Platforms, 2010 [6] Xiaoming Gao, Pranav Shah, Adarsh Yoga, Abhijeet Kodgire and Xiao gang Ni. Survey of cloud services provided by different providers, November B649 projects [7] Amazon S, [8] Amazon Web Services, [9] Windows Azure Platform, [10] Michael Wehle. EMC Atmos Cloud Storage Architecture. November EMC Corporation G. Storage for Cloud Computing There is a need of for storing data and intermediate results in order to run an application or software on the cloud. Cloud provides for IaaS, PaaS, and SaaS services. For example, to run payroll software on cloud, the data required to run the application is stored on the cloud. EMC Atmos, Amazon S and Window Azure Blob Storage are cloud providers that are used to store and manage the unstructured data. Table 2 shows different types of data stored on the cloud and the support provided by the data providers for storing the data. The three cloud providers provide services for backup and archiving data, data warehousing and analysis, to store content for geographical distributed access and for of large objects. These cloud providers also take help from other services provided by their respective vendors to support different types of data. TABLE 2 TYPES OF DATA STORED BY DIFFERENT CLOUD STORAGE PROVIDERS S. no Type of data EMC Amazon Azure Atmos S blob 1 Backup Yes Yes Yes 2 Archiving data Yes Yes Yes Data warehousing & analysis Yes Yes Yes 4 Content distribution Yes Yes Yes 5 Storage of large objects Yes Yes Yes 6 Databases in the cloud No No No 7 Storage for cloud computing No No No VII. CONCLUSION Storage of data on the cloud redefines the way that we construct, access, and manage within an enterprise. Cloud providers allow of data on the cloud. The cloud providers provide features that make them more beneficial for storing the data on the cloud than the traditional way. Also some kind of data is more preferable to be stored on the cloud than others. [11] SNIA- Cloud Storage Use Cases, Snia Technical Council and Cloud Storage TWG, June 2009 [12] Joseph Baron,RobertSchneider, Storage Options in the AWS Cloud: Use Cases, Amazon web services, Dec 2010 [1] Bard Calder et al., Window Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency, SOSP: Proceedings of 2 rd ACM Symposium on Operating Systems Principles, Oct 2011 [14] [15] [16] Zhou ke, Wang huaand LIChunhua. Cloud Storage Technology and Its Application, ZTE Communications, (4), pp [17] Zhang Jian-hua and Zhang Nan. Cloud Computing-based Data Storage and Disaster Recovery, International Conference on Future Computer Science and Education, 2011, pp [18] A. Aljabr, Cloud Computing for Increased Business Value, International Journal of Business &Social Science, Vol. No.1, 2012 [19] Cong Wang et al., Towards Secure And Dependable Storage Services in Cloud Computing, IEEE Trans. on Services Computing, June