Synchronization of Data and Resources in Distributed, Cooperative Virtual Data Centers

Size: px
Start display at page:

Download "Synchronization of Data and Resources in Distributed, Cooperative Virtual Data Centers"

Transcription

1 Vol.110 (ISI 2015), pp Synchronization of Data and Resources in Distributed, Cooperative Virtual Data Centers Eun-Kyu Lee Dept. of Information and Telecommunication Engineering Incheon National University, Incheon, Korea Abstract. The cooperation is to reduce both capital and operational expenditures in running a data center while providing a greener solution. In the aspect of performance in such a distributed solution, performance can be improved by replicating the server and subsequently dividing the work. But, the main concern is keeping all replicas up-to-date, and we have observed a variety of synchronous and consistency models. This paper investigates a few synchronous schemes for a cooperative virtual data center between multiple entities which is founded on the principal of fair resource sharing among the entities. Keywords: Virtual data center, cloud computing, reliability, fault tolerance, synchronization 1 Introduction Existing data centers are each individually owned and operated by a single entity. This situation creates an excessive financial burden upon each entity through the need to over-provision for hardware. While this state of affairs enables a secure and an efficient maintenance scheme for the data center, the financial drawbacks are perhaps excessive. Instead, we propose the idea of a cooperative virtual data center between multiple entities which is founded on the principal of fair resource sharing among the entities. Conceptually, each data center can reduce the amount of over-provisioning of resources by cooperatively sharing computing power with other data centers. Since data centers generally have transient and predictable periods of peak load, it is conceivable that a data center can borrow computing resources, in the form of virtual machines, from other data centers which have spare resources for shortlived periods. Moreover, this form of cooperative sharing of computing resources provides a level of topographically distributed fault tolerance as the cooperating data centers may be geographically separated from one another. The reduced provisioning of servers has the added benefit of diminishing operating costs through abating energy requirements and minimizing required support personnel. Furthermore, cooperative VDC enable the active use of idle resources thereby creating a green computing solution [6]. According to [3], cooperative VDC have the potential to tackle 85% of the current costs associated ISSN: ASTL Copyright 2015 SERSC

2 Vol (ISI E.-K. 2015) Lee with running a data center in terms of servers, infrastructure, and power draw. Cooperative VDC, nonetheless, suffer from reliability issues that come from the fact that the cooperation runs on a distributed, multi-ownership environment. The main goal of this paper is to promote the feasibility of a cooperative virtual data center among several independent entities which can still maintain data reliably under minimal assumptions. The paper is organized as follows. Section 2 provides the motivation and background behind our work and Section 3 introduce our proposed architecture with challenges. Section 4 discusses the main approach of our solution. Finally, Section 5 presents our conclusions. 2 Data Centers It is worth noting that there are very few giant-sized data centers comprised of tens to hundreds of thousands of servers [7]. Most data centers are much smaller, on the order of hundreds to thousands of servers instead, run by mid-sized to small companies. Even with a few hundred servers, the financial commitment becomes a large burden for these more constrained companies. Data centers, on average, use far less resources than which they are equipped for. This is in large part due to the over-provisioning by data centers in situations of peak workloads, which are transient and often predictable. The redundancy which is provided for by data centers is unnecessary in most situations, but the operators of the data center must pay the cost of supporting this redundancy year-long. Energy costs for powering and cooling idle servers cannot be avoided in such cases. 3 Cooperation among Multiple Data Centers The cooperation is to reduce both capital and operational expenditures in running a data center while providing a greener solution. The perception of the cooperative virtual data center operates around the notion that a data center is capable of utilizing heterogeneous idle resources form other private data centers when needed in periods of peak workload in order reduce over-provisioning. This allows for a reduction in capital, thereby propagating to reductions in operational expenditures. This form of cooperative resource sharing provides for better data center cost effectiveness. It is currently envisioned that there would be an alliance of a group of small to mid-sized companies which each have non-conflicting periods of known peak load. The alliance membership size is perceived as being no more than 10 to 20 participants, perhaps even less. Moreover, an added benefit of this scheme enables fault tolerance if the members of the alliance happen to have their data centers geographically segregated. This reconstruction the virtual data center breaks sharply from one proposed by [6] in the sense that the physical data centers which form a virtual data center are each individually operated by a separate entity which originates as part of a corporation s private cloud. This differentiation, while minor changes the fundamental dynamics of data 14 Copyright 2015 SERSC

3 Synchronization in Cooperative DataVol.110 Centers (ISI 2015) 3 Full connec)on? Company A Company C A Internet B Company B C Fig. 1: Independently-operated data centers Fig. 2: A model of cooperative virtual data center Company A VM A VM B VM C Company C Tamper- proof VM Monitor Private storages Company B Fig. 3: Realization of the cooperative virtual data center center cost structures, financial risks, and utilization. Independently operated data centers will eventually move to a model of the cooperative virtual data centers where multiple data centers together build a single virtual computing platform by sharing their resources. Fig. 1 and Fig. 2 illustrate the transition. Challenges. Cooperation among individual data centers while far more efficient than operating separate data centers does have a number of drawbacks. The most common issues with such distributed solutions range into the category of reliability. If a company needs to borrow resources from the collective pool and there are no resources available, then this can become a critical situation of under-provisioning. In such situations, the lack of reliability can have disastrous economic consequences. There must be a recovery mechanism in place when situations arise where communal resources are not available. Furthermore, synchronization issues between the data centers can also result in unreliable or unanticipated situations. 4 Synchronization of Replicated Data The primary architecture underlying the cooperative virtual data center is shown in Fig. 3. Here we see three companies, A, B, and C, where company A and C are shown to be borrowing resources from company B through a leasing of virtual machines. Company A and C both store their private data on their own dedicated hard disks located within the storage area network (SAN) of Copyright 2015 SERSC 15

4 4Vol.110 (ISI E.-K. 2015) Lee company B. It is expected that these disks would likely be placed into a more cost-effective iscsi SAN connected to the physical servers using fibre channel host bus adapters, where each company is expected to provide their own hard disks. The choice here of using an iscsi approach over that of fibre channel primarily involves the cost factor of deployment with commodity resources and the expected access patterns for the storage. Adoption of fibre channel can also be justified depending upon the expected needs of the system. Nonetheless, the actual underlying implementation of the SAN technology is up to the company hosting each private storage disk. The virtual machine monitor (VMM) is responsible for ensuring that a running virutal machine (VM) can only accesses disks which the VM owns. In this case, V M A, run by company A can only access the disk drives which company A has placed in company B, henceforth, V M C would not be able to access the disk of company A. In our proposed implementation, Xen 1 is assumed to be the hypervisor used to perform these domain security validations and access checks to disk since it enables high performance buffered I/O transfers using asynchronous buffer-descriptor rings [1]. In the followings we focus on discussing our solutions to the reliability and synchronization issues with the solution. Synchronization Two primary reasons for replicating data among data centers are for reliability and performance. In the aspect of reliability, data centers can continue operating after one of their replicas crashes by simply switch to one of the other replicas; also, it becomes possible to provide better protection against corrupted data. In the aspect of performance, when the number of processes attempting to access data managed by a server increases, performance can be improved by replicating the server and subsequently dividing the work; additionally, a copy of data can be placed in the proximity of the process accessing the information to reduce the time of data access. However, main concern is keeping all replicas up-to-date. Intuitively, a collection of copies is consistent when the copies are always the same, that is, a read operation performed at any copy will always return the same results. Consequently, when an update operation is performed on one copy, the update should be propagated to all copies before a subsequent operation takes place. Achieving such a tight consistency incurs high cost because updates need to be executed as atomic operations, and global synchronization is required. The synchronization scheme used in GFS [2] works this way with acceptable delay only because the replicas are stored within a local area of the primary copy which does not necessarily translate to the cooperative VDC case. The only real solution is to loosen the consistency constraints. Various consistency models have already been proposed and used for replication in distributed systems [8]. A simple synchronization scheme. Synchronization is done in two steps, the first for detecting files that have been modified since the last run, and the second for propagating the updated data. In the worst case, i.e. when all nodes Copyright 2015 SERSC

5 Synchronization Advanced in Cooperative Science and Data Technology Centers Letters 5 Vol.110 (ISI 2015) Fig. 4: Three rounds can synchronize eight nodes. have updated files, an all-to-all communication is needed. In a naive approach, each node would send its updates to all partners, resulting in O(n(n 1)/2) updates. Each single connection would trigger an independent local disk access, provoking many updates concurrently and therefore resulting in a slow data transfer rate. To avoid this, [8] proposed an efficient synchronization scheme which uses only node-to-node (n2n) syncs. In this scheme, each node participates in at most one n2n sync at a time. Therefore, at most n/2 n2n syncs are run concurrently. Based on the fact that in each n2n sync a node propagates not only the modifications made to its own data, but also the modifications it received from other nodes in earlier rounds of the same sync. In [8], each node needs a maximum of log(n) n2n syncs in a complete graph. The sync process is given by a list of rounds of parallel n2n syncs. No barrier operation is executed between the rounds and therefore rounds may overlap. Fig. 4 depicts the synchronization of eight nodes. The process is split into three rounds. Each of them contains four parallel N2N syncs: {A B, C D, E F, G H}, {A C, B D, E G, F H}, {A E, B F, C G, D H}. Gossip in complete graphs. For optimal synchronization based on n2n synchronization gossip algorithms [8] can be also used. The constant model takes only the startup cost of a connection into account. In the linear model the communication cost c is proportional to the size l of the data volume: c = β + lτ, where β is the startup cost and τ is the transfer time of a unit-length message. τ is assumed to be constant for all links. Determining a cost-optimal gossip plan for arbitrary graphs is an NP-hard problem [5]. We therefore simplify this problem by not supporting arbitrary graphs but only hierarchies of regular graph classes. Additionally, we treat network hubs like switches among data centers, thereby neglecting possible hub congestion. With these two restrictions, we can generates optimal plans for some classes of graphs, which are hierarchically composed by heuristics to allow for arbitrary networks. Complete homogeneous graphs can be used to model switched networks. For this purpose, we use the optimal algorithm described in [4]. It needs at most log n rounds in graphs with an even (resp. odd) number of nodes. Depending on the amount of updated repositories, the synchronization time varies between O(n log n) for a one-to-all broadcast and O(n) for an all-to-all broadcast. Similar algorithms for other graphs like rings and busses are also known [4], but they are only optimal in the constant model where the link bandwidth is ignored. Copyright 2015 SERSC 17

6 Vol.110 (ISI 2015) 6 E.-K. Lee 5 Conclusion This paper proposed the idea of a cooperative virtual data center (CVDC) between multiple entities, which is founded on the principal of fair resource sharing among the entities. This new architecture also enables a secure and an efficient maintenance scheme for the data center. To realize this novel architecture, we discussed the main reliability fault of a CVDC and corresponding agreement issues between data centers. From this, we proposed a new research direction to provide a certain level of resource availability and reliability. Therefore, based on our realization discussion, we believe that our architecture is feasible and will reduce capital and operational costs. References 1. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In ACM symposium on Operating systems principles, S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. In ACM SOSP, A. Greenberg, J. Hamilton, D. A. Maltz, and P. Patel. The Cost of a Cloud: Research Problems in Data Center Networks. ACM SIGCOMM Computer Communication Review, 39(1):68 73, January J. Hromkovic, C. Klasing, B. Monien, and R. Peine. Dissemination of information in interconnection networks. In Combinatorial Network Theory, pages , D. W. Krumme, G. Cybenko,, and K. N. Venkataraman. Gossiping in minimal time. In SIAM J. Comput., pages , E. M. Maximilien. Green Computing. University of California, Los Angeles, June R. Miller. Who Has the Most Web Servers? com/archives/2009/05/14/whos-got-the-most-web-servers/, May T. Schutt, F. Schintke, and A. Reinefeld. Efficient synchronization of replicated data in distributed systems. In Prentice-Hall Inc, Copyright 2015 SERSC

Cooperative Virtual Data Center: Sharing Data and Resources among Multiple Computing Entities

Cooperative Virtual Data Center: Sharing Data and Resources among Multiple Computing Entities , pp. 137-152 http://dx.doi.org/10.14257/ijseia.2015.9.11.13 Cooperative Virtual Data Center: Sharing Data and Resources among Multiple Computing Entities Eun-Kyu Lee Dept. of Information and Telecommunication

More information

Efficient Synchronization of Replicated Data in Distributed Systems

Efficient Synchronization of Replicated Data in Distributed Systems Efficient Synchronization of Replicated Data in Distributed Systems Thorsten Schütt, Florian Schintke, Alexander Reinefeld Zuse Institute Berlin (ZIB) Abstract. We present nsync, a tool for synchronizing

More information

Efficient and Enhanced Load Balancing Algorithms in Cloud Computing

Efficient and Enhanced Load Balancing Algorithms in Cloud Computing , pp.9-14 http://dx.doi.org/10.14257/ijgdc.2015.8.2.02 Efficient and Enhanced Load Balancing Algorithms in Cloud Computing Prabhjot Kaur and Dr. Pankaj Deep Kaur M. Tech, CSE P.H.D prabhjotbhullar22@gmail.com,

More information

Migration of Virtual Machines for Better Performance in Cloud Computing Environment

Migration of Virtual Machines for Better Performance in Cloud Computing Environment Migration of Virtual Machines for Better Performance in Cloud Computing Environment J.Sreekanth 1, B.Santhosh Kumar 2 PG Scholar, Dept. of CSE, G Pulla Reddy Engineering College, Kurnool, Andhra Pradesh,

More information

Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT

Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT S. Gobinath, S. Saravanan PG Scholar, CSE Dept, M.Kumarasamy College of Engineering, Karur, India 1 Assistant Professor,

More information

Xen Live Migration. Networks and Distributed Systems Seminar, 24 April 2006. Matúš Harvan Xen Live Migration 1

Xen Live Migration. Networks and Distributed Systems Seminar, 24 April 2006. Matúš Harvan Xen Live Migration 1 Xen Live Migration Matúš Harvan Networks and Distributed Systems Seminar, 24 April 2006 Matúš Harvan Xen Live Migration 1 Outline 1 Xen Overview 2 Live migration General Memory, Network, Storage Migration

More information

The Google File System

The Google File System The Google File System By Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung (Presented at SOSP 2003) Introduction Google search engine. Applications process lots of data. Need good file system. Solution:

More information

CS2510 Computer Operating Systems

CS2510 Computer Operating Systems CS2510 Computer Operating Systems HADOOP Distributed File System Dr. Taieb Znati Computer Science Department University of Pittsburgh Outline HDF Design Issues HDFS Application Profile Block Abstraction

More information

CS2510 Computer Operating Systems

CS2510 Computer Operating Systems CS2510 Computer Operating Systems HADOOP Distributed File System Dr. Taieb Znati Computer Science Department University of Pittsburgh Outline HDF Design Issues HDFS Application Profile Block Abstraction

More information

International Journal of Advance Research in Computer Science and Management Studies

International Journal of Advance Research in Computer Science and Management Studies Volume 2, Issue 8, August 2014 ISSN: 2321 7782 (Online) International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online

More information

Distributed File Systems

Distributed File Systems Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.

More information

CHAPTER 7 SUMMARY AND CONCLUSION

CHAPTER 7 SUMMARY AND CONCLUSION 179 CHAPTER 7 SUMMARY AND CONCLUSION This chapter summarizes our research achievements and conclude this thesis with discussions and interesting avenues for future exploration. The thesis describes a novel

More information

36 January/February 2008 ACM QUEUE rants: feedback@acmqueue.com

36 January/February 2008 ACM QUEUE rants: feedback@acmqueue.com 36 January/February 2008 ACM QUEUE rants: feedback@acmqueue.com Virtu SCOTT RIXNER, RICE UNIVERSITY Network alization Shared I/O in ization platforms has come a long way, but performance concerns remain.

More information

CURTAIL THE EXPENDITURE OF BIG DATA PROCESSING USING MIXED INTEGER NON-LINEAR PROGRAMMING

CURTAIL THE EXPENDITURE OF BIG DATA PROCESSING USING MIXED INTEGER NON-LINEAR PROGRAMMING Journal homepage: http://www.journalijar.com INTERNATIONAL JOURNAL OF ADVANCED RESEARCH RESEARCH ARTICLE CURTAIL THE EXPENDITURE OF BIG DATA PROCESSING USING MIXED INTEGER NON-LINEAR PROGRAMMING R.Kohila

More information

Massive Data Storage

Massive Data Storage Massive Data Storage Storage on the "Cloud" and the Google File System paper by: Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung presentation by: Joshua Michalczak COP 4810 - Topics in Computer Science

More information

Dynamic resource management for energy saving in the cloud computing environment

Dynamic resource management for energy saving in the cloud computing environment Dynamic resource management for energy saving in the cloud computing environment Liang-Teh Lee, Kang-Yuan Liu, and Hui-Yang Huang Department of Computer Science and Engineering, Tatung University, Taiwan

More information

Efficient Data Replication Scheme based on Hadoop Distributed File System

Efficient Data Replication Scheme based on Hadoop Distributed File System , pp. 177-186 http://dx.doi.org/10.14257/ijseia.2015.9.12.16 Efficient Data Replication Scheme based on Hadoop Distributed File System Jungha Lee 1, Jaehwa Chung 2 and Daewon Lee 3* 1 Division of Supercomputing,

More information

Distributed File Systems

Distributed File Systems Distributed File Systems Mauro Fruet University of Trento - Italy 2011/12/19 Mauro Fruet (UniTN) Distributed File Systems 2011/12/19 1 / 39 Outline 1 Distributed File Systems 2 The Google File System (GFS)

More information

A Dynamic Resource Management with Energy Saving Mechanism for Supporting Cloud Computing

A Dynamic Resource Management with Energy Saving Mechanism for Supporting Cloud Computing A Dynamic Resource Management with Energy Saving Mechanism for Supporting Cloud Computing Liang-Teh Lee, Kang-Yuan Liu, Hui-Yang Huang and Chia-Ying Tseng Department of Computer Science and Engineering,

More information

A Study on Data Analysis Process Management System in MapReduce using BPM

A Study on Data Analysis Process Management System in MapReduce using BPM A Study on Data Analysis Process Management System in MapReduce using BPM Yoon-Sik Yoo 1, Jaehak Yu 1, Hyo-Chan Bang 1, Cheong Hee Park 1 Electronics and Telecommunications Research Institute, 138 Gajeongno,

More information

INCREASING SERVER UTILIZATION AND ACHIEVING GREEN COMPUTING IN CLOUD

INCREASING SERVER UTILIZATION AND ACHIEVING GREEN COMPUTING IN CLOUD INCREASING SERVER UTILIZATION AND ACHIEVING GREEN COMPUTING IN CLOUD M.Rajeswari 1, M.Savuri Raja 2, M.Suganthy 3 1 Master of Technology, Department of Computer Science & Engineering, Dr. S.J.S Paul Memorial

More information

Parallel Computing. Benson Muite. benson.muite@ut.ee http://math.ut.ee/ benson. https://courses.cs.ut.ee/2014/paralleel/fall/main/homepage

Parallel Computing. Benson Muite. benson.muite@ut.ee http://math.ut.ee/ benson. https://courses.cs.ut.ee/2014/paralleel/fall/main/homepage Parallel Computing Benson Muite benson.muite@ut.ee http://math.ut.ee/ benson https://courses.cs.ut.ee/2014/paralleel/fall/main/homepage 3 November 2014 Hadoop, Review Hadoop Hadoop History Hadoop Framework

More information

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

More information

Relational Databases in the Cloud

Relational Databases in the Cloud Contact Information: February 2011 zimory scale White Paper Relational Databases in the Cloud Target audience CIO/CTOs/Architects with medium to large IT installations looking to reduce IT costs by creating

More information

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes

More information

Load Rebalancing for File System in Public Cloud Roopa R.L 1, Jyothi Patil 2

Load Rebalancing for File System in Public Cloud Roopa R.L 1, Jyothi Patil 2 Load Rebalancing for File System in Public Cloud Roopa R.L 1, Jyothi Patil 2 1 PDA College of Engineering, Gulbarga, Karnataka, India rlrooparl@gmail.com 2 PDA College of Engineering, Gulbarga, Karnataka,

More information

A Distributed Storage Architecture based on a Hybrid Cloud Deployment Model

A Distributed Storage Architecture based on a Hybrid Cloud Deployment Model A Distributed Storage Architecture based on a Hybrid Cloud Deployment Model Emigdio M. Hernandez-Ramirez, Victor J. Sosa-Sosa, Ivan Lopez-Arevalo Information Technology Laboratory Center of Research and

More information

AN ADAPTIVE DISTRIBUTED LOAD BALANCING TECHNIQUE FOR CLOUD COMPUTING

AN ADAPTIVE DISTRIBUTED LOAD BALANCING TECHNIQUE FOR CLOUD COMPUTING AN ADAPTIVE DISTRIBUTED LOAD BALANCING TECHNIQUE FOR CLOUD COMPUTING Gurpreet Singh M.Phil Research Scholar, Computer Science Dept. Punjabi University, Patiala gurpreet.msa@gmail.com Abstract: Cloud Computing

More information

A Reputation Replica Propagation Strategy for Mobile Users in Mobile Distributed Database System

A Reputation Replica Propagation Strategy for Mobile Users in Mobile Distributed Database System A Reputation Replica Propagation Strategy for Mobile Users in Mobile Distributed Database System Sashi Tarun Assistant Professor, Arni School of Computer Science and Application ARNI University, Kathgarh,

More information

Journal of science STUDY ON REPLICA MANAGEMENT AND HIGH AVAILABILITY IN HADOOP DISTRIBUTED FILE SYSTEM (HDFS)

Journal of science STUDY ON REPLICA MANAGEMENT AND HIGH AVAILABILITY IN HADOOP DISTRIBUTED FILE SYSTEM (HDFS) Journal of science e ISSN 2277-3290 Print ISSN 2277-3282 Information Technology www.journalofscience.net STUDY ON REPLICA MANAGEMENT AND HIGH AVAILABILITY IN HADOOP DISTRIBUTED FILE SYSTEM (HDFS) S. Chandra

More information

UPS battery remote monitoring system in cloud computing

UPS battery remote monitoring system in cloud computing , pp.11-15 http://dx.doi.org/10.14257/astl.2014.53.03 UPS battery remote monitoring system in cloud computing Shiwei Li, Haiying Wang, Qi Fan School of Automation, Harbin University of Science and Technology

More information

Scaling Cloud Storage. Julian Chesterfield Storage & Virtualization Architect

Scaling Cloud Storage. Julian Chesterfield Storage & Virtualization Architect Scaling Cloud Storage Julian Chesterfield Storage & Virtualization Architect Outline Predicting Cloud IO Workloads Identifying the bottlenecks The distributed SAN approach OnApp s integrated storage platform

More information

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl Big Data Processing, 2014/15 Lecture 5: GFS & HDFS!! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl 1 Course content Introduction Data streams 1 & 2 The MapReduce paradigm Looking behind

More information

Designing a Cloud Storage System

Designing a Cloud Storage System Designing a Cloud Storage System End to End Cloud Storage When designing a cloud storage system, there is value in decoupling the system s archival capacity (its ability to persistently store large volumes

More information

ioscale: The Holy Grail for Hyperscale

ioscale: The Holy Grail for Hyperscale ioscale: The Holy Grail for Hyperscale The New World of Hyperscale Hyperscale describes new cloud computing deployments where hundreds or thousands of distributed servers support millions of remote, often

More information

Snapshots in Hadoop Distributed File System

Snapshots in Hadoop Distributed File System Snapshots in Hadoop Distributed File System Sameer Agarwal UC Berkeley Dhruba Borthakur Facebook Inc. Ion Stoica UC Berkeley Abstract The ability to take snapshots is an essential functionality of any

More information

Enhancing the Performance of Live Migration of Virtual Machine s with WSClock Replacement Algorithm

Enhancing the Performance of Live Migration of Virtual Machine s with WSClock Replacement Algorithm Enhancing the Performance of Live Migration of Virtual Machine s with WSClock Replacement Algorithm C.Sagana M.Geetha Dr R.C.Suganthe PG student, Assistant Professor, Professor, Dept of CSE, Dept of CSE

More information

Virtualization benefits in High Performance Computing Applications

Virtualization benefits in High Performance Computing Applications Journal of Computer Science and Information Technology June 2014, Vol. 2, No. 2, pp. 101-109 ISSN: 2334-2366 (Print), 2334-2374 (Online) Copyright The Author(s). 2014. All Rights Reserved. Published by

More information

Keywords Distributed Computing, On Demand Resources, Cloud Computing, Virtualization, Server Consolidation, Load Balancing

Keywords Distributed Computing, On Demand Resources, Cloud Computing, Virtualization, Server Consolidation, Load Balancing Volume 5, Issue 1, January 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Survey on Load

More information

Resource Scalability for Efficient Parallel Processing in Cloud

Resource Scalability for Efficient Parallel Processing in Cloud Resource Scalability for Efficient Parallel Processing in Cloud ABSTRACT Govinda.K #1, Abirami.M #2, Divya Mercy Silva.J #3 #1 SCSE, VIT University #2 SITE, VIT University #3 SITE, VIT University In the

More information

High Availability for Database Systems in Cloud Computing Environments. Ashraf Aboulnaga University of Waterloo

High Availability for Database Systems in Cloud Computing Environments. Ashraf Aboulnaga University of Waterloo High Availability for Database Systems in Cloud Computing Environments Ashraf Aboulnaga University of Waterloo Acknowledgments University of Waterloo Prof. Kenneth Salem Umar Farooq Minhas Rui Liu (post-doctoral

More information

A Novel Method for Resource Allocation in Cloud Computing Using Virtual Machines

A Novel Method for Resource Allocation in Cloud Computing Using Virtual Machines A Novel Method for Resource Allocation in Cloud Computing Using Virtual Machines Ch.Anusha M.Tech, Dr.K.Babu Rao, M.Tech, Ph.D Professor, MR. M.Srikanth Asst Professor & HOD, Abstract: Cloud computing

More information

Cloud Based Application Architectures using Smart Computing

Cloud Based Application Architectures using Smart Computing Cloud Based Application Architectures using Smart Computing How to Use this Guide Joyent Smart Technology represents a sophisticated evolution in cloud computing infrastructure. Most cloud computing products

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

High Availability with Windows Server 2012 Release Candidate

High Availability with Windows Server 2012 Release Candidate High Availability with Windows Server 2012 Release Candidate Windows Server 2012 Release Candidate (RC) delivers innovative new capabilities that enable you to build dynamic storage and availability solutions

More information

Cyber Forensic for Hadoop based Cloud System

Cyber Forensic for Hadoop based Cloud System Cyber Forensic for Hadoop based Cloud System ChaeHo Cho 1, SungHo Chin 2 and * Kwang Sik Chung 3 1 Korea National Open University graduate school Dept. of Computer Science 2 LG Electronics CTO Division

More information

A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems

A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems Aysan Rasooli Department of Computing and Software McMaster University Hamilton, Canada Email: rasooa@mcmaster.ca Douglas G. Down

More information

Scalable Multiple NameNodes Hadoop Cloud Storage System

Scalable Multiple NameNodes Hadoop Cloud Storage System Vol.8, No.1 (2015), pp.105-110 http://dx.doi.org/10.14257/ijdta.2015.8.1.12 Scalable Multiple NameNodes Hadoop Cloud Storage System Kun Bi 1 and Dezhi Han 1,2 1 College of Information Engineering, Shanghai

More information

Load Balancing in Distributed Data Base and Distributed Computing System

Load Balancing in Distributed Data Base and Distributed Computing System Load Balancing in Distributed Data Base and Distributed Computing System Lovely Arya Research Scholar Dravidian University KUPPAM, ANDHRA PRADESH Abstract With a distributed system, data can be located

More information

IMPLEMENTATION OF VIRTUAL MACHINES FOR DISTRIBUTION OF DATA RESOURCES

IMPLEMENTATION OF VIRTUAL MACHINES FOR DISTRIBUTION OF DATA RESOURCES INTERNATIONAL JOURNAL OF ADVANCED RESEARCH IN ENGINEERING AND SCIENCE IMPLEMENTATION OF VIRTUAL MACHINES FOR DISTRIBUTION OF DATA RESOURCES M.Nagesh 1, N.Vijaya Sunder Sagar 2, B.Goutham 3, V.Naresh 4

More information

Building the Virtual Information Infrastructure

Building the Virtual Information Infrastructure Technology Concepts and Business Considerations Abstract A virtual information infrastructure allows organizations to make the most of their data center environment by sharing computing, network, and storage

More information

IMPROVED PROXIMITY AWARE LOAD BALANCING FOR HETEROGENEOUS NODES

IMPROVED PROXIMITY AWARE LOAD BALANCING FOR HETEROGENEOUS NODES www.ijecs.in International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 2 Issue 6 June, 2013 Page No. 1914-1919 IMPROVED PROXIMITY AWARE LOAD BALANCING FOR HETEROGENEOUS NODES Ms.

More information

Energy Conscious Virtual Machine Migration by Job Shop Scheduling Algorithm

Energy Conscious Virtual Machine Migration by Job Shop Scheduling Algorithm Energy Conscious Virtual Machine Migration by Job Shop Scheduling Algorithm Shanthipriya.M 1, S.T.Munusamy 2 ProfSrinivasan. R 3 M.Tech (IT) Student, Department of IT, PSV College of Engg & Tech, Krishnagiri,

More information

Principles and characteristics of distributed systems and environments

Principles and characteristics of distributed systems and environments Principles and characteristics of distributed systems and environments Definition of a distributed system Distributed system is a collection of independent computers that appears to its users as a single

More information

Virtualization for Future Internet

Virtualization for Future Internet Virtualization for Future Internet 2010.02.23 Korea University Chuck Yoo (hxy@os.korea.ac.kr) Why Virtualization Internet today Pro and con Your wonderful research results Mostly with simulation Deployment

More information

Distributed Systems LEEC (2005/06 2º Sem.)

Distributed Systems LEEC (2005/06 2º Sem.) Distributed Systems LEEC (2005/06 2º Sem.) Introduction João Paulo Carvalho Universidade Técnica de Lisboa / Instituto Superior Técnico Outline Definition of a Distributed System Goals Connecting Users

More information

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

More information

Index Terms : Load rebalance, distributed file systems, clouds, movement cost, load imbalance, chunk.

Index Terms : Load rebalance, distributed file systems, clouds, movement cost, load imbalance, chunk. Load Rebalancing for Distributed File Systems in Clouds. Smita Salunkhe, S. S. Sannakki Department of Computer Science and Engineering KLS Gogte Institute of Technology, Belgaum, Karnataka, India Affiliated

More information

A Locality Enhanced Scheduling Method for Multiple MapReduce Jobs In a Workflow Application

A Locality Enhanced Scheduling Method for Multiple MapReduce Jobs In a Workflow Application 2012 International Conference on Information and Computer Applications (ICICA 2012) IPCSIT vol. 24 (2012) (2012) IACSIT Press, Singapore A Locality Enhanced Scheduling Method for Multiple MapReduce Jobs

More information

A Brief Analysis on Architecture and Reliability of Cloud Based Data Storage

A Brief Analysis on Architecture and Reliability of Cloud Based Data Storage Volume 2, No.4, July August 2013 International Journal of Information Systems and Computer Sciences ISSN 2319 7595 Tejaswini S L Jayanthy et al., Available International Online Journal at http://warse.org/pdfs/ijiscs03242013.pdf

More information

Dynamic Virtual Cluster reconfiguration for efficient IaaS provisioning

Dynamic Virtual Cluster reconfiguration for efficient IaaS provisioning Dynamic Virtual Cluster reconfiguration for efficient IaaS provisioning Vittorio Manetti, Pasquale Di Gennaro, Roberto Bifulco, Roberto Canonico, and Giorgio Ventre University of Napoli Federico II, Italy

More information

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency WHITE PAPER Solving I/O Bottlenecks to Enable Superior Cloud Efficiency Overview...1 Mellanox I/O Virtualization Features and Benefits...2 Summary...6 Overview We already have 8 or even 16 cores on one

More information

Implementation of Reliable Fault Tolerant Data Storage System over Cloud using Raid 60

Implementation of Reliable Fault Tolerant Data Storage System over Cloud using Raid 60 International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-2 E-ISSN: 2347-2693 Implementation of Reliable Fault Tolerant Data Storage System over Cloud using Raid

More information

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms

Analysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms Volume 1, Issue 1 ISSN: 2320-5288 International Journal of Engineering Technology & Management Research Journal homepage: www.ijetmr.org Analysis and Research of Cloud Computing System to Comparison of

More information

Chapter 18: Database System Architectures. Centralized Systems

Chapter 18: Database System Architectures. Centralized Systems Chapter 18: Database System Architectures! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types 18.1 Centralized Systems! Run on a single computer system and

More information

An Overview of Distributed Databases

An Overview of Distributed Databases International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 4, Number 2 (2014), pp. 207-214 International Research Publications House http://www. irphouse.com /ijict.htm An Overview

More information

SAN Conceptual and Design Basics

SAN Conceptual and Design Basics TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer

More information

Load Balancing in Fault Tolerant Video Server

Load Balancing in Fault Tolerant Video Server Load Balancing in Fault Tolerant Video Server # D. N. Sujatha*, Girish K*, Rashmi B*, Venugopal K. R*, L. M. Patnaik** *Department of Computer Science and Engineering University Visvesvaraya College of

More information

A Study on Workload Imbalance Issues in Data Intensive Distributed Computing

A Study on Workload Imbalance Issues in Data Intensive Distributed Computing A Study on Workload Imbalance Issues in Data Intensive Distributed Computing Sven Groot 1, Kazuo Goda 1, and Masaru Kitsuregawa 1 University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan Abstract.

More information

Scala Storage Scale-Out Clustered Storage White Paper

Scala Storage Scale-Out Clustered Storage White Paper White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current

More information

Windows Server Failover Clustering April 2010

Windows Server Failover Clustering April 2010 Windows Server Failover Clustering April 00 Windows Server Failover Clustering (WSFC) is the successor to Microsoft Cluster Service (MSCS). WSFC and its predecessor, MSCS, offer high availability for critical

More information

International Journal of Computer Science Trends and Technology (IJCST) Volume 3 Issue 3, May-June 2015

International Journal of Computer Science Trends and Technology (IJCST) Volume 3 Issue 3, May-June 2015 RESEARCH ARTICLE OPEN ACCESS Ensuring Reliability and High Availability in Cloud by Employing a Fault Tolerance Enabled Load Balancing Algorithm G.Gayathri [1], N.Prabakaran [2] Department of Computer

More information

Energy Constrained Resource Scheduling for Cloud Environment

Energy Constrained Resource Scheduling for Cloud Environment Energy Constrained Resource Scheduling for Cloud Environment 1 R.Selvi, 2 S.Russia, 3 V.K.Anitha 1 2 nd Year M.E.(Software Engineering), 2 Assistant Professor Department of IT KSR Institute for Engineering

More information

EMC XtremSF: Delivering Next Generation Performance for Oracle Database

EMC XtremSF: Delivering Next Generation Performance for Oracle Database White Paper EMC XtremSF: Delivering Next Generation Performance for Oracle Database Abstract This white paper addresses the challenges currently facing business executives to store and process the growing

More information

Big Data Storage Architecture Design in Cloud Computing

Big Data Storage Architecture Design in Cloud Computing Big Data Storage Architecture Design in Cloud Computing Xuebin Chen 1, Shi Wang 1( ), Yanyan Dong 1, and Xu Wang 2 1 College of Science, North China University of Science and Technology, Tangshan, Hebei,

More information

Big data management with IBM General Parallel File System

Big data management with IBM General Parallel File System Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers

More information

Research on Job Scheduling Algorithm in Hadoop

Research on Job Scheduling Algorithm in Hadoop Journal of Computational Information Systems 7: 6 () 5769-5775 Available at http://www.jofcis.com Research on Job Scheduling Algorithm in Hadoop Yang XIA, Lei WANG, Qiang ZHAO, Gongxuan ZHANG School of

More information

www.rackwareinc.com RackWare Solutions Disaster Recovery

www.rackwareinc.com RackWare Solutions Disaster Recovery RackWare Solutions Disaster Recovery RackWare Solutions Disaster Recovery Overview Business Continuance via Disaster Recovery is an essential element of IT and takes on many forms. The high end consists

More information

Performance Isolation of Network Virtualization for Cloud Computing

Performance Isolation of Network Virtualization for Cloud Computing KSII The third International Conference on Internet (ICONI) 2011, December 2011 1 Copyright c 2011 KSII Performance Isolation of Network Virtualization for Cloud Computing Sung-Won Ahn 1 and Chuck Yoo

More information

NETWORK ATTACHED STORAGE DIFFERENT FROM TRADITIONAL FILE SERVERS & IMPLEMENTATION OF WINDOWS BASED NAS

NETWORK ATTACHED STORAGE DIFFERENT FROM TRADITIONAL FILE SERVERS & IMPLEMENTATION OF WINDOWS BASED NAS INTERNATIONAL International Journal of Computer JOURNAL Engineering OF COMPUTER and Technology (IJCET), ENGINEERING ISSN 0976-6367(Print), ISSN 0976 & 6375(Online) TECHNOLOGY Volume 4, Issue (IJCET) 3,

More information

Virtualization of the MS Exchange Server Environment

Virtualization of the MS Exchange Server Environment MS Exchange Server Acceleration Maximizing Users in a Virtualized Environment with Flash-Powered Consolidation Allon Cohen, PhD OCZ Technology Group Introduction Microsoft (MS) Exchange Server is one of

More information

Server and Storage Virtualization: A Complete Solution A SANRAD White Paper

Server and Storage Virtualization: A Complete Solution A SANRAD White Paper Server and Storage Virtualization: A Complete Solution A SANRAD White Paper copyright SANRAD 2008 SANRAD Inc. www.sanrad.com Server and Storage Virtualization: A Complete Solution A SANRAD Whitepaper Server

More information

Dynamic Resource management with VM layer and Resource prediction algorithms in Cloud Architecture

Dynamic Resource management with VM layer and Resource prediction algorithms in Cloud Architecture Dynamic Resource management with VM layer and Resource prediction algorithms in Cloud Architecture 1 Shaik Fayaz, 2 Dr.V.N.Srinivasu, 3 Tata Venkateswarlu #1 M.Tech (CSE) from P.N.C & Vijai Institute of

More information

SOLUTION BRIEF KEY CONSIDERATIONS FOR LONG-TERM, BULK STORAGE

SOLUTION BRIEF KEY CONSIDERATIONS FOR LONG-TERM, BULK STORAGE SOLUTION BRIEF KEY CONSIDERATIONS FOR LONG-TERM, BULK STORAGE IT organizations must store exponentially increasing amounts of data for long periods while ensuring its accessibility. The expense of keeping

More information

Nutanix Tech Note. Failure Analysis. 2013 All Rights Reserved, Nutanix Corporation

Nutanix Tech Note. Failure Analysis. 2013 All Rights Reserved, Nutanix Corporation Nutanix Tech Note Failure Analysis A Failure Analysis of Storage System Architectures Nutanix Scale-out v. Legacy Designs Types of data to be protected Any examination of storage system failure scenarios

More information

technology brief RAID Levels March 1997 Introduction Characteristics of RAID Levels

technology brief RAID Levels March 1997 Introduction Characteristics of RAID Levels technology brief RAID Levels March 1997 Introduction RAID is an acronym for Redundant Array of Independent Disks (originally Redundant Array of Inexpensive Disks) coined in a 1987 University of California

More information

Scaling Microsoft SQL Server

Scaling Microsoft SQL Server Recommendations and Techniques for Scaling Microsoft SQL To support many more users, a database must easily scale out as well as up. This article describes techniques and strategies for scaling out the

More information

Infrastructure as a Service (IaaS)

Infrastructure as a Service (IaaS) Infrastructure as a Service (IaaS) (ENCS 691K Chapter 4) Roch Glitho, PhD Associate Professor and Canada Research Chair My URL - http://users.encs.concordia.ca/~glitho/ References 1. R. Moreno et al.,

More information

High Availability Design Patterns

High Availability Design Patterns High Availability Design Patterns Kanwardeep Singh Ahluwalia 81-A, Punjabi Bagh, Patiala 147001 India kanwardeep@gmail.com +91 98110 16337 Atul Jain 135, Rishabh Vihar Delhi 110092 India jain.atul@wipro.com

More information

Distributed and Cloud Computing

Distributed and Cloud Computing Distributed and Cloud Computing K. Hwang, G. Fox and J. Dongarra Chapter 3: Virtual Machines and Virtualization of Clusters and datacenters Adapted from Kai Hwang University of Southern California March

More information

A Migration of Virtual Machine to Remote System

A Migration of Virtual Machine to Remote System ISSN (Online) : 2319-8753 ISSN (Print) : 2347-6710 International Journal of Innovative Research in Science, Engineering and Technology Volume 3, Special Issue 3, March 2014 2014 International Conference

More information

Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load

Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load Pooja.B. Jewargi Prof. Jyoti.Patil Department of computer science and engineering,

More information

Advanced File Sharing Using Cloud

Advanced File Sharing Using Cloud Advanced File Sharing Using Cloud Sathish.Y #1, Balaji.S *2, Surabhi.S #3 Student, Department of CSE,Angel College of Engineering and Technology,Tirupur,India. [1] Asst.Prof, Department of CSE,Angel College

More information

A Novel Switch Mechanism for Load Balancing in Public Cloud

A Novel Switch Mechanism for Load Balancing in Public Cloud International OPEN ACCESS Journal Of Modern Engineering Research (IJMER) A Novel Switch Mechanism for Load Balancing in Public Cloud Kalathoti Rambabu 1, M. Chandra Sekhar 2 1 M. Tech (CSE), MVR College

More information

WHITE PAPER The Storage Holy Grail: Decoupling Performance from Capacity

WHITE PAPER The Storage Holy Grail: Decoupling Performance from Capacity WHITE PAPER The Storage Holy Grail: Decoupling Performance from Capacity Technical White Paper 1 The Role of a Flash Hypervisor in Today s Virtual Data Center Virtualization has been the biggest trend

More information

A SWOT ANALYSIS ON CISCO HIGH AVAILABILITY VIRTUALIZATION CLUSTERS DISASTER RECOVERY PLAN

A SWOT ANALYSIS ON CISCO HIGH AVAILABILITY VIRTUALIZATION CLUSTERS DISASTER RECOVERY PLAN A SWOT ANALYSIS ON CISCO HIGH AVAILABILITY VIRTUALIZATION CLUSTERS DISASTER RECOVERY PLAN Eman Al-Harbi 431920472@student.ksa.edu.sa Soha S. Zaghloul smekki@ksu.edu.sa Faculty of Computer and Information

More information

Scheduling using Optimization Decomposition in Wireless Network with Time Performance Analysis

Scheduling using Optimization Decomposition in Wireless Network with Time Performance Analysis Scheduling using Optimization Decomposition in Wireless Network with Time Performance Analysis Aparna.C 1, Kavitha.V.kakade 2 M.E Student, Department of Computer Science and Engineering, Sri Shakthi Institute

More information

Balancing Server in Public Cloud Using AJAS Algorithm

Balancing Server in Public Cloud Using AJAS Algorithm Balancing Server in Public Cloud Using AJAS Algorithm Ramya.B 1, Pragaladan R 2, M.Phil Part-Time Research Scholar, Assistant Professor Department of Computer Science, Department of Computer Science, Sri

More information

An Enhanced CPU Scheduler for XEN Hypervisor to Improve Performance in Virtualized Environment

An Enhanced CPU Scheduler for XEN Hypervisor to Improve Performance in Virtualized Environment An Enhanced CPU Scheduler for XEN Hypervisor to Improve Performance in Virtualized Environment Chia-Ying Tseng 1 and Yi-Ling Chung Department of Computer Science and Engineering, Tatung University #40,

More information