Cooperative Virtual Data Center: Sharing Data and Resources among Multiple Computing Entities

Transcription

1 , pp Cooperative Virtual Data Center: Sharing Data and Resources among Multiple Computing Entities Eun-Kyu Lee Dept. of Information and Telecommunication Engineering Incheon National University Abstract Existing data centers are each individually owned and operated by a single entity. This situation creates an excessive financial burden upon each entity through the need to overprovision for hardware. While this state of affairs enables a secure and an efficient maintenance scheme for the data center, the financial drawbacks are perhaps excessive. To address the concern, we propose the idea of a cooperative virtual data center among multiple entities that is founded on the principal of fair resource sharing amongst the entities. Keywords: Data center, Cloud computing, Secure communication, Storage, Network 1. Introduction Data centers represent massive financial undertakings by corporations to construct, operate, and maintain. Yet, the operational cost of data centers is greatly exaggerated. The primary reason of the over inflation of data center costs results from the over-provisioning of resources required for operation during rare occasions of abnormally high workloads. The average utilization of a data center is typically around 20%, with the other 80% overprovisioned for periods of peak loads [33]. The result of this over-provisioning has multiple effects on the fixed and variable operational costs of running a data center. First, there is a significant increase in the capital costs. A corporation must invest five times more than necessary into hardware than typically needed. With this additional hardware expenditure comes increased server storage costs. Second, there is an additional operational overhead that comes with increased hardware. The additional servers require a larger support staff to operate, to upkeep, and to maintain. Finally, there is an elevated cost associated with powering and cooling the redundant servers. While the energy requirement for powering a quiescent server is far less than an active one, it is an extra cost that cannot be avoided [32]. We see that the concept of virtual data centers (VDC) [20] is able to circumvent the additional costs incurred by over-provisioning. However, virtual data centers in the conventional sense cannot reduce data center costs alone. Instead, we notice an opportunity to exploit resource sharing amongst data centers similarly to [11]. Existing data centers are each individually owned and operated by a single entity. This situation creates an excessive financial burden upon each entity through the need to over-provision for hardware. While this state of affairs enables a secure and an efficient maintenance scheme for the data center, the financial drawbacks are perhaps excessive. Instead, we propose the idea of a cooperative virtual data center between multiple entities that is founded on the principal of fair resource sharing amongst the entities. Conceptually, each data center can reduce the amount of over-provisioning of resources by cooperatively sharing computing power with other data centers. Since data centers generally have transient and predictable periods of peak load, it is conceivable that a data center can borrow computing resources, in the form of virtual machines [31], from ISSN: IJSEIA Copyright c 2015 SERSC

2 other data centers that have spare resources for short-lived periods. Moreover, this form of cooperative sharing of computing resources provides a level of topographically distributed fault tolerance as the cooperating data centers may be geographically separated from one another. The reduced provisioning of servers has the added benefit of diminishing operating costs through abating energy requirements and minimizing required support personnel. Furthermore, cooperative VDC enable the active use of idle resources thereby creating a green computing solution [25]. According to [12], cooperative VDC have the potential to tackle 85% of the current costs associated with running a data center in terms of servers, infrastructure, and power draw. Cooperative VDC, nonetheless, suffer from the same privacy issues as cloud computing. The cooperative VDC scheme may require confidential data to be stored within another entity's private cloud, thereby exposing the discreet data. Currently there is no good way to ensure confidentiality in such situations [25]. The main goal of this paper is to promote the feasibility of a cooperative virtual data center amongst several independent entities that can still maintain private data confidentiality under minimal trust assumptions. We note that this paper extends previous research [22]. The paper is organized as follows. Section 2 provides the motivation and background behind our work and Section 3 deals with our proposed architecture. The challenges to our proposed solution are described in Section 4. Section 5 talks about some assumptions made about underlying technologies involved in this solution. Section 6 discusses the main approach of our solution. Finally, Section 7 presents our conclusions. 2. Motivations and Background Truly large-scale data centers are rare. Only a handful of companies in the world have the resources to maintain and to operate one. These data centers are comprised of tens to hundreds of thousands of servers [4, 26]. Most data centers are much smaller, on the order of hundreds to thousands of servers instead, run by mid-sized to small companies. Even with a few hundred servers, the financial commitment becomes a large burden for these more constrained companies. Realistically, data centers, on average, use far less resources than which they are equipped for. This is in large part due to the over-provisioning by data centers in situations of peak workloads, which are transient and often predictable. The redundancy which is provided for by data centers is unnecessary in most situations, but the operators of the data center must pay the cost of supporting this redundancy year-long (24/7/52). Energy costs for powering and cooling idle servers cannot be avoided in such cases. The following subsections discuss the predictable and transient natures of these workloads as well as the green impact of over-provisioning Predictable and Transient Peak Loads Periods of high workload can generally be predicted either based off of historical data or common sense in some cases. For example, an e-commerce web site can generally expect an increase of traffic starting on Black Friday [2]. In this case, the times of peak load generally last from a day to a week. Alternatively, online tax service providers may see an increased amount of web traffic when the tax deadline approaches, and will dwindle down after the deadline passes. A multitude of data mining approaches can be taken into ac- count to analyze traffic history to determine patterns of elevated workload. Knowing these patterns can help a data center plan for periods of increased utilization or vice versa in determining when resources are idle. Moreover, periods of peak loads will dissipate over time. These spans can range from a few minutes to a few hours to a few days, however the average utilization of a data center will eventually average out, with the redundant servers quiescent more often than not. 138 Copyright c 2015 SERSC

3 2.2. The Green Effect Over-provisioning creates a large cadre of idle servers, each consuming power and requiring additional cooling expenditures to maintain. The superfluous power requirements go to waste as the servers remain passive. The increased power consumption potentially implies the increased burning of fossil fuels or worse. It is estimated by Gartner that nearly a quarter of total information and communications technology (ICT) global CO2 emissions are the result of operating data centers [9]. Additionally, the lack of facilities to house current and future high density servers in most existing data centers worsen the power requirements and makes energy management a priority. 3. Cooperative Virtual Data Center The conceptualization of the cooperative virtual data center seeks to reduce both capital and operational expenditures in running a data center while providing a greener solution. The perception of the cooperative virtual data center operates around the notion that a data center is capable of utilizing heterogeneous idle resources form other private data centers when needed in periods of peak work- load in order reduce overprovisioning. This allows for a reduction in capital, thereby propagating to reductions in operational expenditures. Match applica ons to available resources Applica on Virtualiza on Access data at any loca on, in any format Data Virtualiza on Infrastructure Virtualiza on Pool and share aggregated resources En Virtualiza on among DCs es share their resources. Figure 1. The Eventual Goal of the Cooperated Virtual Data Centers This form of cooperative resource sharing provides for better data center cost effectiveness. Figure 1 shows the high level goal of cooperative resource sharing. It is currently envisioned that there would be an alliance of a group of small to mid-sized companies which each have non-conflicting periods of known peak load. The alliance membership size is perceived as being no more than 10 to 20 participants, perhaps even less. Moreover, an added benefit of this scheme enables fault tolerance if the members of the alliance happen to have their data centers geographically segregated. This reconstruction the virtual data center breaks sharply from one proposed by [11, 25] in the sense that the physical data centers which form a virtual data center are each individually operated by a separate entity which originates as part of a corporation s private cloud. This differentiation, while minor changes the fundamental dynamics of data center cost structures, financial risks, and utilization. Independently operated data centers will eventually move to a model of the cooperative virtual data centers where multiple data centers together build a single virtual computing platform by sharing their resources. Figure 2 and Figure 3 illustrate the transition. Copyright c 2015 SERSC 139

4 Company A Company C Internet Company B Figure 2. Independent, Separately Operated Data Centers A Full connec on? B Figure 3. A Model of Cooperative Virtual Data Center As commonly accepted, most data centers operate at approximately 20% utilization on average, with the rest of the servers placed in a quiescent mode for redundancy and peaks in workload. The amount of over-provisioning in such cases is unnecessary and wasteful. Instead of having 80% over-provisioning of hard- ware, it is possible in the cooperative VDC scheme to simply over-provision by as little as an extra 20%, a 75% savings in over-provisioning requirements assuming economies of scale do not hold. This major compaction of over-provisioning leads to a munificent decrease in costs across the board as well as other benefits. This will hopefully make operating data centers much more affordable and thereby feasible for companies. 4. Challenges Cooperation amongst individual data centers while far more efficient than operating separate data centers does have a number of drawbacks. The most common issues with such distributed solutions range into the categories of reliability and security. However, the most prevalent issues with resource sharing amongst data centers are the same as those which have prevented mass adoption of cloud computing storage. Namely, any data placed into a cloud computing storage environment becomes public. Privacy or the lack thereof is one of the main hindrances to the adoption of storage of all data in the cloud. This, likewise, carries over to the cooperative virtual data center concept. Confidential data may be stored on a disk in a foreign data center, open to reading for anyone in the other data center. This situation is unacceptable for most private companies and is considered one of the major drawbacks to cloud storage in general. While there have been proposed solutions to segregate the data from the applications to secure cloud C 140 Copyright c 2015 SERSC

5 computing, the latency and performance requirements may make this approach unacceptable [25]. Yet, unless this issue of privacy is resolved, it is unlikely that any company would support or adopt the cooperative virtual data center model. Furthermore, security mechanisms must be taken into account on securing the communications between the individual data centers when communicating and transferring data. Not only must network security be taken into account when starting a virtual machine in foreign data center, but it must also be considered inside the foreign data center itself. Within the foreign data center there are threats beyond just internal network security, but to the virtual machine monitor (VMM) itself [19, 27]. In the worst case, there could be malicious employees in the foreign data center who intentionally packet sniffs secret data. Finally, reliability issues must be taken into account for the cooperative virtual data center. If a company needs to borrow resources from the collective pool and there are no resources available, then this can become a critical situation of under-provisioning. In such situations, the lack of reliability can have disastrous economic consequences. There must be a recovery mechanism in place when situations arise where communal resources are not available. Furthermore, synchronization issues between the data centers can also result in unreliable or unanticipated situations. 5. Assumptions This section begins by describing our assumption for the realization of the pro- posed cooperative VDC. At first, a paravirtualization scheme is taken as our virtualization platform because of its outstanding properties of high performance and secure isolation between virtual machines (VMs). Researches on VMs indicate that paravirtualization vulnerabilities primarily stem from a compromised VMM [19]. While VM security is an interesting and significant research topic, it is out of our scope. Instead, the proposed architecture assumes that a VMM is not compromised, i.e., a tamper-proof VMM. This implies that virtualized resources in a VM such as CPU and memory are secure. Henceforth, resources provided by the VM and VMM can be considered secure, enabling a trust amongst co-operating entities. We argue that this assumption is reasonable. Each entity is unwilling to cheat on purpose, i.e., attacking other entity s data in its server, since its own private data is also processed in the others servers. In the cloud computing sense, people are using cloud computing services, e.g., Azure and EC2, based on the same level of trust with the service providers such as Microsoft and Google. There is an implicit assumption that such providers will not misuse a user s private data. Trust of an individual entity, however, does not guarantee that the data within a data center is fully protected. When considering data privacy as the dominant reason why enterprises hesitate to move toward cloud computing, a privacy mechanism must be found to fill this hole. In this sense, we determine two areas that can possibly be attacked by internals. Despite trust on a reliable entity or company providing services, this does not indicate that we can also trust all the employees of the company. A malicious employee can easily break the security of systems within a data center. The first threat model is network sniffing. Since the malicious employee sits inside your firewall, he can hear every data packet going through the protected local network. Network analysis tools enable him to access sensitive data of customers with less effort. The other threat comes from his ability to directly access file storage or disk. He is definitely prohibited from accessing data in storage through the VMM. However, the storage is physically out of control from the VMM, indicating another vulnerability. He can access the storage or a Storage Area Network (SAN) from his personal computer without intervention of an administrator. The worst scenario involves the theft of the physical storage device. Then, secured information can be extracted from the storage disk. One interesting observation is that this is not a problem specific to the proposed architecture. All the existing data centers also Copyright c 2015 SERSC 141

6 face to the same challenge, but do not provide any answer yet. We believe that the proposed cooperative VDC architecture can provide a better solution to overcome this obstacle. Company A VM A VM B VM C Company C Tamper-proof VM Monitor Private storages Company B Figure 4. Realization of the cooperative virtual data center 6. Proposed Approach The primary architecture underlying the cooperative virtual data center is shown in Figure 4. Here we see three companies, A, B, and C, where company A and C are shown to be borrowing resources from company B through a leasing of virtual machines. Company A and C both store their private data on their own dedicated hard disks located within the storage area network (SAN) of company B. It is expected that these disks would likely be placed into a more cost-effective iscsi SAN connected to the physical servers using fibre channel host bus adapters, where each company is expected to provide their own hard disks. The choice here of using an iscsi approach over that of fibre channel primarily involves the cost factor of deployment with commodity resources and the expected access patterns for the storage. Adoption of fibre channel can also be justified depending upon the expected needs of the system. Nonetheless, the actual underlying implementation of the SAN technology is up to the company hosting each private storage disk. The virtual machine monitor (VMM) is responsible for ensuring that a running virtual machine (VM) can only accesses disks that the VM owns. In this case, VM A, run by company A can only access the disk drives which company A has placed in company B, henceforth, VM C would not be able to access the disk of company A. In our proposed implementation, Xen 1 is assumed to be the hypervisor used to perform these domain security validations and access checks to disk since it enables high performance buffered I/O transfers using asynchronous buffer-descriptor rings [3]. Furthermore, all communications between company A and VM A running within company B s data center is expected to be secured. The virtual firewall- router (VFR) running within Xen will be capable of filtering results from each virtual interface (VIF) to ensure that communication from company C cannot affect VM A. In the following subsections we focus on discussing our solutions to enhancing storage security and privacy under a cooperative virtual data center as well as the reliability and synchronization issues with the solution. Namely, we aim to briefly tackle the following issues: How to make secure communication and thus keep malicious employees of a foreign data center from sniffing the internal network traffic for confidential information. How to secure information stored on a hard drive to keep unauthorized users from reading the data Copyright c 2015 SERSC

7 How to synchronize data between data centers. How to ensure reliability of resource pooling Securing Network Traffic Figure 5. DMZ with Reverse Firewall For secured network communication, this paper proposes to create a demilitarized zone (DMZ) subnetwork for the VDC. As shown in Figure 5, an additional firewall is installed between the local network of the host company, i.e., company B, and its VDC network. This executes a reverse firewall. In addition, a dedicated network interface handles the entire traffic to/from guest VMs. The initial design issue of DMZ was defense in depth with flexible provisioning of services [24], i.e., some servers like a web server is likely to be exposed to anonymous Internet users. This denotes a vulnerability of the local network if the server is within the same network as the other server systems. In the proposed architecture, on the other hand, the DMZ subnetwork for VDC is created as a safe zone as its name represents. Two firewalls keep dangerous network traffic out of the DMZ and the dedicated network interface would enable the network administrator to give exceptional attention to guest VMs packets. The new DMZ is implemented through a rule of defense in depth. A traditional firewall protects a local network from outside attacks. In contrast, the concept of a reverse firewall [1] is to protect the outside from packet flooding attacks, e.g., Distributed Denial of Service (DDoS), that emanate from within a local network. In coordinated DDoS attacks, it is much more likely that some hosts inside a local network will be taken over by remote attackers for use as zombies. The reverse firewall filters such illegitimate traffic out. One of functions in the reverse firewall to achieve this goal is a fair service to visible sources. The reverse firewall focuses on additional path information that has been added to the packet, such as the packet s source data. In the proposed scheme, its modification is taken into account to limit access attempts to the protected VDC systems. The reverse firewall also provides a secure isolation of the VDC network from the host company s local network, which enhances the level of data protection. Dedicated network interfaces is another underlying scheme to support secure transmission of VDC data packets. All the data traffic associated with guest VMs, i.e., VM A and VM C in this figure, go through the specified network interface that is separated from the other network interface that the host VM, i.e. VM B, would use exclusively. One benefit is that this can guarantee secure isolation of network resources. Since a guest VM s data uses a different interface from a host s for networking, it does not allow any connection to the local net- work s systems, which avoids internal attacks. The dedicated network interface can also enhance the degree of security against network threats since firewalls can give precise protection to the communication point. Data encryption has been always the starting point in network security. In the architecture, Internet Protocol Copyright c 2015 SERSC 143

8 Security (IPSec) or Secure Socket Layer (SSL) can be used for secure communication between end systems in the network. The new DMZ for the proposed VDC comprising of firewalls and dedicated network interfaces can provide protection against network threats and provide secure isolation of network resources. With authentication schemes using lever- aging data encryption, the DMZ also guarantees that the plain data is only revealed only within the tamper-proof VMM, supporting data privacy Securing the Storage Device To ensure data privacy, we propose the usage of hardware-based full disk encryption (FDE). In such a scheme, there are two general approaches to ensuring hardware-based FDE; the first is a chipset based approach and a second is a hard disk drive (HDD) solution. A chipset encryption method is relatively new and was most prominently introduced with the Intel Danbury software architecture originally slated for the Eaglelake chipset [13]. However, as it turns out, the chipset FDE solution seems inappropriate for our architecture solution. Chipset based encryption suggests that the decryption keys work in parallel with a trusted platform module (TPM) which essentially locks a disk to a particular chip co-located with the disk. While this may be acceptable if both the disk and TPM were located in a company s own private data center, however this is not the case as both the disk and TPM would be located in a foreign data center. Additionally, this approach may be difficult to implement in the SAN environment. Instead, we look into HDD FDE as a solution. In this case, key management is handled within the HDD itself, leaving key protection the duty of the drive firmware. This solution is much easier to adapt to our architecture and was chosen for another security reason discussed in Section 6.2. However, there must still be some form of authentication to take place within the CPU to en- able the decryption of the disk through the firmware. The solution to this is to use a software-based Pre-Boot Authentication (PBA) Environment. The PBA Environment will not decrypt the keys stored in the HDD that has been used to encrypt the rest of the disk until an external authentication key has been inputed into the system. The proposed PBA mechanism is to use a Preboot Execution Environment (PXE). Intel Corporation and Systemsoft introduced the PXE during the late 90s in [18]. Today the technology is supported by a variety of vendors on a number of different platforms. Originally, PXE was designed to allow for the remote booting of a system using the network interface independent of hard disks and operating systems. PXE is specifically chosen as the technology for introducing an external key to the HDD for two reasons. First, the components that the protocol relies on are currently supported by most vendors. Moreover, there is strong indication that it will continually be supported in the future [17, 23]. Second, the PXE protocol is a fairly robust protocol capable of being adopted in chunks and modified slightly for our purposes. Company B PXE Client (VMM) Extended DHCPDISCOVER : Look for loca on of boot server DHCPOFFER : Informa on on boot server loca on Extended DHCPREQUEST : Request bootstrap image Extended DHCPACK : Informa on on NBP file loca on Network Bootstrap Program (NBP) download via TFTP Execute boot image downloaded on RAM Figure 6. PXE Client-Server protocol DHCP Server Boot Server Boot Service + TFTP Service B A 144 Copyright c 2015 SERSC

9 Figure 6 shows the high-level operational client-server protocol specified by [18] and [15]. In this figure, two companies, A and B are shown. When company A attempts to start a VM image in company B s data center, the VMM within company B s data center takes the role of the PXE client. It is the job of the PXE client to retrieve an outside key to enable decryption of the HDD. To do this, the PXE client will issue a DHCPDISCOVER message to port 67 of the local DHCP server. The discovery message will request a list of boot servers that contains the network bootstrap program (NBP) that can be used to decrypt the hard disk. The DHCP server will then reply with an Extended DHCPOFFER to port 68 of the PXE client with the list of boot servers that it has requested. After the PXE client has obtained a list of the boot servers, the PXE client will send a DHCPREQUEST to either port 67 or port 4011, depending on the boot server capabilities, to the boot service. Upon receiving the DHCPREQUEST, the boot service of the boot server returns a DHCPACK to the source port of the request with the boot file name, options required to successfully run the NBP, and potentially the configuration parameters of the MTFTP (Multicast Trivial File Transfer Protocol) if it is used. After receiving the DHCPACK, the PXE client can then contact the TFTP service on port 69 or MTFTP service based on the DHCPACK s MTFTP configuration parameters to download the NBP. Once the NBP has been downloaded, the PXE client can run the NBP to provide an authentication module to decrypt the HDD Modifications to the PXE Protocol: Referring to Figure 6, the description of the above PXE protocol requires some small modification in the cooperative VDC context. Most notably is the DHCPREQUEST message sent to the boot service on the boot server. In our architecture of a cooperative virtual data center, the boot server is located in company A s secure network. Clearly in such a situation, DHCP messages from a PXE client in company B will not be appropriate. However, this does not mean that a similar protocol cannot be applied to achieve the same results. Additionally, the PXE protocol was designed to boot a system from scratch. In the cooperative VDC situation, the hypervisor is already running and takes the place of a running operating system. Instead, the VMM needs to decrypt a hard disk to load the VM image in order to boot up the guest OS. The downloaded NBP should essentially contain either the external key to decrypt HDD or a software program that can decrypt the disk, no authentication method need be necessary Implementation of the PXE Protocol: The implementation of the above modifications to the PXE can be easily accomplished in Xen. There is only one logical place within the hypervisor to implement the PXE client protocol base. Since Domain0 is started by the VMM at boot time with the ability to run a guest OS with privileged access to the control interface, it is able to host application-level management software [3]. This makes the control plane the likeliest candidate as source to implement the PXE client functions. Seeing that the control plane running under Domain0 is also responsible for the creation of other domains within Xen, it is capable of making a call to either a library or another application to perform the PXE client operations. This enables a natural integration for the functions of the PXE client within the Xen hypervisor architecture. Modern DHCP servers already support the PXE protocols. It is mostly a matter of configuring the servers to properly return the appropriate list of boot server addresses. However, as mentioned previously, there needs to be an implementation of a different protocol mechanism to replace the DHCPREQUEST message to the boot service of the boot server. The specific details of this protocol are beyond the scope of this paper, however it is conceivable that it is relatively trivial to implement. Likewise, the file transfer protocol (FTP) mentioned previously is considered as a given. Copyright c 2015 SERSC 145

10 Alternatives to Hardware-based FDE: There are of course alternatives to hardware-based disk encryption. Disk encryption can just as easily be implemented in software. However the drawback to using software instead of hard- ware lies in the performance impact. It is generally accepted that hardware-based FDE is considerably faster than software-based solutions. The improved performance of hardware-based FDE lies in the fact that there is no added extra overhead for the CPU or the hard disk itself. Only on initial boot of a VM is there any extra processing which needs to be performed to execute the operations of a PXE client in the cooperative virtual data center architecture. Furthermore, softwarebased encryption suffers from the need that there must generally be a small segment or partition of the hard disk that must be left un-encrypted. This segment is used to typically store the master boot record (MBR) that is necessary to be left un-encrypted to enable the initial booting of an operating system Weaknesses of Hardware-based FDE: While hardware-based FDE provides a greater sense of data privacy, it is not flawless. Specifically, hardware-based FDE solutions are largely vulnerable to cold boot attacks citehalderman08. This primarily stems from the remanence effects of DRAM where computer memory retains the stored data for a brief amount of time even after the system is turned off. Experiments by Chan et al. have shown that such remanence effects of memory can easily compromise system security [5]. Moreover, there have even recently been open source tools released specifically to perform such cold boot attacks [28]. While Halderman, et al., indicate that hardware-based FDE is vulnerable to cold boot attacks in general, they mention that encrypting the data in the hard disk controller can be used to help prevent cold boot attacks. Their perspective suggests that as long as the write-only key register of the disk firmware is cleared before each boot, then HDD FDE can be protected from cold boot attacks [14]. Furthermore, this technique is currently implemented in certain HDD FDE systems such as Seagate s DriveTrust [30] technology; therefore HDD FDE is highly appropriate for the cooperative virtual data center architecture. 6.3 Synchronization of Replicated Data Two primary reasons for replicating data amongst data centers are for reliability and performance. In the aspect of reliability, data centers can continue operating after one of their replicas crashes by simply switch to one of the other replicas; also, it becomes possible to provide better protection against corrupted data. In the aspect of performance, when the number of processes attempting to access data managed by a server increases, performance can be improved by replicating the server and subsequently dividing the work; additionally, a copy of data can be placed in the proximity of the process accessing the information to reduce the time of data access. However, main concern is keeping all replicas up-to-date. Intuitively, a collection of copies is consistent when the copies are always the same, that is, a read operation performed at any copy will always return the same results. Consequently, when an update operation is performed on one copy, the update should be propagated to all copies before a subsequent operation takes place. Achieving such a tight consistency incurs high cost because updates need to be executed as atomic operations, and global synchronization is required. The synchronization scheme used in GFS [10] works this way with acceptable delay only because the replicas are stored within a local area of the primary copy which does not necessarily translate to the cooperative VDC case. The only real solution is to loosen the consistency constraints. Various consistency models have already been proposed and used for replication in distributed systems [29]. 146 Copyright c 2015 SERSC

11 Figure 7. Three Rounds can Synchronize Eight Nodes A Simple Synchronization Scheme: Synchronization is done in two steps, the first for detecting files that have been modified since the last run, and the second for propagating the updated data. In the worst case, i.e., when all nodes have updated files, an all-to-all communication is needed. In a naive approach, each node would send its updates to all partners, resulting in O(n(n 1)/2) updates. Each single connection would trigger an independent local disk access, provoking many updates concurrently and therefore resulting in a slow data transfer rate. To avoid this, [29] proposed an efficient synchronization scheme that uses only node-to-node (N2N) syncs. In this scheme, each node participates in at most one N2N sync at a time. Therefore, at most n/2 N2N syncs are run concurrently. Based on the fact that in each N2N sync a node propagates not only the modifications made to its own data, but also the modifications it received from other nodes in earlier rounds of the same sync. In [29], each node needs a maximum of log(n) N2N syncs in a complete graph. The sync process is given by a list of rounds of parallel N2N syncs. No barrier operation is executed between the rounds and therefore rounds may overlap. Figure 7 depicts the synchronization of eight nodes. The process is split into three rounds. Each of them contains four parallel N2N syncs: (A B, C D, E F, G H), (A C, B D, E G, F H), (A E, B F, C G, D H) Gossip in Complete Graphs: For optimal synchronization based on N2N synchronization gossip algorithms [29] can be also used. The constant model takes only the startup cost of a connection into account. In the linear model the communication cost c is proportional to the size l of the data volume: c = β + lτ, where β is the startup cost and τ is the transfer time of a unit-length message. τ is assumed to be constant for all links. Determining a cost-optimal gossip plan for arbitrary graphs is an NP-hard problem [21]. We therefore simplify this problem by not supporting arbitrary graphs but only hierarchies of regular graph classes. Additionally, we treat network hubs like switches amongst data centers, thereby neglecting possible hub congestion. With these two restrictions, we can generate optimal plans for some classes of graphs, which are hierarchically composed by heuristics to allow for arbitrary networks. Complete homogeneous graphs can be used to model switched networks. For this purpose, we use the optimal algorithm described in [16]. It needs at most logn rounds in graphs with an even (resp. odd) number of nodes. Depending on the amount of updated repositories, the synchronization time varies between O(n log n) for a one-to-all broadcast and O(n) for an all-to-all broadcast. Similar algorithms for other graphs like rings and busses are also known [16], but they are only optimal in the constant model where the link bandwidth is ignored. Copyright c 2015 SERSC 147

12 100Mbps switched net 1 A B 56 Kbps 2 2 D C 1 100Mbps switched net A B 56 Kbps 1, 4 2, 5 D C 3 30MB new data Plain gossip 30MB new data Topology op mized Figure 8. Gossip Considering Bandwidth Gossip in Practical Scenarios: Figure 8 illustrates the importance of considering the network characteristics in the synchronization process. The left-hand side in Figure 9 shows a straightforward but inferior gossip plan where the slow link between the left subnetwork and node D is used twice. If we treat this graph as a hierarchy of subgraphs, we can achieve better performance. Let us regard nodes A, B, C as a combined node and synchronizes it to D. More precisely, we first collect all updates of the subnetworks on their respective proxy nodes C and D (rounds 1 and 2), then synchronizes the proxy nodes via the slow link (round 3), and finally propagate the new data in each subnetwork (rounds 4 and 5). This composition of gossips in the subnetworks can be done with algorithms that are specialized for rings, chains, grids, switches, etc. In the synchronization process, the optimal broadcast plan of gossips can be generated by combinatorial optimization with proper constraints based on topological information and data dependency in polynomial time. It remains as part of our future work Providing Reliability of Resources Discussing reliability in a cooperative virtual data center (CVDC), we assume that each entity already has sufficient resources when sharing amongst data centers based on their usage expectation. Unless there are some unexpected problems in a data center or between data centers, there will be sufficient resources so that no data center within the CVDC faces an under-provisioning situation. The CVDC can be a fragile architecture. Faults can occur as a result of minor errors or catastrophic meltdowns. A CVDC system must be designed to be as fault tolerant as possible. Faults can occur in any of the components of a data center (DC) or connections between DCs. In these following subsections, we discuss a major fault which can occur in a CVDC causing a temporary under- provisioning and a possible solution to this situation Network Failures: Network failures keep DCs from communicating with each other. We will look at the failures resulting in total loss of communication amongst DCs. A major problem arises from this: one-way links and network partitions. One-way links cause problems similar to DC slowdown in a cooperative VDC. For example, DC A can send messages to DC B but cannot receive messages from DC B. DC C can talk to both DC A and DC B. So each DC has a different view of the other states of the other DCs. DC A might think that DC B has failed since it does not receive any signals from it. DC C thinks both DC A and DC B are working properly since it can send and receive messages from both. Network partitions occur in a network failure that connects two DCs. So DC A and DC B can communicate with each other in one DC but cannot communicate with DC C and DC D in another DC and visa versa Agreement: In order for a CVDC to function properly, each DC within the cooperative must share the same view of the overall CVDC. This requires a form of mutual agreement between the DCs for conflict resolution that requires extra overhead in 148 Copyright c 2015 SERSC

13 terms of either storage, computation or messaging. This area of distributed agreement has been studied in depth and has had many practical applications. For instance, Amazon s Bigtable [7] storage system uses the Paxos [6] algorithm as a form of conflict resolution, however, this algorithm requires multiple replicas and extra messaging and processing to resolve conflicts. Amazon s Dynamo [8] key-value store on the other hand, may temporarily provide inaccurate data but has an off-line conflict resolution scheme that requires versioning and extra computation to merge the separate versions of data. Either of these schemes may be appropriate for conflict resolution between the cooperating data centers depending on circumstances Solutions to System Failures: It is envisioned that each participating DC in the CVDC will notify other DCs in the cooperative of its current status that includes the amount of resources shared in the communal pool. Additionally, it s expected that each DC within the CVDC will share the same view of this information based on an agreement mechanism discussed above. In the event of a catastrophic event or temporary component failure amongst any of the participating DCs in the CVDC, and it is determined that the amount of resources in the communal pool is not enough to satisfy the needs of any single DC in the cooperative, then a fallback mechanism must occur; in this situation to prevent a potential temporary under-provisioning. The solution proposed under this CVDC condition is described as follows. Each DC is assumed to be able to monitor the utilization of resources for each VMM within its own network. Additionally, each DC is considered to be able to specify a threshold value, θ, where θ < 1 for each physical server. Moreover, each VMM is able to keep its utilization under the threshold. The threshold is reported to the other DCs in the CVDC as the 100% capacity. When it is determined that the community resource pool falls below the minimum provisioning requirement of any cooperative member s expected needs, then each DC can increase the θ threshold value to increase the provided resources. This has the net effect announcing additional resources available to the communal resource pool. This threshold value can be lowered after it is determined that the emergency situation has subsided. 7. Conclusion Existing data centers create an excessive financial burden upon each entity through the need to over-provision for hardware. In order to solve this, we pro- pose the idea of a cooperative virtual data center (CVDC) between multiple entities, which is founded on the principal of fair resource sharing amongst the entities. This new architecture also enables a secure and an efficient maintenance scheme for the data center. To realize this novel architecture, we propose to create a safe zone, the DMZ, in which CVDC systems are placed. The DMZ, consisting of two firewalls and dedicated network interfaces, protects the systems from network threats, originating from both the Internet and the local network. It also supports secured isolation of network resources, which increases the level of security. Furthermore, our proposal for FDE ensures an enhanced level of data privacy. In terms of data synchronization, we discussed a protocol that can be used to perform data synchronization by avoiding performance bottlenecks through reduced message exchanges during synchronization. For the reliability, we give an overview of the main reliability fault of a CVDC and corresponding agreement issues between the DCs. From this, we propose a new research direction to provide a certain level of resource availability and reliability. Therefore, based on our realization discussion, we believe that our architecture is feasible and will reduce capital and operational costs and enhance data protection. Copyright c 2015 SERSC 149

14 References [1] The Reverse Firewall: Defeating DDoS Attacks Emerging from Local Area Networks, [2] Cyber Monday Traffic Increases 26 Percent, Amazon Again Top Retail Site, (2007), amazon-again-topretail-site /. [3] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt and A. Warfield Xen and the art of virtualization, In ACM symposium on Operating systems principles, (2003). [4] R. Bose, S. Sahana and D. Sarddar, An Energy Efficient Dynamic Schedule based Server Load Balancing Approach for Cloud Data Center, Int l Journal of Future Generation Communication and Networking, vol. 8, no. 3, (2015) June, pp [5] E. M. Chan, J. C. Carlyle, F. M. David, R. Farivar and R. H. Campbell, Boot-jacker: compromising computers using forced restarts, In ACM Conference on Computer and communications security, (2008). [6] T. Chandra, R. Griesemer and J. Redstone, Paxos Made Live - An Engineering Perspective, In ACM PODC, (2007). [7] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes and R. E. Gruber, Bigtable: A Distributed Storage System for Structured Data, In ACM OSDI, (2006). [8] G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall and W. Vogels, Dynamo: Amazon s Highly Available Key-value Store, In ACM SOSP, (2007). [9] Gartner, Inc. Gartner: Data Centres Account for 23% of Global ICT CO2 Emissions, (2007) November, [10] S. Ghemawat, H. Gobioff and S.-T. Leung, The google file system, In ACM SOSP, (2003). [11] S. Graupner, V. Kotov and H. Trinks, Resource-Sharing and Service Deployment in Virtual Data Centers, In International Conference on Distributed Computing Systems, (2002), pp [12] A. Greenberg, J. Hamilton, D. A. Maltz and P. Patel, The Cost of a Cloud: Research Problems in Data Center Networks, ACM SIGCOMM Computer Communication Review, vol. 39, no, 1, (2009) January, pp [13] S. Grobman, N. Smith and P. Parke, Making Security Practical in the Enterprise with Client Technologies, (2007). [14] J. A. Halderman, S. D. Schoen, N. Heninger, W. Clarkson, W. Paul, J. A. Calandrino, A. J. Feldman, J. Appelbaum and E. W. Felten, LestWe Remember: Cold Boot Attacks on Encryption Keys, In USENIX Security Symposium, (2008) July. [15] M. Henry, D. Koeppen, E. Dittert and V. Viswanathan, Intel Preboot Execution Environment, Internet draft, IETF, (1999) June. [16] J. Hromkovic, C. Klasing, B. Monien and R. Peine. Dissemination of information in interconnection networks. In Combinatorial Network Theory, (1995), pp [17] T. Huth, J. Freimann, V. Zimmer and D. Thaler, DHCPv6 option for network boot, Internet draft, IETF, (2009) April. [18] Intel Corporation, Preboot Execution Environment (PXE) Specification, Specification Version 2.1, Intel Corporation, (1999) September. [19] J. Kirch, Virtual Machine Security Guidelines Version 1.0. The Center for Internet Security, The Center for Internet Security, (2007) September. [20] V. Kotov, On Virtual Data Centers and Their Operating Environments, Technical Report HPL , HP Labs Technical Report, (2001) March. [21] D. W. Krumme, G. Cybenko and K. N. Venkataraman, Gossiping in minimal time, In SIAM J. Comput., (1992), pp [22] E.-K. Lee, Synchronization of Data and Resources in Distributed, Cooperative Virtual Data Centers, In Int l Conf. on Information Science and Industrial Applications, (2015) September. [23] J. M. and V. S. Dynamic Host Configuration Protocol (DHCP) Options for the Intel Preboot Execution Environment (PXE), RFC 4578, IETF, (2006) November. [24] E. Maiwald, Network Security: A Beginner s Guide, McGraw-Hill/Osborne, second edition, (2003). [25] E. M. Maximilien, Green Computing, University of California, Los Angeles, (2009) June. [26] R. Miller, Who Has the Most Web Servers?, (2009) May, com/archives/2009/05/14/whos-got-the-most-web-servers/. [27] T. Ormandy, An Empirical Study into the Security Exposure to Hosts of Hostile Virtualized Environments, Technical report, Google, Inc., (2007). [28] K. Rankin, Cold boot attack tools for linux, Linux J., (2009), pp Copyright c 2015 SERSC

15 [29] T. Schutt, F. Schintke and A. Reinefeld, Efficient synchronization of replicated data in distributed systems, In Prentice-Hall Inc., (1995). [30] Seagate Corporation, DriveTrust Technology: A Technical Overview. Technology Paper TP564, Seagate Corporation, (2006) October. [31] B. Sotomayor, K. Keahey and I. Foster, Combining batch execution and leasing using virtual machines, In ACM International Symposium on High Performance Distributed Computing, (2008). [32] U.S. Department of Energy, Five Ways to Reduce Data Center Server Power Consumption, (2008), energymatters/articles.cfm/article_i d=289. [33] R. White and T. Abels, Energy Resource Management in the Virtual Data Center, In IEEE International Symposium on Electronics and the Environment, (2004) May, pp Author Eun-Kyu Lee, he is an assistant professor in the Department of Information and Telecommunication Engineering of Incheon National University, Korea. He has been involved in many research projects in the fields of smart grid, vehicular network, location-based services, and Telematics. He received Ph.D. in Computer Science from UCLA in His recent research interest includes the Internet of Things - wireless networking, cyber-physical security, middleware systems, and mobile edge computing. Copyright c 2015 SERSC 151