RCL: Software Prototype

Transcription

1 Business Continuity as a Service ICT FP RCL: Software Prototype D3.2.1 June 2014

2 Document Information Scheduled delivery Actual delivery Version 1.0 Responsible Partner IBM Dissemination Level PU Public Revision History Date Editor Status Version Changes Ronen Kat Draft 0.1 Outline Yossi Draft 0.2 Added section 3 Kuperman Ronen Kat Draft 0.3 Merge IBM and RedHat updates Ronen Kat Draft 0.4 Added introduction and finishing Ronen Kat Draft 0.5 Input from internal reviewers Dave Gilbert Draft 0.6 Address comments from reviewers on Red Hat sections Ronen Kat Final 1.0 Finishing Contributors Ronen Kat IBM, Yossi Kuperman - IBM Dave Gilbert Red Hat, Andrea Arcangeli Red Hat Internal Reviewers Vasileios Anagnostopoulos Luis Tomás Copyright This report is by IBM and other members of the ORBIT Consortium Its duplication is allowed only in the integral form for anyone's personal use and for the purposes of research or education. Acknowledgements The research leading to these results has received funding from the EC Seventh Framework Programme FP7/ under grant agreement n /15

3 Glossary of Acronyms Acronym D DoW EC PM PO WP MTU Definition Deliverable Description of Work European Commission Project Manager Project Officer Work Package Maximum Transition Unit 3/15

4 Table of Contents 1. Executive Summary Introduction I/O Consolidation Memory Consolidation and Externalization Cloud Management References /15

5 List of Figures Figure 1: Split I/O deployment...8 Figure 2: Memory externalization nested test-bed...11 List of Tables Table 1: I/O consolidation: components internal names...9 Table 2: I/O consolidation: status of components...9 Table 3: I/O consolidation: status of API...10 Table 4: Memory consolidation and externalization: status of components /15

6 1. Executive Summary This document summarizes the prototype development work done as part of WP3. For this project interval, the first nine months of the project, we report for Task 3.1 (T3.1) and Task 3.2 (T3.2) as Task 3.3 (T3.3) has not yet started. Work for developing the I/O consolidation layer (T3.1), and memory consolidation and externalization layer (T3.2) is well under way and the development completeness coverage is well beyond the required 25% of this deliverable per the quality plan in D1.2.1 [1]. The next deliverable of the prototype D3.2.2 is scheduled for June month 21 of the project. 6/15

7 2. Introduction This first software prototype delivery realizes the design and specification outlined in deliverable D3.1.1 [2]. The work and status of developing the I/O consolidation layer is described in Chapter 3, the work and status of memory consolidation and externalization layer is described in Chapter 4, and the work and status of the cloud management is described in Chapter Progress toward feature and API completeness Development is well under way, and progress of the feature completeness is close to (or even above) 50% of the planned features. The progress for the API development is about 25% for the I/O consolidation layer and above 50% percent for the memory consolidation and externalization layer. Work on cloud management has not started yet, and is scheduled to start on October 2014 per the project plan. 7/15

8 3. I/O Consolidation Our objective is to externalize I/O resources and consolidate all of them in a single dedicated appliance. We did so by detaching the back-end logic responsible for handling I/O from the hypervisor software stack and moved this logic from all the compute servers used to host the VMs to a remote server dedicated for I/O virtualization. The prototype is comprised from the components described in deliverable D3.1.1, the internal names of the components are listed in Table 1, and the status is listed in Table Status of prototype The prototype implementation is capable of creating block and network virtual devices. The exposed virtual devices are partially functional and not yet ready to be deployed. It is possible to communicate over a virtual network device with the load generator and read/write a block of data from a remote block device that resides at the I/O hypervisor memory. We deployed Split I/O prototype on 3 machines in our lab (depicted at Figure 1 from right to left): the load generator, the I/O hypervisor (back-end) and the host that runs the VM (frontend). Each machine is an IBM System x3550 M4, equipped with two 8-cores sockets of Intel Xeon E2660 CPU running at 2.2 GHz. Each machine is further equipped with 56GB of memory and an Intel x520 dual port 10Gbps SRIOV NIC. Figure 1: Split I/O deployment The prototype is implemented for Linux 3.9 kernel as a set of new kernel modules. Each module corresponds to a component described at deliverable (see Table 1). Modules 3.1.1, , , and are installed on the I/O hypervisor machine, and modules modules 3.1.1, , and are installed on the VM. Note that module is installed both on the VM and I/O hypervisor. Note that component (Ethernet transport) is used by both the front-end and the backend. Its main purpose is to facilities with data transportation from both ends. It does so efficiently by using layer 2 (MAC layer) and thus avoids higher layers such as TCP/IP which incur high overhead. As block I/O requests have arbitrary sizes, and the size of the network packet that can be sent over the wire size is bounded by a Maximum Transition Unit (MTU), component should fragment the requests before sending them to the other end for processing. The fragment size is determined by the MTU. To instantiate a virtual device, we created a special user-space utility that operates the (generic back-end) kernel module (module ). Invoking the utility on the I/O hypervisor with the following details: device type (block or net), backing device (e.g. local block device) 8/15

9 and a remote VM s address, will pass the request to the I/O hypervisor kernel (via ioctl), which in turn instructs the VM to expose a virtual device. Bellow is a mapping between our internal modules names and the components described at D Component Module Name Installed On Split I/O Ethernet Transport vrio_eth.ko VM + I/O hypervisor Split I/O Generic Front-End vrio_generic.ko VM Split I/O Block Front-End vrio_gblk.ko VM Split I/O Net Front-End vrio_gnet.ko VM Split I/O Generic Back-End vrio_generic.ko I/O hypervisor Split I/O Block Back-End vrio_hblk.ko I/O hypervisor Split I/O Net Back-End vrio_hnet.ko I/O hypervisor Split I/O management module vrio.py I/O hypervisor Table 1: I/O consolidation: components internal names Notes: 1. vrio (virtual Remote I/O) is the internal development name for Split I/O. 2. Module vrio_generic.ko is used both for the generic front-end and the generic back-end External interactions Management for the Split I/O hypervisor is provided through the split I/O hypervisor python module. Status of development is described in the Table 2. The management library will be used by the cloud management layer as part of T Completion status of components In the next table (Table 2) we show the development status of split I/O modules included in this prototype. Component Status and progress Split I/O Ethernet Transport 70% completed Split I/O Generic Front-End 70% completed Split I/O Block Front-End 40% completed Split I/O Net Front-End 50% completed Split I/O Generic Back-End 70% completed Split I/O Block Back-End 40% completed Split I/O Net Back-End 50% completed Split I/O management module Not started Table 2: I/O consolidation: status of components In the next table (Table 3) we show development status of the split I/O API functions. 9/15

10 API RESET_GUEST_DEVICES() CREATE_BLOCK_DEVICE() REMOVE_BLOCK_DEVICE() CREATE_NETWORK_DEVICE() REMOVE_NETWORK_DEVICE() Table 3: I/O consolidation: status of API Status and progress Not started 80% completed Not started 80% completed Not started 10/15

11 4. Memory Consolidation and Externalization Our objective is to provide a mechanism that allows the hypervisor to retrieve memory pages from a remote system for use in a VM; this is being used as part of a post-copy migration implementation where a migrated VM starts running on the destination host prior to all of the memory being copied over Status of prototype The prototype is capable of performing small post-copy migrations, with page requests making the full round trip in limited situations, but is incomplete and not yet stable. The prototype consists of modifications to both QEMU and the Linux kernel (v3.13 currently). The components are as described in deliverable The current deployment is within a testing environment consisting of a nested hypervisor allowing all components to be easily tested on one machine as shown in Figure 2. The two QEMU instances are the source (left) and destination which has a partial copy of it's memory. The Linux kernel running in the L1 guest includes the 'Linux kernel mm subsystem enhancements' (component 3.2.1). These allow the destination QEMU instance to mark an area of memory as 'userfault' i.e. external, which casues QEMU to be notified when the guest accesses the page. At a later point in time, the QEMU uses another kernel modification to atomically (and efficiently) move a page into place to satisfy the previous request. The QEMU running in the L1 guest contains the modifications from components and 'Remote memory front end' routes page requests/data between the kernel on the destination machine and the network towards the source machine. Requests from the kernel are recorded in a page ownership datastructure, and sent as requests along a 'return path' to the source VM 'Remote memory handler' satisfies page requests from the destination machine and routes control messages to the Remote memory front end; these include the messages that initiate the transition to postcopy. 11/15

12 Host system (running Linux with unmodified KVM) Host (unmodified) QEMU instance: L1 guest (running modified Linux kernel) L1 modified QEMU instance (1): L1 modified QEMU instance (2): L2 guest (unmodified) L2 guest (unmodified) Figure 2: Memory externalization nested test-bed. The existing QEMU migration protocol has been modified to include commands to initiate the postcopy mode, and to provide a bidirectional transport allowing the destination to request pages. The mode is enabled using an extra migration-capability flag External interactions None yet Completion status of components The status reflects the status of a prototype that is starting to work; the basics are there but need filling out, making more robust and tidying up before submission to the upstream projects. Component Status and progress Linux kernel mm subsystem enhancements 60% completed Remote memory front end 50% completed Remote memory handler 50% completed Table 4: Memory consolidation and externalization: status of components 12/15

13 4.4. Additional notes The next step is to finish the functionality and stabilise the current version so that arbitrary guests can be transferred. 13/15

14 5. Cloud Management The cloud management is part of Task 3.3 which is scheduled to start at M13 (October 2014). Therefore, no components of the cloud management are included in this deliverable. 14/15

15 6. References [1]. ORBIT document - D Quality Plan, February 12, 2014 [2]. ORBIT document - D RCL Design And Open Specification 15/15