Enhancing Hypervisor and Cloud Solutions Using Embedded Linux Iisko Lappalainen MontaVista

Transcription

1 Enhancing Hypervisor and Cloud Solutions Using Embedded Linux Iisko Lappalainen MontaVista

2 Setting the Stage This presentation will discuss the usage of Linux as a base component of hypervisor components particularily in the context of creating a base for a cloud hosting framework. The basis for the discussion is formed by combining technologies like KVM/QEMU, OpenVSwitch, OpenFlow and OpenStack to form a technical base for the solution, and digging into particular problem areas to see where the experience from the embedded world can help in creating a more competent experience. The presentation also discusses the general characteristics of the above technologies to create an overview of the problem area. In particular the issues that are key to creating a competetive solution are the responsiveness and throughput of the system, the capability for providing logging and tracing facilities across the hypervisor and guest domains, the seamless integration of the components to provide stability and maintainability and providing a suitable level of security throughout the layers of SW in the system. 2

3 Agenda Key Components and Architecture Main Architecture OpenStack OpenvSwitch QEMU/KVM Virtualization Problems and Solutions Further Work on the Topic

4 Example Hypervisor Solution Architecture Carrier Grade Services (HA, Tracing, Logging, Security, CGL) Guest VM KVM/QEMU incl. HW acceleration OpenVSwitch Guest Container LXC OpenFlow OpenStack Management and Provisioning SW and Services OS layer and base services Hardware

5 QEMU and KVM hypervisor solutions At least it will be looking forward Features QEMU: Comprehensive architecture support X86, ARM, MIPS, PPC, etc QEMU: Support for particular board configurations QEMU :Live Migration of guests -based KVM: Integration with HW virtualization capabilities for x86, PPC, ARM Best support on x86 currently ARM seeing a lot of work currently

6 OpenVSwitch OpenVSwitch features Virtual switching ports and connection to physical switch HW Support for OpenFlow control Support for standards such NetFlow, sflow, SPAN What does it provide Facilitates live-migration Much easier to replicate state between hosts Abstraction API Similar interfaces to all switches accessed Performance

7 OpenFlow Software Defined Networking (SDN) means to separate the control plane from the data plane in network switches and routers to different components The expectation is that this leads to massively more scalable network infrastructure The initiative for the standardization of OpenFlow is being led by major end-users like Google and Facebook and not really the equipment vendors OpenFlow is split into target and controller domains API is in essence a adaptation layer Both types of OSS SW for Linux available

8 Open Stack Cloud Management OpenStack is a cloud management system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.

9 OpenStack OpenStack features Control APIs for managing VM images and setups Integration to multiple cloud subsystems User interface, dashboard OpenStack Components Quantum-network, Nova-compute, Keystone-identity Cinder-block storage, Swift-object storage Glance-image service Savanna-Hadoop service What does it provide OpenStack is a key higher-level components in our Cloud solution of today A Cloud Framework like Openstack is usually required in all real Cloud deployments

10 Agenda Key Components and Architecture Main Architecture OpenStack OpenvSwitch QEMU/KVM Virtualization Problems and Solutions Further Work on the Topic

11 Hypervisor Pain Points Real-time Latency Application Guest Network Throughput Tracing, Logging, Security functionality Memory Access Speed HW-independent acceleration API Guest VM KVM/QEMU incl. HW acceleration Kernel Layer Hardware PIC NIC Memory

12 Hypervisor Pain Points Security and Isolation Application Multi-OS support Tracing, Logging, Security functionality Forward portability HW-independent acceleration API Guest VM KVM/QEMU incl. HW acceleration Kernel Layer Hardware PIC NIC Memory

13 Hypervisor Pain Points Application Full-stack Management Interfaces Tracing, Logging, Security functionality HW-independent acceleration API Guest VM KVM/QEMU incl. HW acceleration Kernel Layer Hardware PIC NIC Memory Custom HW adaptation

14 Background: Real-Time / Context Switching Application 1) The interrupt event will enable a thread for running, usually invoking the host-side scheduler first Tracing, Logging, Security functionality Guest VM Scheduler KVM/QEMU 2) The scheduler in the guest-side will get activated to actually thread inside the guest domain 3) The application running in the thread will process the event Kernel Layer Hardware PIC Scheduler ISR NIC -> the critical components are the context switching time and the RT latency related to switching to the right thread

15 Use-case: Real-Time / Context Switching Tracing, Logging, Security functionality Video Stack Application Guest VM Scheduler KVM/QEMU 1) The streaming video protocol stack and endpoint have to process packets at a certain continuous rate to avoid dropping the connection or having low quality picture transfer 2) The application running the stack is dependant on the scheduling mechanism on the guest and host Kernel Layer Hardware PIC Scheduler ISR NIC 3) The timer interrupts are usually controlled by the HW and are used to control the timing, sometimes a network synchronized time source is used

16 Solution: Real-Time / Context Switching Video Stack Application 1) Enhance latency response by integrating Real-Time Linux to the hypervisor Tracing, Logging, Security functionality Scheduler Guest VM KVM/QEMU 2) Improve the context switching response by customizing the IPI interrupt handling for the VM guest Scheduler Kernel Layer ISR Hardware PIC NIC Page 16

17 Background: Network Throughput Tracing, Logging, Security functionality Kernel Layer Hardware Guest Network Throughput Host Network Throughput NIC Application Guest VM KVM/QEMU 1) The application inserts data into the OS mechanisms that control the HW packet transmission inside the guest VM 2) The network handler in the guest OS will feed the data to the virtual interface (for example virtio) 3) The host OS will handle transmission from the virtual interface to the actual HW, possibly multiplexing the HW IP interface -> the critical components are the propagation time from the guest to the host and the effectiveness of the host IP stack Page 17

18 Use-Case: Network Throughput Control Plane Server Application 1) A control plane server application is used to provide large amount Tracing, Logging, Security functionality Kernel Layer Guest Network Throughput Host Network Throughput Guest VM KVM/QEMU 2) It uses standard networking to send data from inside a virtualized guest OS, which is used to provide load-balancing and failover facilities in the total system. 3) The packet traffic is routed through the guest and host OS and if affected by delays at each of the points, therefore affecting the total capacity of the server Hardware NIC

19 Solution: Network Throughput Tracing, Logging, Security functionality Guest Network Throughput Control Plane Server Application Host Network Throughput Guest VM KVM/QEMU 1) Implement packet handling on the host to bypass userspace and enter the kernel or dedicated HW directly Eliminate processing for traffic copy Eliminate context transitions 2) Implement control mechanisms for prioritizing and balancing traffic in case of multiple VMs Allow better QoS capabilities for higher-priority functions, like streaming data Kernel Layer Hardware NIC

20 Multi-OS Support Several Cloud users especially targeting embeddedtype deployments have significant SW legacy HW consolidation of legacy OSs Multi-version support for forward portability future versions of the platform Issues faced Real-time IO performance However, usually new HW can do much better in relation..

21 Security and Isolation Hypervisor and Guest-to-Guest vulnerabilities are seen as new fruitful ground for hackers Multi-tenancy Multi-node consolidation Guests of lesser importance can be used to gain access to the hypervisor and the more important Guest domains Hypervisor priviledge escalation Careful management of CVEs needed SELinux - svirt Not only exploits, also important data must not be visible to non-authorized Guests Packet traffic Memory, IO

22 Full-Stack Management Interfaces Key usability characteristic of a Cloud solution is the accessibility to the provisioning and monitoring systems Custom GUIs for end solutions Need access to APIs exposed by underlying facilities Monitoring Solutions Hypervisor and Guest logging OpenStack interface adaptation OpenStack plugin adapation Libvirt as a key underlying facility

23 Agenda Key Components and Architecture Main Architecture OpenStack OpenvSwitch QEMU/KVM Virtualization Problems and Solutions Further Work on the Topic

24 The Way Forward Co-operative Scheduling Paravirtualized Easy Adaptation of Applications Across HW and Middleware Adaptation layer for DPDK, SE, LWE, RTE etc Performance and Manageability OpenStack Value-add-solutions

25 Thank you! Questions? Comments on chosen components?