Development of Type-2 Hypervisor for MIPS64 Based Systems

Transcription

1 Development of Type-2 Hypervisor for MIPS64 Based Systems High Performance Computing and Networking Lab Al-Khwarizmi Institute of Computer Science University of Engineering & Technology Lahore Pakistan

2 Agenda Introduction Scope of Project Type-II Hypervisor for MIPS64 Based Systems Design Architecture and Execution Flow Challenges Demonstration

3 Introduction

4 Virtualization Creating software representation of a device or resource E.g. server, storage device, network, OS Broadly classified as Server Virtualization Client or Desktop virtualization Network virtualization

5 Applications of Virtualization Cloud services Security E.g. Cloud user isolation Optimal hardware utilization E.g. Multiple services on a single machine Software development and testing E.g. Legacy code execution Dynamic load balancing in data centers E.g. Server migration

6 Hypervisor A piece of software/firmware Used to create Virtual Machines (VMs) Provides a virtual environment to run guest OS Type-1 (aka native, bare metal) Type-2 (aka hosted)

7 Scope of Project

8 Virtualization Solutions Virtualization software Type1 Hyper-V, Xen, Vmware ESX Type 2 Virtual PC/Server, Vmware Workstation, Virtual Box Available only for x86 A Hypervisor or Virtual Machine Monitor (VMM) for MIPS is not available

9 Virtualization on MIPS Relatively less explored area Recent work on type-i Longsoon MIPS hypervisor No published work on type-ii MIPS hypervisor MIPS is typically used in embedded systems E.g. Switches, routers, smart TV, set-top boxes Virtualization enabling software can increase the hardware utilization of these embedded systems

10 Type-II hypervisor for MIPS64 Based Systems

11 Important Features Full-Virtualization solution for MIPS MIPS on MIPS Virtualization enabling application Multiple MIPS64 guest OS on single Host machine running linux Multi-cores design Implementation of basic devices Developed on Cavium OCTEON family

12 Type-II Hypervisor Guest is a user level process of Host OS Multiple VMs could be created G. G. G. App 1 App 2... App n Guest Kernel Guest U-Boot : Hypervisor Other processes MIPS Linux MIPS Hardware

13 Multiple VMs Scenario VM 1 VM 2 VM n G. App 1 G. App 2... G. App n G. App 1 G. App 2... G. App n G. App 1 G. App 2... G. App n Guest Kernel Guest U-Boot Guest Kernel Guest U-Boot... Guest Kernel Guest U-Boot : : : Other processes Hypervisor Hypervisor Hypervisor MIPS Linux MIPS Hardware

14 Design Architecture and Execution Flow

15 Design Three major parts of hypervisor ISA emulation Memory management External Device implementation Combined implementation of all three results in a Virtual board

16 Virtual Board A Multithreaded Linux process Cores and devices are threads

17 Block Level ISA Emulation Basic blocks of guest instructions are extracted A stream of straight line code ending with a control shift instruction Whole block is translated to a set of safer instructions Translated block is executed on hardware Block Fetching Identify basic block Instruction Translation Replace each instruction with safer instruction(s) Block Execution Host OS executes translated code

18 Translation For each valid block 1. Translate each instruction of block into a set of unprivileged instructions 2. Concatenate sets of translated instructions to form the translated Block

19 Execution Translated block is executed directly on hardware Control may shift back to hypervisor due to following reasons: Address translation for load/store instructions TLB operations System call External interrupt End of current block Block Execution Block Fetching Block Translation

20 Memory Management Physical memory Virtual memory Virtual to physical memory mapping is handled by kernel Simplest virtual memory view of a system Virtual memory map Mapped kernel segment (kseg 2) Unmapped uncached kernel segment (kseg1) Unmapped cached kernel segment (kseg0) User space (kuseg)

21 MIPS Memory Map (64-bit View )

22 Guest s Memory Management Hypervisor provide 1-1 mapping of guest memory Using Mmap() system call GVA to HVA translation Identify memory segment check for GPA in TLB If found, get GPA and then Get HVA from hash-map else, generate guest exception Guest virtual address (GVA) Guest Physical Address (GPA) Host Virtual Address (HVA) Hypervisor Load/store from HVA

23 External Devices Multi-threaded approach for devices Implemented UART (Universal Asynchronous Receiver/Transmitter) CIU (Central Interrupt Unit) Timer Network Card

24 UART Used for serial communication OS transmits and receives bytes Processor communicates with console & keyboard Implementation in hypervisor Receiver thread Continuously check arrival of new data from keyboard Generate interrupt on receiving Handover the data Transmitter thread Sends data from guest to console

25 Central Interrupt Unit (CIU) Cavium Specific Unit Provides communication between devices and cores Routes the interrupt from devices Using summary and interrupt enable hardware registers

26 CIU Implementation Continuously observes devices interrupt signals Reads corresponding bits of device registers Create summary registers If any device wants to interrupt a CPU core CIU writes cause register to indicate the interrupt

27 On-Demand Timer Embedded in core thread Register timer on guest s demand Reads time from host and presents to guest in nanosecond resolution Much better performance

28 Challenges

29 Technical Challenges We used Cavium Octeon CN57XX as reference board CVM segment CVM specific instruction Customized instructions e.g. BBIT0, MADD, 64 bit multiplication instruction (VMULU) Partially documented Continuously pinging Cavium support staff Or doing rigorous testing on reference board

30 Technical Challenges Testing and verification Unexpected guest behavior e.g. halts, illegal instruction Guest instruction treated as data on host level Soln. Flush Cache after block translation Block fetching from non-contiguous pages Instruction emulation verification LL/SC, atomic load/store, madd

31 High Boot time Technical Challenges Hurdle in development Optimization efforts Cache implementation In-place execution Structural changes Inline functions Macros Customized data structure Improvements in Timer infrastructure

32 Demo

33 Thanks for Listening!

34 Ask questions to find out something about the world itself, not to find out whether or not someone knows it. John Holt