Efficient Implementation of the bare-metal Hypervisor MetalSVM for the SCC Public Release of MetalSVM 0.1 Pablo Reble, Jacek Galowicz, Stefan Lankes and Thomas Bemmerl MARC Symposium, Toulouse CHAIR FOR OPERATING SYSTEMS
Introduction What is MetalSVM? Our Progress with MetalSVM Main Features Performance How to Get and Use MetalSVM Key-Features of MetalSVM 2 Chair for Operating Systems
What is MetalSVM? a minimal operating system for the SCC spin-off from eduos - a kernel developed at RWTH Aachen for educational purposes monolithic, MULTIBOOT-compliant 32 bit lightweight kernel unix-like programming interface sophisticated exploitation of SCC hardware merged with our ircce communication lib Key-Features of MetalSVM 3 Chair for Operating Systems
What is MetalSVM? First public release Version 0.1 Free Software Apache License 2.0 Is probably a great springboard for your bare-metal programming great mascot Mike the Ostrich Key-Features of MetalSVM 3 Chair for Operating Systems
Our Progress with MetalSVM run apps with small efficient kernel Integrate ircce for fast inter-core communication Establish SVM between multiple cores Paravirtualize Linux: One instance per core Run one guest on multiple cores [in progress] App App Kernel Kernel core 0 core n SCC Hardware Key-Features of MetalSVM 4 Chair for Operating Systems
Our Progress with MetalSVM run apps with small efficient kernel Integrate ircce for fast inter-core communication Establish SVM between multiple cores Paravirtualize Linux: One instance per core Run one guest on multiple cores [in progress] App Kernel Communication Layer App Kernel core 0 core n SCC Hardware Key-Features of MetalSVM 4 Chair for Operating Systems
Our Progress with MetalSVM run apps with small efficient kernel Integrate ircce for fast inter-core communication Establish SVM between multiple cores Paravirtualize Linux: One instance per core Run one guest on multiple cores [in progress] MetalSVM Kernel Application Communication Layer Kernel core 0 core n SCC Hardware Key-Features of MetalSVM 4 Chair for Operating Systems
Our Progress with MetalSVM run apps with small efficient kernel Integrate ircce for fast inter-core communication Establish SVM between multiple cores Paravirtualize Linux: One instance per core Run one guest on multiple cores [in progress] MetalSVM App App Linux Linux Hypervisor Hypervisor Kernel Communication Layer Kernel core 0 core n SCC Hardware Key-Features of MetalSVM 4 Chair for Operating Systems
Our Progress with MetalSVM run apps with small efficient kernel Integrate ircce for fast inter-core communication Establish SVM between multiple cores Paravirtualize Linux: One instance per core Run one guest on multiple cores [in progress] MetalSVM Application Paravirtualized Linux Hypervisor Kernel Communication Layer Kernel core 0 core n SCC Hardware Key-Features of MetalSVM 4 Chair for Operating Systems
Main Features Main Features running apps on MetalSVM is like running them bare-metal but not without the comfort of an OS! fast familiarization due to imitated aspects of Unix-like systems ircce included: fast communication core core functionality of RCCE + non-blocking message-passing event-driven Mailbox extension lwip included: TCP-IP communication core core SCC MCPC use BSD sockets in your user space app SVM system PGAS-like functions (explicit svm_alloc(), etc.) strong and lazy release consistency models Key-Features of MetalSVM 5 Chair for Operating Systems
Main Features Main Features (2) low-latency synchronization layer utilizing SCC-specific test and set- & atomic increment registers fast scheduling Round-Robin based with priority support configurable as timeslice based or tickless device drivers use your own character devices from user space basic file system populated from initial ramdisk easily customizable /dev interface for devices newlib C library for user space included SMP support (not on SCC) x86_64 version in progress Key-Features of MetalSVM 6 Chair for Operating Systems
Main Features Kernel Structure Application Programming Interface IPC scheduler timer I/O semaphore mutex mailbox running task ready tasks blocked tasks driver Hardware kernel structure of eduos/metalsvm Key-Features of MetalSVM 7 Chair for Operating Systems
Performance ircce Performance Throughput [MByte/s] 150 100 50 0 RCCE V1.0.3.13 ircce Memcpy pipelined ircce 1 2 4 8 16 32 64 256 1 k 4 k 16 k 64 k 256 k 1 M Packet Size [Byte] Comparing RCCE (V1.0.3.13) and ircce Source: ircce documentation Key-Features of MetalSVM 8 Chair for Operating Systems
Performance emac Device Driver Performance Throughput [MByte/s] 40 30 20 10 MCPC SCC 0 1 4 16 64 256 1 k 4 k 16 k Packet Size [Byte] 40 30 20 10 SCC MCPC Linux (2.6.38.3-jb) emac device MetalSVM emac device L1 cache 0 1 4 16 64 256 1 k 4 k 16 k Packet Size [Byte] Source: The Path to MetalSVM: Shared Virtual Memory for the SCC Key-Features of MetalSVM 9 Chair for Operating Systems
Performance Memory Mapped IP Driver Performance Throughput [MByte/s] 80 60 40 20 Full lwip version Linux (2.6.38.3-jb) MPB device MetalSVM MM device MetalSVM MM device L1 cache MetalSVM MM device via MPB 0 1 4 16 64 256 1 k 4 k 16 k Packet Size [Byte] 80 60 40 20 Sending packets from tile 0 to tile 1 Bypassing version 0 1 4 16 64 256 1 k 4 k 16 k Packet Size [Byte] Source: The Path to MetalSVM: Shared Virtual Memory for the SCC Key-Features of MetalSVM 10 Chair for Operating Systems
Performance SVM Performance Jacobi Over Relaxation algorithm Problem size 1024 512 SCC Platform running with 533 MHz core and 800 MHz memory/mesh Time [s] 400 300 200 100 0 1 2 4 8 16 32 48 Number of Cores ircce lazy release strong Source: Revisiting Shared Virtual Memory Systems for Non-Coherent Memory-Coupled Cores Key-Features of MetalSVM 11 Chair for Operating Systems
Performance Scheduling Overhead 30 10 3 Linux 2.6.38.3-jbrummer MetalSVM MetalSVM tickless 20 ticks 10 6.8 1.9 0 0 1,000 2,000 iteration Source: Efficient Implementation of the bare-metal Hypervisor MetalSVM for the SCC Key-Features of MetalSVM 12 Chair for Operating Systems
How to Get and Use MetalSVM Okay, I am interested! What now? How about getting a copy and playing with it? Key-Features of MetalSVM 13 Chair for Operating Systems
How to Get and Use MetalSVM Get it Just clone MetalSVM from our official git repository... $ git clone git://git.lfbs.rwth-aachen.de/metalsvm.git... do initial setup... $ cd metalsvm $ cp Makefile.scc Makefile $ (cd include/metalsvm; cp config.h.scc config.h)... build... $ make... and run on SCC: $ (cd tools; make SCC && sccboot -g obj) $ sccreset -r Key-Features of MetalSVM 14 Chair for Operating Systems
How to Get and Use MetalSVM Get it Just clone MetalSVM from our official git repository... $ git clone git://git.lfbs.rwth-aachen.de/metalsvm.git... do initial setup... $ cd metalsvm $ cp Makefile.scc Makefile $ (cd include/metalsvm; cp config.h.scc config.h)... build... $ make... and run on SCC: $ (cd tools; make SCC && sccboot -g obj) $ sccreset -r Key-Features of MetalSVM 14 Chair for Operating Systems
How to Get and Use MetalSVM Get it Just clone MetalSVM from our official git repository... $ git clone git://git.lfbs.rwth-aachen.de/metalsvm.git... do initial setup... $ cd metalsvm $ cp Makefile.scc Makefile $ (cd include/metalsvm; cp config.h.scc config.h)... build... $ make... and run on SCC: $ (cd tools; make SCC && sccboot -g obj) $ sccreset -r Key-Features of MetalSVM 14 Chair for Operating Systems
How to Get and Use MetalSVM Get it Just clone MetalSVM from our official git repository... $ git clone git://git.lfbs.rwth-aachen.de/metalsvm.git... do initial setup... $ cd metalsvm $ cp Makefile.scc Makefile $ (cd include/metalsvm; cp config.h.scc config.h)... build... $ make... and run on SCC: $ (cd tools; make SCC && sccboot -g obj) $ sccreset -r Key-Features of MetalSVM 14 Chair for Operating Systems
How to Get and Use MetalSVM Get it (2) Connect and send commands: $ telnet rck00 4711 Build documentation: $ doxygen see documentation/html/index.html Key-Features of MetalSVM 15 Chair for Operating Systems
How to Get and Use MetalSVM Configure it include/metalsvm/config.h configure if you want to include support for PCI, lwip, VGA, software-uart, Keyboard, Multiboot/SCC, change timeslice size, enable tickless scheduling, etc. Makefile Depending on wether you run MetalSVM in qemu/on SCC set compiler, etc. Key-Features of MetalSVM 16 Chair for Operating Systems
How to Get and Use MetalSVM Customize it The most important files and directories apps/ kernel space apps newlib/examples/ user space apps apps/tests.c the init daemon drivers/char/ character device drivers tools/ mostly SCC-specific build stuff Key-Features of MetalSVM 17 Chair for Operating Systems
How to Get and Use MetalSVM Conclusion With MetalSVM as a framework for your research on the SCC, you get: fast, bare-metal like execution of your code basic OS comfort in kernel- and user space fast communication: TCP-IP & ircce blocking & non-blocking, time- & event driven core core, SCC MCPC SVM support a kernel which is easily customizable to your needs Key-Features of MetalSVM 18 Chair for Operating Systems
Finally check out http://metalsvm.org contact@metalsvm.org Questions? Key-Features of MetalSVM 19 Chair for Operating Systems