1 Security Challenges in Virtualized Environments Joanna Rutkowska, Invisible Things Lab Confidence 2008, Krakow, Poland, May 15th, 2008
2 1 Virtualization-based MALWARE 2 Using Virtual Machines for ISOLATION 3 NESTED virtualization
3 Virtualization-based MALWARE
4 Hardware Hardware Hardware Blue Pill OS OS AMD-V Intel VT-x
5 Blue Pill Characteristics NO HOOKS! Cannot be detected using any integrity scanner On the fly installation No boot/bios/etc modifications necessary No I/O virtualization Negligible performance impact (your brand new 3D card will still work!)
6 Blue Pill detection
7 Blue Pill detection Detecting a VMM Detecting virtualization based malware
8 VMM detection Guest time virtualization Direct timing analysis HPET timers Blue Chicken CPU specific behavior TLB profiling
9 VMM detection? Everything is going to be virtualized! Thus the information that there is a hypervisor in the system......would be pretty much useless...
10 Detecting virtualized malware?
11 No Hooks! Search for code Detect activity (e.g. network packets) Heuristics By Pattern Stealth by Design concept Covert channels Nested Page Tables (hardware SPT) Massive malware Simple Obfuscation Won t work 0day malware
12 The whole big deal about Blue Pill is: NO HOOKS in the system!
13 Blue Pill prevention
14 Disable virtualization?
15 How about also disabling your network card so you never got infected from the Internet?
16 Install a trusted hypervisor first?
17 Installing trusted hypervisor Static Root of Trust Measurement Dynamic Root of Trust Measurement BIOS > MBR > VMM e.g. MS Bitlocker SENTER (Intel TXT) SKINIT (AMD SVM)
18 Trusted vs. Secure? SRTM and DRTM only assures that what we load is trusted......at the moment of loading! 3 sec later... it could be exploited and get compromised!
19 Trusted!= Secure known, no bugs not compromised
20 E.g. #1: The famous DMA problem
21 I/O: asks the device to setup a DMA transfer Some device Hardware (Trusted) Hypervisor Read/Write memory access! Some driver OS
22 IOMMU Solution to the problem of DMA attacks Intel calls it: VT-d Not much PC hardware supports it yet Expected to change soon No THIN HYPERVISORS without IOMMU!
23 Other problems with VMMs? Stay tuned...
24 All in all: it s not trivial to have a trusted & secure hypervisor but this is the proper way to go!
25 1 Virtualization-based MALWARE 2 Using Virtual Machines for ISOLATION 3 NESTED virtualization
26 Using Virtual Machines for ISOLATION
27 Originally ISOLATION was supposed to be provided by Operating Systems... Separate processes/address spaces, User accounts & ACLs...
28 But in practice current OSes simply fail at providing isolation!
29 Why OSes fail? Kernel bugs! Kernel bugs!! Kernel bugs!!! Bad design, e.g.: XP and all runs as admin assumption Vista s UAC assumes admin rights should be granted to every installer program!
30 VMMs for the rescue!
31 trusted & secure hypervisor Vista (work projects) Linux + Firefox ( random surfing) Linux + Firefox (online banking) MacOSX ( home, e.g. pics, music, etc)
32 Challenges Performance Why is VMM/hypervisor going to be more secure then OS s kernel?
33 VMM bugs?
34 VMM Bugs Bugs in hypervisors Bugs in additional infrastructure
35 E.g. #1: CVE VMWare ESX Found by Rafal Wojtczuk (McAfee) September 2007 Guest OS can cause memory corruption on the host and potentially allow for arbitrary code execution on the host
36 E.g. #2: CVE Microsoft Virtual Server 2005 R2 Found by Rafal Wojtczuk (McAfee) August 2007 Heap-based buffer overflow allows guest OS to execute arbitrary code on the host OS
37 E.g. #3: CVE Xen Found by Joris van Rantwijk September 2007 By crafting a grub.conf file, the root user in a guest domain can trigger execution of arbitrary Python code in domain 0.
38 E.g. #4: Various Bugs Paper by Tavis Ormandy (Google) April 2007 Disclosed bugs in VMWare, XEN, Bochs, Virtual PC, Prallels A simple fuzzers for: Instruction parsing by VMMs I/O device emulation by VMMs
39 As you see, current VMMs are far from being flawless...
40 To make VMMs more secure we need to keep them ultra-thin and small!
41 Phoenix HyperSpace
44 HyperCore: the type I hypervisor used for HyperSpace
46 The HyperCore Targets desktop/laptop systems Guest OS execute at near-native performance (including fancy graphics) Support for full ACPI (Power Management) Integrity: loaded via SecureCore BIOS (Static Root of Trust Measurement) Very thin - easy to audit!
47 Speeding things up Pass through for most devices SPT: 1-1 mapping for most pages for the Primary OS
48 Power Management ACPI tables exposed to the Primary OS, so that the overall power performance is optimized Efficient intercepts for power management control
49 Integrity Static RTM via Phoenix s SecureCore BIOS Dynamic RTM via Intel s TXT/AMD s SKINIT SMM-based watchdog for HyperCore code
51 1 Virtualization-based MALWARE 2 Using Virtual Machines for ISOLATION 3 NESTED virtualization
52 NESTED virtualization
53 What if a user wants to run e.g. Virtual PC here?
66 Hypervisors expect to have GIF=1 when VMEXIT occurs... They might not be prepared to handle interrupts just after VMEXIT from guests!... but when we resume the nested hypervisor CPU sets GIF=1, because we do this via VMRUN, not VMEXIT...
67 Getting around the GIF Problem We need to emulate that GIF is 0 for the nested hypervisor We stop this emulation when: The nested hypervisor executes STGI The nested hypervisor executes VMRUN How do we emulate it?
68 GIF0 emulation VMCB 1.V_INTR_MASKING = 1 Host s RFLAGS.IF = 0 Intercept NMI, SMI, INIT, #DB and held (i.e. record and reinject) or discard until we stop the emulation
69 Additional details Need to also intercept VMLOAD/VMSAVE Need to virtualize VM_HSAVE_PA ASID conflicts
70 Hypervisor: ASID = 0 Conflicting ASIDs! Nested Hypervisor: ASID = 1 (but thinks that has ASID = 0) Nested Guest: ASID = 1 (assigned by the nested hypervisor)
71 But we can always reassign the ASID in the VMCB prim that we use to run the nested guest.
72 Performance Impact One additional #VMEXIT on every #VMEXIT that would occur in a nonnested scenario One additional #VMEXIT when the nested hypervisor executes: STGI, CLGI, VMLOAD, VMSAVE Lots of space for optimization though
73 Lost already? ;)
74 Don t worry! The main message is...
75 This can be done! & It works!
78 Intel VT-x
79 Nested virtualization on VT-x No GIF bit - no need to emulate GIF0 for the nested hypervisor :) No Tagged TLB - No ASID conflicts :) However: VMX instructions can take memory operands - need to use complex operand parser No tagged TLB - potentially bigger performance impact
80 Nested VT-x: Status We have that working! The VT-x nesting code cannot be published though :(
81 Who else does Nested (hardware-based) Virtualization?
82 IBM z/vm hypervisor on IBM System z mainframe Running z/vm in a virtual machine (that is, z/vm as a guest of z/vm, also known as second-level z/vm) is functionally supported but is intended only for testing purposes for the secondlevel z/vm system and its guests (called third-level guests). -- hcsf8b22.pdf IBM System z10, source: ibm.com
83 Confusion AMD Nested Page Tables!= Nested Virtualization! NPT is a hardware alternative to Shadow Page Tables (a good thing, BTW) NPT is also called: Rapid Virtualization Indexing
Bluepilling the Xen Hypervisor Joanna Rutkowska & Alexander Tereshkin Invisible Things Lab Black Hat USA 2008, August 7th, Las Vegas, NV Xen 0wning Trilogy Part Three Previously on Xen 0wning Trilogy...
Full and Para Virtualization Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF x86 Hardware Virtualization The x86 architecture offers four levels
Virtualization Technology Zhiming Shen Virtualization: rejuvenation 1960 s: first track of virtualization Time and resource sharing on expensive mainframes IBM VM/370 Late 1970 s and early 1980 s: became
Attacking Intel Trusted Execution Technology Rafal Wojtczuk email@example.com Joanna Rutkowska firstname.lastname@example.org ---===[ Invisible Things Lab ]===--- Abstract In this paper we present
Virtual Machines Uses for Virtual Machines Virtual machine technology, often just called virtualization, makes one computer behave as several computers by sharing the resources of a single computer between
Hardware Based Virtualization Technologies Elsie Wahlig email@example.com Platform Software Architect Outline What is Virtualization? Evolution of Virtualization AMD Virtualization AMD s IO Virtualization
V i r t u a l m a c h i n e s a n d o p e r a t i n g s y s t e m s Virtual machines and operating systems Krzysztof Lichota firstname.lastname@example.org A g e n d a Virtual machines and operating systems interactions
Windows Server Virtualization & The Windows Hypervisor Brandon Baker Lead Security Engineer Windows Kernel Team Microsoft Corporation Agenda - Windows Server Virtualization (WSV) Why a hypervisor? Quick
Attacking Hypervisors via Firmware and Hardware Alex Matrosov (@matrosov), Mikhail Gorobets, Oleksandr Bazhaniuk (@ABazhaniuk), Andrew Furtak, Yuriy Bulygin (@c7zero) Advanced Threat Research Agenda Hypervisor
Attacking Hypervisors via Firmware and Hardware Mikhail Gorobets, Oleksandr Bazhaniuk, Alex Matrosov, Andrew Furtak, Yuriy Bulygin Advanced Threat Research Agenda Hypervisor based isolation Firmware rootkit
AMD 64 Virtualization AMD India Developer s Conference Bangalore, David O BrienO Senior Systems Software Engineer Advanced Micro Devices, Inc. Virtual Machine Approaches Carve a System into Many Virtual
s COMP 3361: Operating Systems I Winter 2015 http://www.cs.du.edu/3361 1 Virtualization! Create illusion of multiple machines on the same physical hardware! Single computer hosts multiple virtual machines
Virtualization Technology A Manifold Arms Race Michael H. Warfield Senior Researcher and Analyst email@example.com 2008 IBM Corporation Food for Thought Is Virtual Reality an oxymoron or is it the
x86 Virtualization Hardware Support Pla$orm Virtualiza.on Hide the physical characteris.cs of computer resources from the applica.ons Not a new idea: IBM s CP- 40 1967, CP/CMS, VM Full Virtualiza.on Simulate
Virtualization Virtualization: extend or replace an existing interface to mimic the behavior of another system. Introduced in 1970s: run legacy software on newer mainframe hardware Handle platform diversity
Securing Your Cloud with Xen Project s Advanced Security Features Russell Pavlicek, Xen Project Evangelist CloudOpen North America 2013 Who is the Old, Fat Geek Up Front? Xen Project Evangelist Employed
12 January 2010 Virtualization Technologies Alex Landau (firstname.lastname@example.org) IBM Haifa Research Lab What is virtualization? Virtualization is way to run multiple operating systems and user applications on
Virtualization System Security Bryan Williams, IBM X-Force Advanced Research Tom Cross, Manager, IBM X-Force Security Strategy 2009 IBM Corporation Overview Vulnerability disclosure analysis Vulnerability
Hypervisors and Virtual Machines Implementation Insights on the x86 Architecture DON REVELLE Don is a performance engineer and Linux systems/kernel programmer, specializing in high-volume UNIX, Web, virtualization,
Basics of Virtualisation Volker Büge Institut für Experimentelle Kernphysik Universität Karlsruhe Die Kooperation von The x86 Architecture Why do we need virtualisation? x86 based operating systems are
Hypervisors Credits: P. Chaganti Xen Virtualization A practical handbook D. Chisnall The definitive guide to Xen Hypervisor G. Kesden Lect. 25 CS 15-440 G. Heiser UNSW/NICTA/OKL Virtualization is a technique
Contents Introduction...1 Overview of x86 Virtualization...2 CPU Virtualization...3 The Challenges of x86 Hardware Virtualization...3 Technique 1 - Full Virtualization using Binary Translation...4 Technique
Virtualization Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/ What is Virtualization? Virtualization is the simulation of the software and/ or hardware upon which other software runs. This
Hybrid Virtualization The Next Generation of XenLinux Jun Nakajima Principal Engineer Intel Open Source Technology Center Legal Disclaimer INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL
COS 318: Operating Systems Virtual Machine Monitors Kai Li and Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Introduction u Have
Technical White Paper LINUX OPERATING SYSTEMS www.novell.com SUSE Linux Enterprise 10 SP2: Virtualization Technology Support Content and modifications. The contents of this document are not part of the
Virtualization Jukka K. Nurminen 23.9.2015 Virtualization Virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms,
Microkernels, virtualization, exokernels Tutorial 1 CSC469 Monolithic kernel vs Microkernel Monolithic OS kernel Application VFS System call User mode What was the main idea? What were the problems? IPC,
Implementation of a Purely Hardware-assisted VMM for x86 Architecture Saidalavi Kalady, Dileep P G, Krishanu Sikdar, Sreejith B S, Vinaya Surya, Ezudheen P Abstract Virtualization is a technique for efficient
1 / 16 Virtualization P. A. Wilsey The text highlighted in green in these slides contain external hyperlinks. 2 / 16 Conventional System Viewed as Layers This illustration is a common presentation of the
Survey On Hypervisors Naveed Alam School Of Informatics and Computing Indiana University Bloomington email@example.com ABSTRACT Virtual machines are increasing in popularity and are being widely adopted.
Cloud Computing CS 15-319 Virtualization Case Studies : Xen and VMware Lecture 20 Majd F. Sakr, Mohammad Hammoud and Suhail Rehman 1 Today Last session Resource Virtualization Today s session Virtualization
Enterprise-Class Virtualization with Open Source Technologies Alex Vasilevsky CTO & Founder Virtual Iron Software June 14, 2006 Virtualization Overview Traditional x86 Architecture Each server runs single
Virtualizing a Virtual Machine Azeem Jiva Shrinivas Joshi AMD Java Labs TS-5227 Learn best practices for deploying Java EE applications in virtualized environment 2008 JavaOne SM Conference java.com.sun/javaone
Cloud^H^H^H^H^H Virtualization Technology Andrew Jones (firstname.lastname@example.org) May 2011 Outline Promise to not use the word Cloud again...but still give a couple use cases for Virtualization Emulation it's
VMware Server 2.0 Essentials Virtualization Deployment and Management . This PDF is provided for personal use only. Unauthorized use, reproduction and/or distribution strictly prohibited. All rights reserved.
Nested Virtualization State of the art and future directions Bandan Das Yang Z Zhang Jan Kiszka 2 Outline Introduction Changes and Missing Features for AMD Changes and Missing Features for Intel Working
Operating System and Hypervisor Support for IOMMUs Muli Ben-Yehuda IBM Haifa Research Lab email@example.com p. 1/3 Table of Contents The what and why of IOMMUs. How much does it cost? What can we do about
Starting Point: A Physical Machine Virtualization Based on materials from: Introduction to Virtual Machines by Carl Waldspurger Understanding Intel Virtualization Technology (VT) by N. B. Sahgal and D.
IBM T. J. Watson Research Center Trusted Virtual Datacenter Radically simplified security management Stefan Berger, Ramón Cáceres, Dimitrios Pendarakis, Reiner Sailer, Ray Valdez Secure Systems Department,
kvm: Kernel-based Virtual Machine for Linux 1 Company Overview Founded 2005 A Delaware corporation Locations US Office Santa Clara, CA R&D - Netanya/Poleg Funding Expertise in enterprise infrastructure
Qubes OS Architecture Version 0.3 January 2010 Joanna Rutkowska Invisible Things Lab firstname.lastname@example.org Rafal Wojtczuk Invisible Things Lab email@example.com This pre-release version
Software and hardware support for Network Virtualization Knut Omang Ifi/Oracle 19 Oct, 2015 Motivation Goal: Introduction to challenges in providing fast networking to virtual machines Prerequisites: What
A Hypervisor IPS based on Hardware assisted Virtualization Technology 1. Introduction Junichi Murakami (firstname.lastname@example.org) Fourteenforty Research Institute, Inc. Recently malware has become more
Virtualization Based on materials from: Introduction to Virtual Machines by Carl Waldspurger Understanding Intel Virtualization Technology (VT) by N. B. Sahgal and D. Rodgers Intel Virtualization Technology
KVM KERNEL BASED VIRTUAL MACHINE BACKGROUND Virtualization has begun to transform the way that enterprises are deploying and managing their infrastructure, providing the foundation for a truly agile enterprise,
RPM Brotherhood: KVM VIRTUALIZATION TECHNOLOGY Syamsul Anuar Abd Nasir Fedora Ambassador Malaysia 1 ABOUT ME Technical Consultant for Warix Technologies - www.warix.my Warix is a Red Hat partner Offers
Performance tuning Xen Roger Pau Monné email@example.com Madrid 8th of November, 2013 Xen Architecture Control Domain NetBSD or Linux device model (qemu) Hardware Drivers toolstack netback blkback Paravirtualized
Nested Virtualization Introduction and improvements Bandan Das Karen Noel 2 Outline Introduction When things don't work Note on AMD Speeding up Wrap-up References 3 Introduction Nested Virtualization Linux
Virtualization Pradipta De firstname.lastname@example.org Today s Topic Virtualization Basics System Virtualization Techniques CSE506: Ext Filesystem 2 Virtualization? A virtual machine (VM) is an emulation
Hypervisor Software and Virtual Machines Learning Objectives Understand the common features of today s desktop virtualization products Select and implement a desktop virtualization option on a Linux, Mac,
Server Virtualization and Consolidation An Ideal cost effective solution to maximize your Return on Investment of your organization's hardware infrastructure It is quit evident today that Business owners,
Hypervisor-Based Systems for Malware Detection and Prevention Yoshihiro Oyama ( 大 山 恵 弘 ) The University of Electro-Communications ( 電 気 通 信 大 学 ), Tokyo, Japan This Talk I introduce two hypervisor-based
Disaster Recovery Infrastructure An Ideal cost effective solution to protect your organizations critical IT infrastructure with business continuity. Organization's IT infrastructure usually evolves with
Windows Server Virtualization & The Windows Hypervisor Background and Architecture Reference Brandon Baker Lead Security Engineer Windows Kernel Team Microsoft Corporation Agenda - Windows Server Virtualization
atsec information security corporation 9130 Jollyville Road, Suite 260 Austin, TX 78759 Tel: 512-349-7525 Fax: 512-349-7933 www.atsec.com KVM Security Comparison a t s e c i n f o r m a t i o n s e c u
Virtualization P. A. Wilsey The text highlighted in green in these slides contain external hyperlinks. 1 / 16 Conventional System Viewed as Layers This illustration is a common presentation of the application/operating
Hardware Virtualization Rootkits Dino A. Dai Zovi Agenda Introductions Virtualization (Software and Hardware) Intel VT-x (aka Vanderpool ) VM Rootkits Implementing a VT-x based Rootkit Detecting Hardware-VM
CSE 501 Monday, September 09, 2013 Kevin Cleary email@example.com What is Virtualization? Practical Uses What can be virtualized Popular virtualization products Demo Question, answer, discussion Can
Virtualization Technologies and Blackboard: The Future of Blackboard Software on Multi-Core Technologies Kurt Klemperer, Principal System Performance Engineer firstname.lastname@example.org Agenda Session Length:
Subverting the Xen hypervisor Rafał Wojtczuk Invisible Things Lab Black Hat USA 2008, August 7th, Las Vegas, NV Xen 0wning Trilogy Part One Known virtulizationbased rootkits Bluepill and Vitriol They install
The Sandbox Roulette: are you ready to gamble? Rafal Wojtczuk email@example.com Rahul Kashyap firstname.lastname@example.org What is a sandbox? Environment designed to run untrusted (or exploitable) code, in a manner
Poacher turned gamekeeper: Lessons learned from eight years of breaking hypervisors Rafal Wojtczuk email@example.com 27 Jul 2014 Summary Hypervisors have become a key element of both cloud and client computing.
Basics in Energy Information (& Communication) Systems Virtualization / Virtual Machines Dr. Johann Pohany, Virtualization Virtualization deals with extending or replacing an existing interface so as to
Horst Görtz Institute for IT-Security, Chair for System Security VMRay GmbH Hypervisor-Based, Hardware-Assisted System Monitoring VB2013 October 2-4, 2013 Berlin Carsten Willems, Ralf Hund, Thorsten Holz
www.xensource.com Virtualization benefits Introduction to XenSource How Xen is changing virtualization The Xen hypervisor architecture Xen paravirtualization Interoperable virtualization The XenEnterprise*
Measuring and Modeling the Performance of the Xen VMM Jie Lu, Lev Makhlis, Jianjiun Chen BMC Software Inc. Waltham, MA 2451 Server virtualization technology provides an alternative for server consolidation
Windows 7 XP Mode for HP Business PCs Table of Contents: Introduction...1 Disclaimer...1 Main features and benefits... 2 Hardware Requirements...2 Minimum Hardware Requirements... 3 Recommended Hardware
Intel Virtualization Technology Overview Yu Ke SSG System Software Division Agenda Virtualization Overview Intel Virtualization Technology 2 What is Virtualization VM 0 VM 1 VM n Virtual Machines (VMs)
Anh Quach, Matthew Rajman, Bienvenido Rodriguez, Brian Rodriguez, Michael Roefs, Ahmed Shaikh Introduction History, Advantages, Common Uses OS-Level Virtualization Hypervisors Type 1 vs. type 2 hypervisors
WHITE PAPER Mainstreaming Server Virtualization: The Intel Approach Sponsored by: Intel John Humphreys June 2006 Tim Grieser IDC OPINION Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200
Introduction to Virtualization Dr. Qingni Shen Peking University Intel UPO Supported Main Points Status and trends in data center Definition of virtualization Common types of virtualization Key technologies
Detection of virtual machine monitor corruptions Benoît Morgan, Eric Alata, Vincent Nicomette LAAS-CNRS - Dependable Computing and Fault Tolerance (TSF) Team Journée SEC 2 - June 30th, 2015 Detection of
1 Lecture Embedded System Security Dynamic Root of Trust and Execution Prof. Dr.-Ing. Ahmad-Reza Sadeghi System Security Lab Technische Universität Darmstadt (CASED) Germany Summer Term 2014 Dynamic Root
Data Centers and Cloud Computing CS377 Guest Lecture Tian Guo 1 Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Case Study: Amazon EC2 2 Data Centers
SecureSwitch: BIOS-Assisted Isolation and Switch between Trusted and Untrusted Commodity OSes! Kun Sun, Jiang Wang, Fengwei Zhang, Angelos Stavrou! Center for Secure Information Systems! George Mason University!
OPEN SOURCE VIRTUALIZATION TRENDS SYAMSUL ANUAR ABD NASIR Warix Technologies / Fedora Community Malaysia WHAT I WILL BE TALKING ON? Introduction to Virtualization Full Virtualization, Para Virtualization
Virtualization Dr. Yingwu Zhu What is virtualization? Virtualization allows one computer to do the job of multiple computers. Virtual environments let one computer host multiple operating systems at the
BHyVe BSD Hypervisor Neel Natu Peter Grehan 1 Introduction BHyVe stands for BSD Hypervisor Pronounced like beehive Type 2 Hypervisor (aka hosted hypervisor) FreeBSD is the Host OS Availability NetApp is
Introduction to Virtualization & KVM By Zahra Moezkarimi ICT Research Institute Software Platform Laboratory Outline Virtualization History Overview Advantages and Limitations Types of virtualization Virtualization
Kernel Virtual Machine Shashank Rachamalla Indian Institute of Technology Dept. of Computer Science November 24, 2011 Abstract KVM(Kernel-based Virtual Machine) is a full virtualization solution for x86
ing Without a for a More Secure Cloud Xin Jin Princeton University Joint work with Eric Keller(UPenn) and Jennifer Rexford(Princeton) 1 Public Cloud Infrastructure Cloud providers offer computing resources
Xen and the Art of Virtualization Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauery, Ian Pratt, Andrew Warfield University of Cambridge Computer Laboratory, SOSP