Privacy Protection in Virtualized Multi-tenant Cloud: Software and Hardware Approaches

Transcription

1 Privacy Protection in Virtualized Multi-tenant Cloud: Software and Hardware Approaches Haibo Chen Institute of Parallel and Distributed Systems Shanghai Jiao Tong University

2 A Review on a Keynote Speech by Justin Ratter in ISCA 2008

3 Integrated Approach to CA & OS?

4 This Talk Identify security issues with current cloud platform Describe two approaches to privacy protection of VMs Software approach: nested virtualization Hardware approach: secure processor A case of hybrid approach to systems research

5 Security Issues with Current Cloud Platform

6 Virtualization: enabling the cloud VM Image Control VM User VM User VM User VM User Auth. & Payment VMM

7 Can we simply believe in cloud? Is this bubble trustworthy?

8 Security Concerns in Cloud IDC Survey on Cloud 8

9 7 Threats of Cloud Security [Gartner 08] Privileged operator access Regulatory compliance Data location Data segregation Recovery Investigative support Long-term viability 9

10 Why we cannot trivially believe in multi-tenant cloud?

11 Reason#1: curious or malicious operators..., peeking in on s, chats and Google Talk call logs for several months

12 Reason#2: huge TCB for cloud KLOCs Xen Code Size Control VM Tools Kernel Guest VM VMM Trusted Compu-ng Base 0 VMM Dom0 Kernel Tools TCB The TCB is growing to 9 Million LOCs by 2011 One point of penetragon leads to full compromise 37 security issues are found in Xen and 53 in VMWare by Oct [CVE 12] The virtualiza-on stack should be untrusted

13 How Can You Break Virtualization Layer? Oper at or Management VM Kernel 2 Guest VM Guest VM Guest VM 1 at t ac ki ng s ur f ac e at t ac k VMM Hardware

14 Result: Limited Security Guarantees in Public Cloud Amazon AWS User Agreement, 2010 MicrosoB Windows Azure PlaJorm Privacy Statement, Mar 2011

15 Outline Overall Idea Software-based Protection CloudVisor: privacy protection of VMs in multi-tenant cloud with nested virtualization (SOSP 2011) Hardware-based Protection HyperCoffer: processor-rooted trust for guest VM protection (HPCA 2013)

16 A Design Principle Any problem in computer science can be solved with another level of indirection. David Wheeler in Butler Lampson s 1992 ACM Turing Award speech

17 Functionality of Virtualization- Layer Major functionality Resource management: manage memory, devices and CPU cores Multi-tenancy: multiplexing hardware resources to tenants, i.e., creating/running multiple VMs Cloud management: VM save, clone, migration Minor functionality Security protection (e.g., isolation, access control) Unfortunately, they are intertwined together in the same hypervisor-layer Forming a large trusted computing base (TCB)

18 Main Idea Minimize TCB (Trusted Computing Base) Privileged operator access is the top threat [Gartner 08] The hypervisor layer is getting more complex and vulnerable Add another layer of indirection Separate security protection from other main functionalities Software-based protection: nested virtualization Add one thin layer below the hypervisor: CloudVisor Hardware-based protection: reducing TCB to a secure processor Leverage a secure processor to do security and privacy protection 18

19 CloudVisor: Security Protection of Virtual Machines Using a Nested Hypervisor

20 Goal of CloudVisor Defend again curious or malicious cloud operators Ensure privacy and integrity of a tenant s VM Transparent with existing cloud infrastructure Little or no changes to virtualization stack (OS, VMM) Minimized TCB for cloud Easy to verify correctness (e.g., formal verification) Non-goals Side-channel attacks, exploiting a user-vm from network, Execution correctness of a VM

21 Observation and idea Key observation: protection logics for VMs are mostly fixed Idea: separate resource management from security protection CloudVisor: another layer of indirection Responsible for security protection of VMs (Unmodified) VMM VM multiplexing and management Result Minimized TCB VMM and CloudVisor separately designed and evolved

22 Architecture (logically) of CloudVisor

23 VM protection approach Bootstrap CPU states Memory Pages I/O data Intel TXT for late launch CloudVisor Hash of CloudVisor is stored in TPM Interpose control switches between VMM and VM (i.e., VMExit) Interpose address translagon from guest physical address to host physical address Transparent whole VM image encrypgon Decrypt/encrypt I/O data in CloudVisor See our paper for more details

24 Implementation Xen VMM Run unmodified Windows, Linux Virtual Machine 1 LOC change to Xen to late launch CloudVisor 100 LOCs patch to Xen to reduce VMExit (Optional) Run on SMP and support SMP VMs 5500 LOCs small TCB, might be suitable for formal verification

25 Performance Normalized Slowdown Compared to Xen % 0.2% 2.6% 1.9% 2.7% Xen CV 0 KBuild apache SPECjbb memcached Average Average slowdown 2.7%

26 Remaining Issues What if an adversary break into hardware?

27 Physical Threats can be Real Hardware Maintenance Thousands of machine failures per year [Schroeder et al, SIGMETRICS 09] in a datacenter Replacement of memory and disk has becomes daily routine Data Residual Memory bus sniffer Non-volatile memory Cold-boot attack [Halderman 09] Surveillance camera is NOT Enough! 27

28 HyperCoffer: Processor-Rooted Transparent Protection of VMs HyperCoffer

29 Goal: Minimalize TCB to Processor Trusted Untrusted Dom-0 VM-1 VM-2 Dom-0 VM-1 VM-2 Software Hypervisor Hypervisor Hardware CPU Memory Bus Sec-CPU Memory Bus Disk NIC Other Disk NIC Other Traditional System HyperCoffer Reduce TCB to only Secure Processor

30 Background: Secure Processor Previous Work on Secure Processor Data Privacy Data is encrypted outside CPU Data is decrypted only in cache Data Integrity Update hash tree at every write from cache to memory Check hash tree at every read from memory to cache Mainly used to protect application from Untrusted OS 30

31 Address-Independent Seed Encryption Address Data Cache Counter Cache Data (PlainText) Counter XOR PAD Encrypt VM- Key Secure Processor Data (CypherText) Counters 31

32 Secure Processor: Merkle Hash Tree Secure Processor Root Data (CypherText) Counters Hash 32

33 Secure Processor: Bonsai Merkle Tree Secure Processor Root Data (CypherText) Counters Hash 33

34 Prior Hardware Approaches Not really considers systems issues HyperWall (ASPLOS 12) requires an OS to specify which pages should be protected Ignore complex interactions between hypervisor and guest VM Not compatible with existing VM operations H-SVM (MICRO 11) Use microprogram to do memory isolation, no defense against hardware attacks Require OS to designate which memory is protected or not Most others focus on fine-grained protection of applications or app modules (e.g., SecureME, Bastion)

35 Challenges 1. Interaction between hypervisor and VMs Selectively expose fields of CPU context to the hypervisor Auxiliary info for instruction emulation (e.g. guest page table) I/O emulation for both disk and NIC 2. Backward compatibility with existing OS Minimize the cost of deployment 3. Supporting VM operations Not limited by data structure on the chip HyperCoffer Retains OS-transparency with VM-Shim

36 Design Overview VM-1 VM-2 App App Guest Mode OS OS Data Exchange Shim Mode Shim Shim Data Exchange Host Mode Hypervisor I/O Dev Memory Secure CPU 36

37 Design Overview 1. Complete Isolation VM-Table to support multiple VMs Tagged cache for different VMs Dedicated EPT memory 2. Controlled Interaction VM-Shim: Control Interposition Shim mode VM-Shim: Data Interaction 3 types of interactive data 37

38 STEP-1: COMPLETE ISOLATION 38

39 Leveraging Secure Processor Data Privacy: AISE All the data in VM is encrypted Different VM has different key VM-keys are saved in-chip Malicious hypervisor/hardware cannot read VM data Data Integrity: BMT BMT (Bonsai Merkel Tree) Root hash is saved in-chip Every memory read will check hash value Every memory write will update BMT 39

40 VM-Table VM-Table Contents Each running VM has one entry Each entry contains a K VM and vm_vector Kvm is used to encrypt VM, it is per-vm based vm_vector contains VM info for secure processor, used to verify a VM image E.g., the root hash of the BMT VM-Table is saved in a preserved memory region in encrypted form 40

41 BMT & AISE are NOT Enough 1. Inter-VM Remapping Attack Malicious hypervisor remaps VM-A s page to VM-B If the data of the page is in cache, then VM-B gets it VM- A EPT VM- B EPT CPU GPA Cache- line HPA Problem: data in cache is plaintext Memory Solution: tag cache-line with VMID 41

42 BMT & AISE are NOT Enough 2. Intra-VM Remapping Attack Malicious hypervisor remaps a VM s page-a to page-b If the data is in cache, then two pages are switched GPA A B A B HPA X Y X Y Solution: dedicated memory for EPT Flush cache when EPT is changed Optimization: lazy TLB flushing only at n-tlb missing 42

43 STEP-2: CONTROLLED INTERACTION 43

44 VM-Shim Mode VM-Shim A piece of code runs between hypervisor & VM Exchange data for the two VMExit VM Shim-Mode Shim-Mode can access VM s data VM cannot access Shim s data Shim 3 Hypervisor 44

45 VM-Shim: Data Interaction Specification of Interactive Data Describe the format of interactive data to store then Our implementation: use shim s memory Two New Instructions raw_st: store data without encryption raw_ld: load data without integrity check VM s Memory CPU Context Shim s Memory InteracGve Data Encrypted Plaintext 45

46 VM-Shim: Interactive Data Minimize Interactive Data Different for different VMEXITs Data Specification Communication protocol between hypervisor and shim CPU Context Disk I/O Network I/O Auxiliary Info Register value of guest VMs Meta- data of I/O operagons Both meta- data and I/O data E.g., page table entry, trapped instrucgon 46

47 Example: Trap & Emulate I/O Instruction: in %dx %eax Read from I/O port %dx and put value into %eax CPU VM %dx %eax Shim insn %dx %eax Shim s Memory Hypervisor I/O Dev 47

48 Example: DMA in Network I/O DMA in Network I/O Maintain states by interposition all I/O operations to device Use Shim s own memory as shadow buffer for DMA VM data VM s Memory Shim data Shim s Memory Hypervisor I/O Dev 48

49 On Emulator Based on Qemu Implementation On Real Machine Use hook of VMExit to implement VM-Shim Components User-level agent: 200 LOCs VM-Shim: 1100 LOCs Xen: 180 LOCs 49

50 Evaluation on Simulator Simulator Dinero-IV LLC: 8MB 8-way set-associative Counter Cache: 64KB and 8-way set-associative Cache using LRU replacement, 64-byte block Memory: 512 MB, with latency 350 cycles AES encryption: 80 cycles Virtualization Software Xen-4.0.1, domain-0 with Linux Virtual Machine One or more core, 1GB memory 20GB virtual disk, virtual NIC Unmodified Debian with kernel , x64 Windows XP SP2, x64 50

51 Evaluation on Simulator Normalized Slowdown with Xen (%) AISL+BMT AISL+BMT+Shim

52 Evaluation on Real-Machine Software Xen-4.0.1, domain-0 with Linux Hardware AMD quad-core CPU, 4GB memory 100Mb NIC, 320GB disk Virtual Machine One or more core, 1GB memory 20GB virtual disk, virtual NIC Unmodified Debian with kernel , x64 Windows XP SP2, x64 52

53 Evaluation on Real-Machine Performance Overhead over Xen (%) Single- core Qual- core kbuild dbench netperf memcached specjbb- xp 53

54 Summary Lack security guarantee in multi-tenant cloud A case on integrated approach to computer systems Two software/hardware systems to secure cloud CloudVisor: whole-vm protection with nested hypervisor HyperCoffer: hardware-rooted whole-system security See our papers for more detailed information

55 Thanks CloudVisor/Hypercoffer Ques]ons? One (small) ring to Rule them (cloud) all Institute of Parallel and Distributed Systems