Virtualization Detection: New Strategies and Their Effectiveness

Transcription

1 Virtualization Detection: New Strategies and Their Effectiveness Christopher Thompson Maria Huntley Chad Link ABSTRACT Previous methods for detecting execution within a virtual machine monitor have typically focused on specific anomalies of the implementation [4][14], have required running in kernel mode [5], or have been mitigated by newer versions of processor virtualization extensions [16]. We analyze a a basic non-privileged loop benchmarking test against MAVMM, a VMM designed for transparency [11]. We also implement and analyze the counter-based timing method discussed in [6][16], and a low-level cache interaction test using the sensitive unprivileged instruction CPUID. All of our implementations focus on detection from within the VMM, and without privileged instructions. Our results show that even virtual machine monitors specially designed to maximize their transparency and to prevent detection are susceptible to simple timing benchmarks. Additionally, we find that counter-based timing methods and low-level cache effects methods can distinguish execution in a virtual machine monitor from native execution. We also discuss how cache-flushing based methods are more complicated than previous work has suggested. Categories and Subject Descriptors C.0 [General]: Hardware/software interface; C.4 [Performance of systems]: Performance attributes; D.4.7 [Operating Systems]: Organization and design General Terms Algorithms, Performance, Experimentation, Security. Keywords Virtualization; Virtual Machine Monitor; x86; 1. INTRODUCTION A virtual machine monitor has three goals, as established by Popek and Goldberg in 1974 [12]: 1. Fidelity. Execution on the VMM must be identical to execution on native hardware, except for timing effects. 2. Performance. The VMM must intervene on only a small fraction of the guest's instructions. 3. Safety. All hardware resources are managed by the VMM. In order to maintain the safety of the system, a virtual machine monitor must trap and handle any sensitive instruction (ones that might affect the state of system resources). In the x86 architecture, there exist sensitive instructions that are not privileged [15]. Various methods have been developed to ensure that a VMM traps all sensitive instructions. Due to the existence of sensitive instructions, there can exist timing variations between native executions and executions within a VMM. Whatever method a VMM uses to ensure sensitive instructions are trapped, after it is trapped the VMM must expend additional time handling the instruction in a safe manner. In the x86 architecture, even with processor virtualization extensions, the shadow state must be acted on separately from true hardware state [1]. Thus the VMM code that handles the trapped instruction will add overhead, which can have effects on the actual state of the system. The most direct effect would be on the CPU time-stamp counter (TSC), which tracks processor clock cycles. Some VMMs account for overhead of the sensitive instructions by modifying the TSC by some skew value before returning control to the guest. TLB flushing can be prevented in the latest versions of the AMD-V and Intel-VT virtualization extensions [7]. Other effects on the system, as investigated in this paper, are possibly unpreventable barring extensive reworking of the x86 architecture. These timing and system state effects, as well as specific artifacts of individual implementations of VMMs make the detection of VMMs possible. There are many applications for detecting execution within a VMM. Malware that can detect a VMM can execute alternative code paths to avoid analysis. Some commercial software is packed with such code to avoid reverse engineering. And virtual machine based rootkits have been the topic of much research [9][16] anti-malware programs might one day detect if your machine has been migrated into a VMM without your knowledge. Previous methods for detecting execution within a virtual machine monitor have typically focused on specific artifacts of the implementation, such as hardware naming, guest-to-host communication systems, or memory addresses, among others [4][14]; have required running in kernel mode [5]; or have been mitigated by newer versions of processor virtualization extensions [16]. Our primary technical contributions are: (1) a simple break in the transparency of the MAVMM system using the TSC; (2) experimental analysis of previously discussed techniques and novel variations that have not previously been implemented and analyzed; and (3) the beginning of an analysis on the effects of a VMM on the x86 memory model. The rest of this paper is organized as follows. In Section 2 we test the timing effects in MAVMM, which claims to be transparent to detection methods. In Section 3 we look at counter-based timing methods and test and analyze such a method against various

2 VMMs. In Section 4 we look at cache-effects based methods, testing a new variation of the method against various VMMs. In Section 5 we look at related work. In Section 6 we present the conclusions we have gathered from our results. In Section 7 we discuss possible future work. All of our implementations (source and instructions) are available for download DETECTING TRANSPARENT VIRTUAL MACHINE MONITORS The research in VMM detection has played out as an arms race, not dissimilar to the problem of virus detection. Special purpose VMMs have been designed, and MAVMM [11] is an example of a VMM particularly designed to be transparent to detection. MAVMM was developed as a possible platform for malware analysis, specifically with strong introspection facilities and a focus on transparency over other factors. We wanted to verify the transparency of the MAVMM system. To do so, we created a simple timing benchmarking program. This consisted of timing using the rdtsc instruction, and a loop of instructions that only operated on registers. An excerpt of this loop is shown in Figure 1. rdtsc mov $121072, %edi loop: xorl %eax, %eax addl %ebx, %ebx movl %ecx, %ecx... sub $1, %edi jnz loop rdtsc Figure 1. Excerpt of the assembly code loop used in the simple timing benchmark. This is a variation on the trace cache loop in [Fuzzy Detection]. Because MAVMM is currently designed to run inside AMD's SimNow hardware simulator [2], we tested our detector on a Linux installation inside SimNow, and on a Linux installation inside MAVMM inside SimNow. The detector was run with loop lengths of 2000 to instructions. The results of our tests are shown in Figure 2. We saw that on average MAVMM took 436 more processor cycles to run the detector program than on baremetal. For most of the loop lengths the difference is noticeable at a 0.05 significance level. Thus we can see that even a VMM designed for transparency betrays significant timing discrepancies. 3. COUNTER-BASED TIMING The method of counter-based timing has been previously discussed, and proof-of-concept code mentioned [5][6][16], but no analysis of its effectiveness is currently available. Counter-based timing is of particular note because it is not dependent on any time source, inside or outside the target system. Because a VMM can skew results from internal timers (processor time-stamp counters, etc.), and network timing introduces fuzziness from latency and other factors, being able to performing timing without an actual time source is particularly interesting. Figure 2. A comparison of the CPU cycles used by the detector in MAVMM and on baremetal in SimNow. The means of 12 trials are shown along with 95% confidence intervals for those means. Counter-based timing races two threads running on a multi-core system (so that they are in roughly equal contention for processor cycles). One thread executes a continuous loop of NOP instructions. The other thread executes a continuous loop of CPUID instructions. Since CPUID is a sensitive unprivileged instruction [6], it must be trapped by the VMM (therefore incurring additional latency), but we can call it from user space. 3.1 Implementation We implemented a multi-threaded counter-based timing program in C using the POSIX threads library. Although this particular method requires a machine with multiple execution cores, there are other potentially sources of concurrency that could be used to create similar execution races. An additional thread was used with a signal timer to log the running counts in each of the looping threads at even intervals. This program let us compare how many CPUID instructions a core can execute over a time interval versus how many NOP instructions another core can execute over that same time interval, without needing to using explicit timing. If we were concerned about the accuracy of the signal timer in the status thread, we could instead run one of the looping threads until a predetermined count, and then stop both looping threads and compare their counts at that time. 3.2 Experimental Analysis We tested our counter-based timing implementation on baremetal, QEMU [13], VMWare Workstation [17], and KVM [10]. Each system had two processor cores, simulated or real, and was running Ubuntu Linux with a 2.6 series Linux kernel. For each system, we gathered three runs of the detector over at least 50,000 msec per run. Counts were logged every 500 msec. 1

3 KVM is notable as it displays a much greater disparity in the speed of a NOP to a CPUID than any of the other VMMs. In this case, the hardware VMM added its overhead to the already expensive CPUID instruction. Figure 4 shows that the means of the ratios for each system are statistically distinguishable to a 0.05 significance level. Thus, given a baseline understanding regarding how much longer a CPUID instruction should take to execute compared to a NOP instruction, this technique is immediately useful as a method for detecting virtualization. Figure 3. Comparison of NOP count to CPUID count over time for a baremetal system, QEMU, VMWare, and KVM. Figure 3 compares the ratio of the running count of NOP instructions executed to the running count of CPUID instructions executed over time for each system. VMWare Workstation, in full system virtualization mode using dynamic binary translation, showed a fairly small, but still detectable difference in its behavior from the tests on baremetal. Dynamic binary translation most likely allows VMWare to never have to trap for this instruction at all, substituting its own routine and modifying the registers accordingly instead. The results from the tests in QEMU show that a CPUID instruction takes a comparable amount of time to a NOP---a ratio of counts near As a full system emulator, QEMU most likely has a simplified implementation of CPUID and can immediately complete the instruction (without ever executing a CPUID on the host CPU), where on a native system the processor might take upward of 200 processor cycles to execute it. Figure 4. Comparison of the means and distribution of the ratio of NOPs executed to CPUIDs executed. 4. CACHE-EFFECTS When a VMM traps an instruction, a context-switch to VMM code must occur. On a typical machine, this means that registers, page tables, etc. of the process that trapped are saved, to be restored when it is context-switched back. However, certain parts of system state are not, or can not, be saved in this manner. Previous detection methods have taken advantage of the fact that the Translation Lookaside Buffer is one such part of system state [16]. On context-switch, the TLB would be flushed. By filling the TLB and forcing a context-switch to the VMM, a detector could compare timing information before and after to determine if such a flush occurred, and thus a VMM was present. Since then, both AMD-V and Intel-VT virtualization extensions have brought support for tagging entries in the TLB by process, and not explicitly flushing the entire TLB on context-switch [7]. Strategically filling the entire TLB and testing if any entries were overwritten could still be used to detect whether VMM code was executed, but it is much less effective than when the entire TLB was flushed. While the TLB can be controlled through fairly simple modifications to the virtualization extensions, lower level elements of the x86 memory model are much less controllable. These elements of the memory model are abstracted by the processor, and often vary greatly in their actual implementation and behavior [3]. For our experiment, we focused on the level 1 and level 2 processor instruction caches. While these caches are much more difficult to fill (due to typically being 8-way set associative and much more volatile), they are equally difficult for the VMM to control. 4.1 Implementation A simple Python script served as a wrapper of the compilation process. First, a C program was compiled that profiled how many CPU cycles one execution of the CPUID instruction takes on the target platform. Then, the cycle count is used to create two loops: one with a leading CPUID (the dirty loop) and one with a loop of NOP instructions to account for the overhead of the CPUID (the clean loop). Thus, the difference in the execution time between a CPUID and a NOP are mitigated for each platform. For example, on KVM, a typical CPUID instruction might take 2500 cycles, while on baremetal it might only take 250 cycles. The CPUID instruction [7] serves as both a serializing instruction and as our sensitive unprivileged instruction to cause a context-switch to the VMM (if present). Each loop has 10 instructions that only affect the CPU registers. This creates a total loop size that can readily fit inside the processor instruction caches.

4 While both loops will be occasionally context-switched by the process scheduler, we would expect the dirty loop to be switched significantly more often than the clean loop when inside a VMM, as it forces a context-switch every time it executes the CPUID instruction. 4.2 Experimental analysis We gathered 150 trials of the detector on each system. Each trial consisted of 1000 executions of the clean loop and 1000 executions of the dirty loop, which were then used to calculate an average number of processor cycles used for each loop in each trial. In each case, either only one virtual processor was made available in the VMM, or processor affinity was forced to increase the accuracy of readings of the CPU time-stamp counter. Figure X shows the results we gathered when testing our detector on a baremetal system and on KVM, QEMU, and VMWare Workstation. The distributions appear to be at least approximately normal around the mean. Figure 5. The distributions for each system of the ratio of cycles for the dirty loop to cycles for the clean loop. Baremetal shows a roughly even ratio, while the VMMs show significantly lower ratios. From our results we see that each of the VMMs show a much lower ratio than on baremetal. We actually that the dirty loop is taking less CPU cycles when in a VMM than the clean loop---opposite of our expected behavior. As shown in Section 3, QEMU appears to have a streamlined implementation of the CPUID instruction. However, we might have expected the initial profiling of the system to compensate for this. VMMs are able to offset the time-stamp counter in order to account for overhead. We may be seeing an over-correction for perceived overhead by the VMMs. 5. RELATED WORK Individual implementations of VMMs often introduce small bugs, or have features that are detectable from the guest. Peter Ferrie surveyed many such detectable anomalies, ranging from the location of the system Descriptor Tables to guest-to-host communication mechanisms [4]. Our work focuses on more general timing and system state introduced by the presence of a VMM instead of effects specific to any one implementation. Virtual machine-based rootkits (VMBRs) take advantage of the isolation of the host and guest provided by a virtual machine to hide rootkit code. Some systems, such as BluePill [16], can even migrate a live system into a malicious VMM. If VMBRs become prevalent in malware, anti-malware software may need to employ generic heuristic detection, such as the method we analyzed in this paper, alongside signature based detection, in order to detect such infections. In our work we focus only on a passive VMM. An active VMM or debugger modifies a running program to disable the code paths responsible for detection. Progress has been made on using dynamic analysis of a program to disable such detection routines [8]. Research has been done on using a remote detector machine to avoid the issues with local timing. A generic approach to this style of detection is analyzed in [5]. This work requires a remote machine, and code running in kernel space on the target. Our methods can detect a VMM from inside the guest, and require no privileged instructions. 6. CONCLUSION The timing effects that can be introduced by a VMM seem to be ever present. Even VMMs that have explicit goals of transparency (such as MAVMM) are still susceptible to leaking their presence through simple VMM overhead. We have shown the effectiveness of counter-based timing methods and cache-effects based detection methods. The differences in execution on native hardware is highly significantly different from execution inside a VMM for both methods. We have also demonstrated that the complexity and volatility of low level machine state can have a strong effect on the execution of programs inside a VMM. The inaccessibility of these low level parts of the x86 memory model will make it difficult for a VMM to prevent its effects. 7. FUTURE WORK There are many avenues we may pursue as further analysis of our methods. Extending our methods to use other sensitive, unprivileged instructions may yield additional insight into how overhead is created for different VMM actions. For the counter-based timing methods, there may be useful information in the trends of the ratios over time. In our results, KVM appears to have an upward trend, while the baremetal system appears to have a downward trend. More testing and analysis would be required to see if this is meaningful. For the cache-effects based methods, actively investigating the effects that can be produced on individual components of the low level memory model would be an interesting direction to pursue. There are some naïve assumptions about how these components act, but the cache architecture of x86 CPUs is becoming increasingly complex [3]. Also, are there any ways around these low level effects other than extremely specialized hardware? As virtualization extends to include standard desktop machines, we must also look into what this means for detecting virtualization. Nestable VMMs are currently available [16]. Are there any effects that are magnified by the number of VMMs being nested?

5 8. REFERENCES [1] Adams, K., and Agesen, O A comparison of software and hardware techniques for x86 virtualization. In Proceedings of the 12th international conference on Architectural support for programming languages and operating systems (2006), ACM, pp [2] AMD SimNow. Pages/default.aspx/. [3] Drepper, U What every programmer should know about memory. cpumemory.pdf. [4] Ferrie, P Attacks on more virtual machine emulators. Symantec Technology Exchange. [5] Franklin, J., Luk, M., McCune, J., Seshadri, A., Perrig, A., and Doorn, L Towards sound detection of virtual machines. Botnet Detection (2008), [6] Garfinkel, T., Adams, K., Warfield, A., and Franklin, J Compatibility is not transparency: VMM detection myths and realities. In Proceedings of the 11th USENIX workshop on Hot topics in operating systems (2007), USENIX Association, pp [7] Intel 64 and IA-32 Architectures Software Developer s Manual. [8] Kang, M., Yin, H., Hanna, S., McCamant, S., and Song, D Emulating emulation-resistant malware. In Proceedings of the 1st ACM workshop on Virtual machine security (2009), ACM, pp [9] King, S., Chen, P., Wang, Y., Verbowski, C., Wang, H., and Lorch, J SubVirt: Implementing malware with virtual machines. IEEE Symposium on Security and Privacy (May 2006). [10] KVM. [11] Nguyen, A., Schear, N., Jung, H., Godiyal, A., King, S., and Nguyen, H MAVMM: Lightweight and Purpose Built VMM for Malware Analysis In 2009 Annual Computer Security Applications Conference (2009), [12] Popek, G. and Goldberg, R Formal requirements for virtualizable third generation architectures. Communications of the ACM 17, 7 (1974), 421. [13] QEMU. [14] Quist, D. and Smith, V Further Down the VM Spiral. [15] Robin, J. and Irvine, C Analysis of the Intel Pentium s ability to support a secure virtual machine monitor. In Proceedings of the 9th conference on USENIX Security Symposium-Volume 9 (2000), USENIX Association, 10. [16] Rutkowska, J. and Tereshkin, A IsGameOver () Anyone. Black Hat, USA (2007). [17] VMWare Workstation. workstation/.