Experimental Evaluation of Distributed Middleware with a Virtualized Java Environment Nuno A. Carvalho, João Bordalo, Filipe Campos and José Pereira HASLab / INESC TEC Universidade do Minho MW4SOC 11 December 12, 2011 2011 Nuno Alexandre Carvalho MINHA
Motivation Service oriented architectures span a wide range of application scenarios Geographically dispersed Deployed outside enterprise information systems Comprehensive evaluation requirements Correctness Performance 2
Motivation Current evaluation solutions Simulation models: useful while the whole system isn t available, but can only validate design and not the middleware and service implementation Actual deployment: most realistic but costly and time consuming, also requires the availability of the entire system 3
Overview Traditional experimental middleware evaluation JVM n... JVM 2 JVM 1 Platform libraries (java.*) Platform libraries (java.*) Platform libraries (java.*) Java Interpreter / JIT Compiler Java Interpreter / JIT Compiler Java Interpreter / JIT Compiler Network 4
Overview Traditional experimental middleware evaluation Multiple instances of an application are deployed in multiple JVMs JVMs are scattered across multiple physical hosts The amount of the required hardware resources is o&en prohibitive JVM n... JVM 2 JVM 1 Platform libraries (java.*) Java Interpreter / JIT Compiler Platform libraries (java.*) Platform libraries (java.*) Java Interpreter / JIT Compiler Java Interpreter / JIT Compiler Network 5
Overview MINHA middleware evaluation... Virtual JVM 1 JVM Virtual JVM n Virtual JVM 1 Bytecode instrumentation MINHA Platform libraries (java.*) Java Interpreter / JIT Compiler 6
Overview MINHA middleware evaluation Reproduces the same distributed run within a single JVM Application and middleware classes for each vjvm are automatically transformed Some simulation models are developed from scratch, others are produced by translating native libraries... JVM Virtual JVM 1 Virtual JVM 1 Bytecode instrumentation MINHA Platform libraries (java.*) Java Interpreter / JIT Compiler Virtual JVM n 7
Overview MINHA middleware evaluation advantages Global observation without interference Simulated components... Virtual JVM 1 Virtual JVM 1 Bytecode instrumentation JVM Virtual JVM n Large scale scenarios Automated What-If analysis MINHA Platform libraries (java.*) Java Interpreter / JIT Compiler 8
Agenda Simulation Kernel Virtualized JVM Input/Output Models Calibration Case Study 9
Simulation Kernel Event-based simulation kernel Abstract resource management primitives JVM... Virtual JVM n Virtual JVM 1 Virtual JVM 1 Simulated events and resources Platform libraries (java.*) Java Interpreter / JIT Compiler 10
Simulation Kernel Combination of real and simulated code: Measuring the time of execution and management of a simulated processor Allowing sequential Java code to execute by eliminating the inversion of control resultant from the event simulation JVM... Virtual JVM n Virtual JVM 1 Virtual JVM 1 Time virtualization Simulated events and resources Platform libraries (java.*) Java Interpreter / JIT Compiler 11
Simulation Kernel public class Foo { public static void main(...){ int i = 0; while (i<100) i++; } } 12
Simulation Kernel JVM class load public class Foo { start public static void main(...){ int i = 0; while (i<100) i++; } } 12
Simulation Kernel JVM class load public class Foo { start public static void main(...){ int i = 0; while (i<100) i++; } } For simplicity, let s assume that this segment is a thread 12
Simulation Kernel Thread JVM class load public class Foo { start() start public static void main(...){ int i = 0; while (i<100) i++; } } For simplicity, let s assume that this segment is a thread 12
Simulation Kernel Thread JVM class load public class Foo { event.run() start() start public static void main(...){ int i = 0; while (i<100) i++; } } For simplicity, let s assume that this segment is a thread 12
Simulation Thread Thread Simulation Kernel JVM st.pause() class load public class Foo { starttime() st.wakeup() event.run() start() start public static void main(...){ int i = 0; while (i<100) i++; } stoptime() st.pause() } For simplicity, let s assume that this segment is a thread 12
Simulation Thread Thread Simulation Kernel JVM st.pause() class load public class Foo { simulation time delta starttime() st.wakeup() event.run() start() start public static void main(...){ int i = 0; while (i<100) real time delta } i++; stoptime() st.pause() } For simplicity, let s assume that this segment is a thread 12
Virtualized JVM Reflect real time of execution of a sequence of code in the occupation of a simulated processor Blocking operations (thread synchronization and I/O) must be intercepted and translated into corresponding simulation primitives Code executing in different virtual instances cannot interfere directly through shared variables 13
Virtualized JVM Bytecode manipulation: custom class loader that uses ASM Java bytecode manipulation and analysis framework to rewrite classes Isolation: each virtual JVM has its own separate instance of the class loader acting like a sandbox Bytecode instrumentation Virtual JVM 1 Bytecode instrumentation... Translated platform libraries (moved.java.*) JVM Virtual JVM 2 Trans Byte Time virtualization Interaction: A subset of classes, containing the simulation kernel and models, are kept global providing a controlled channel for virtual JVMs to interact Simulated events and Platform libraries ( Java Interpreter / JIT 14
Virtualized JVM Bytecode manipulation: custom class loader that uses ASM Java bytecode manipulation and analysis framework to rewrite classes Isolation: each virtual JVM has its own separate instance of the class loader acting like a sandbox Bytecode instrumentation Virtual JVM 1 Bytecode instrumentation... Translated platform libraries (moved.java.*) JVM Virtual JVM 2 Trans Byte Time virtualization Interaction: A subset of classes, containing the simulation kernel and models, are kept global providing a controlled channel for virtual JVMs to interact Simulated events and Platform libraries ( Java Interpreter / JIT 14
Virtualized JVM Bytecode manipulation: custom class loader that uses ASM Java bytecode manipulation and analysis framework to rewrite classes Isolation: each virtual JVM has its own separate instance of the class loader acting like a sandbox Bytecode instrumentation Virtual JVM 1 Bytecode instrumentation... Translated platform libraries (moved.java.*) JVM Virtual JVM 2 Trans Byte Time virtualization Interaction: A subset of classes, containing the simulation kernel and models, are kept global providing a controlled channel for virtual JVMs to interact Simulated events and Platform libraries ( Java Interpreter / JIT 15
Virtualized JVM Bytecode manipulation: custom class loader that uses ASM Java bytecode manipulation and analysis framework to rewrite classes Isolation: each virtual JVM has its own separate instance of the class loader acting like a sandbox Bytecode instrumentation Virtual JVM 1 Bytecode instrumentation... Translated platform libraries (moved.java.*) JVM Virtual JVM 2 Trans Byte Time virtualization Interaction: A subset of classes, containing the simulation kernel and models, are kept global providing a controlled channel for virtual JVMs to interact Simulated events and Platform libraries ( Java Interpreter / JIT 15
Virtualized JVM... JVM Virtual JVM n Bytecode manipulation: custom class loader that uses ASM Java bytecode manipulation and analysis framework to rewrite classes Virtual JVM 1 Translated platform libraries (moved.java.*) Virtual JVM 2 Translated platform libraries (moved.java.*) Bytecode instrumentation Bytecode instrumentation Bytecode instrumentation Isolation: each virtual JVM has its own separate instance of the class loader acting like a sandbox Time virtualization Simulated events and resources Platform libraries (java.*) Simulation models (network,...) Java Interpreter / JIT Compiler Interaction: A subset of classes, containing the simulation kernel and models, are kept global providing a controlled channel for virtual JVMs to interact 16
Virtualized JVM Platform libraries Java prohibits the transformation of classes under java.* package Rewrite classes that contains native methods Overwrite special static methods, like System.nanoTime() The remaining classes are analyzed and processed automatically Translated platform libraries (moved.java.*) Bytecode instrumentation Virtual JVM 1 Time virtualization... Translated platform libraries (moved.java.*) JVM Virtual JVM 2 Trans Stubbed platform libraries Bytecode instrumentation (fake.java.*) Stub Byte Simulated events and Platform libraries ( Java Interpreter / JIT 17
Virtualized JVM Synchronization Primitives in java.util.concurrent.* Rewrite to fake.* Java monitor operations and implicit mutex/condition variables Inject a special fake.java.lang.object ancestor on all translated classes and rewrite monitor operations to invocations to methods on this class Translated platform libraries (moved.java.*) Bytecode instrumentation Virtual JVM 1 Time virtualization... Translated platform libraries (moved.java.*) JVM Virtual JVM 2 Trans Stubbed platform libraries Bytecode instrumentation (fake.java.*) Stub Byte Simulated events and Platform libraries ( Java Interpreter / JIT static synchronized methods are solved in a similar way with a singleton object 18
Input/Output Models Filesystem Reads and writes are intercepted in order to avoid direct invocation of native methods, thus providing separate filesystems to different virtual JVMs Network Modeled as a resource shared by all communication channels with a finite capacity Access control is performed by the leaky bucket algorithm TCP and UDP sockets, including Multicast, supported through the java.net API 19
Calibration Network Bandwidth Sending and receiving overheads Latency Performed by running two benchmarks: Flood Round-trip 20
Calibration Bandwidth with realistic behavior 1200 1200 1000 1000 BW (Mbit/s) 800 600 400 BW (Mbit/s) 800 600 400 200 0 Real Minha 0 1000 2000 3000 4000 5000 200 0 Real Minha 0 1000 2000 3000 4000 5000 Size (bytes) Size (bytes) a) Writing b) Reading 21
Calibration Latency with realistic behavior 0.7 Round Trip (ms) 0.6 0.5 0.4 0.3 0.2 0.1 0 Real Minha 0 1000 2000 3000 4000 5000 Size (bytes) c) Latency 22
Case Study Devices Profile for Web Services (DPWS) Standard that defines a set of protocols for devices to achieve seamless networking and interoperability through Web Services Proposed as the base for large scale smart grids and safety critical medical devices Used on recent operating systems, home automation, assembly lines and car industry Web Services for Devices (WS4D-JMEDS) Framework that implements DPWS standard Supports J2SE and J2ME 23
Case Study Membership notification Manager finds peers through multicast Manager sends producers addresses to peers Peers register themselves on producers Producers initiate notification rounds Number of peers go from 10 to 300 Manager Producer Rounds of events C C C C C C... C 24
Case Study In a normal WS4D deploy we would have Each peer on a different device Each device with only one CPU core 25
Case Study In a normal WS4D deploy we would have Each peer on a different device Each device with only one CPU core Due to hardware restrictions we deployed 300 devices on multiple JVMs on a single host with 24 CPU cores Localhost network with minimal latency Producer can send up to 24 notifications in parallel (biasing the results) 25
Case Study MINHA eliminates false latency when all peers run on a single host MINHA is faster than real deployments on I/O bound scenarios (up to 50 times) Latency (ms) 300 250 200 150 100 50 Multiple JVMs Minha Duration (s) 2000 1750 1500 1250 1000 750 500 250 Multiple JVMs Minha (sim) Minha (real) 0 0 50 100 150 200 250 300 0 0 50 100 150 200 250 300 Devices Devices a) Latency b) Duration 26
Conclusion Allows off-the-shelf code (bytecode) to run unchanged including threading, concurrency control and networking Manages a simulated timeline which is updated using accurate measurements of time spent executing real code fragments Provides simulation models of networking primitives and an automatic calibrator Allows off-the-shelf middleware stack evaluation deployed on a large scale system with hundred of devices 27
Conclusion http://gitorious.org/minha 28