Instrumentation Software Profiling

Transcription

1 Instrumentation Software Profiling

2 Software Profiling Instrumentation of a program so that data related to runtime performance (e.g execution time, memory usage) is gathered for one or more pieces of the code during the program's execution Used to analyze and improve a program's performance

3 Software Profiling Typically questions answered using profiling What lines of code are responsible for the bulk of execution time? How many times is this looping construct executed? How many times a method is called and what % of total program run time it consumes? Which approach to coding a block of logic is more efficient?

4 Eclipse TPTP TPTP = Test and Performance Tools Platform ( Part of the open-source Eclipse project Project created in August 2004 Successor of the Eclipse Hyades tools sub-project (2002) Involves several participants: Compuware, IBM, Intel, OC Systems, SAP, Scapa Technologies Offers tool to create tools to automate software Quality Control Four major components / sub-projects: Platform Monitoring tools Testing tools Tracing and Profiling tools Most recent release: TPTP (Sept. 2009)

5 TPTP Objectives Bring software test and performance tools into the Eclipse environment By providing a platform and a set of components to develop testing, profiling and monitoring tools for applications. In a consistent way that maximizes integration with tools used in the other processes of the software lifecycle. Addresses the entire test and performance life cycle, from early testing to production application monitoring, including test editing and execution, monitoring, tracing and profiling, and log analysis capabilities

6 TPTP Objectives Reduce the cost and complexity of implementing effective automated software quality control processes Support a broad spectrum of computing systems including embedded, standalone, enterprise, and high-performance Share data through an OMG-defined test profile model, Common Base Event (CBE) Model and trace model implemented via the Eclipse Modeling Framework (EMF)

7 TPTP Project Structure TPTP Platform Project Core framework upon which the development of the monitoring, testing and tracing and profiling tools relies. Common user interface, standard data models, data collection and communications control, as well as remote execution environments TPTP Monitoring Tools Project Monitoring and logging phases of the application lifecycle, collecting and analyzing system and application resources. Correlate disparate logs from multiple points across an environment. Monitoring of application servers and system performance (e.g. CPU and memory utilization).

8 TPTP Project Structure TPTP Testing Tools Project Addresses testing phase of the application lifecycle. Contains testing editors, deployment and execution of tests, execution environments and associated execution history analysis and reporting. Includes exemplary tools for JUnit based component testing tool, Web application performance testing tool, and a manual testing tool. Future TTCN-3 support?

9 TPTP Project Structure TPTP Tracing and Profiling Tools Project Addresses the tracing and profiling phases of the application lifecycle. Includes exemplary profiling tools for both singlesystem and distributed Java applications through a JVMPI monitoring agent that collects trace and profile data. Includes a generic tool kit for probe insertion.

10 TPTP Architecture Presentation System Target System TPTP Workbench Monitor Trace Platform Test Controller Interface Distributed Control Framework Controller Interface Agent Controller Standard Widgets Application Reference Perspectives Trace Collection Statistical EMF Data Models Log Trace Test Data Loader Distributed Data Collection Framework Data Collection Interface Data Collection Log Collection Execution Environment JVMPI JVMTI Performance Monitor Eclipse Platform Test Execution

11 TPTP Architectural Components (1) On the controller system: User interface: the graphical view from which test execution is directed. Data models: based on the Eclipse Modeling Framework (EMF) Includes models for: test execution data logs traces statistics Eclipse platform: used for windows, editors, etc. Interfaces to the agent controller, and data collection engine.

12 TPTP Architectural Components (2) On the target system: the application to be tested the application s execution environment: includes the Java virtual machine and additional monitoring via virtual machine tool interfaces the test engine, to start the application and its environment data collection

13 Models implemented by TPTP (1) Includes a reference implementation of the UML2 test profile (MOF). Test profile: definition model for the creation of: test suites test cases data pools invocation loops synchronization

14 Models implemented by TPTP (2) Definition model for the definition, creation, management and persisting of test executions over time including: Tests to be executed. Deployments. Locations. Verdicts. Messages and console output from the test execution. A collection of test execution traces and results: the test log.

15 Agent Controller A daemon process that resides on each deployment host and enables client applications to launch host processes and interact with agents that coexist within host processes Contains a server that has the capability to launch and manage local or remote applications from a local TPTP workbench The Java profiler is a type of agent managed by AC that runs in the Java Virtual Machine process and receives notifications of JVM events. The Java profiling engine included uses the Java Virtual Machine Tool Interface to profile local or remote Java applications from a local TPTP workbench Launch and profile local or remote Java applications and import local or remote logs Platform support : Windows, Linux, Solaris, HP-UX, AIX, Z/OS, OS/400

16 Agent Controller Eclipse Workbench Agent Controller Java Profiling Agent Host process

17 Agent Controller Architecture Image 2006 IBM, made available under the EPL

18 Agent Controller Terminology Host process Agent Client The process that contains the application under test A reusable binary file that provides services to the host process, and more importantly, provides a portal by which application data can be forwarded to attached clients A local or remote application (e.g. Eclipse Workbench) that is the terminal destination of host process data that is externalized by an agent The Agent Controller A daemon process that resides on each deployment host and provides the mechanism by which client applications can either launch new host processes, or attach to agents that coexist within existing host processes. The client can reside on the same host as the Agent Controller, or it can be remote. The Agent Controller can only interact with processes on the same node, and it is required on the same machine the targeted JVM is

19 Virtual Machine (VM) Interface JVMTI, the Java Virtual Machine Tool Interface Provides interface for tools that monitor or control the virtual machine: debuggers, profilers, thread monitors, coverage analysis Capabilities: observation / control of memory threads stack frame memory heap local variables break points classes and object instances; methods timers

20 Versions of VM interfaces and the Agent Controller In versions of Java up to 1.4.2, there were two interfaces called JVMPI and JVMDI, where P = profiling and D = debugging These were deprecated in Java 5 and removed in Java 6. Both JVMPI and JVMDI have been replaced by JVMTI.

21 TPTP Java Profiling Tool Broadly useful for performance analysis and for gaining a deeper understanding of a Java program Consists of the Profiling and Logging Perspective and a number of graphical and tabular views Enables to profile and interact with applications, to work with profiling resources, and to examine applications for performance and memory usage problems Help to visualize and understand program execution, pinpoint the operations that take the most resources, as well as to explore patterns of program behavior Enables to test application's performance early in the programming development cycle for improvements

22 TPTP Java Profiling Tool A library that attaches to a JVM to capture and record the Java application's behavior Is a type of agent managed by the Agent Controller Runs in the JVM (Java Virtual Machine) process and receives notifications of JVM events, based on the JVMTI (Java Virtual Machine Tool Interface). Best used to identify performance details such as classes or methods responsible for the poor execution performance, also be used to analyze application heap and find memory leaks Output from the profiling agent is in the form of XML fragments (XML4Profiling) Can be launched from the TPTP workbench, Applications can be in workbench's workspace, binaries that are on the file system or in a server Can also be invoked using the -Xrun JVM option in command line

23 Profiling and Logging Perspective The profiling tools available in the Profiling and Logging perspective provide comprehensive information about the performance of an application The profiling tool (in this scenario i.e. Java Profiling) provides information pertaining to JVM performance Object allocations and references Garbage collection Object methods performance Object->Object interactions Thread interactions to name a few

24 Monitor and Navigator View Profiling Views

25 Monitor and Navigation View Java process associated with a host Time of profile run Monitors

26 Monitor and Navigation View Profiling and Logging perspective provides resources to administering and managing profiling Profiling resources Project: Make a project of your profiling effort Monitor: Aggregate different processes and agents Host: The host you are profiling Process: The executing program Agent : Provides services to a process, a mechanism by which process data can be sent to (attached) clients Profiling Type: Group profile data collection

27 Monitor and Navigation View Project: Monitor: Host: Container of the profiling resources A logical container for the profiling information that is collected from a group of agents. The views at the monitor level show data from these agents. Monitors are useful for aggregating processes and agents from a distributed application Owns the processes that are profiled. A host runs processes. You can specify a host either by its name or by its IP address Process & Agent: In the Agent Controller architecture model, an agent is a binary file that provides services to the host process by which application data can be transferred to attached clients

28 Profiling Views Number of views to visualize and organize profiling data Memory Statistic view Execution Statistic view Coverage Statistic view Object Reference view Execution Flow view and table Method Invocation view and table UML2 Trace Interactions view

29 Memory Statistics View Shows: Total instances: total number of instances of each class. Live instances: The number of instances that are alive (i.e. instances that were not collected by the garbage collector). Total Size (bytes): The size of an instances associated with a specific class Active Size (bytes): The size of an instances associated with a specific class

30 Execution Statistics View

31 Execution Statistics View Shows base time, average time, cumulative, and number of method calls: Base Time: The amount of time (in seconds) a method has taken to execute. Not including the execution time of any other methods called from this method. Average base time: The average base time required to execute a method once. Cumulative base time: The amount of time (in seconds) a method took to execute. Including the execution time of any other methods called from this method. Calls: The number of times a method was invoked.

32 UML2 Execution Flow

33 Thread Analysis

34 Thread Analysis

35 Probe kit Instrumentation Tool in the Test and Performance Tools Platform project Scriptable Byte Code Instrumentation (BCI) engine for Java instrumentation. Probes contain fragments of Java code to be called from classes and at times specified by the programmer including: Method entry Method exit Exception handling Class loading Probes can be used to collect runtime data, including: Method arguments and the this object Exception objects Return values

36 Probe kit Instrumentation Runtime data collected by probes can be used for : Profiling Send any numeric data for graphing: heap size, CPU load, anything-per-second rates Report when a method takes more than X amount of time to run. Debugging target program runs at nearly full speed even while evaluating conditions, unlike running under the debugger. Code analysis Send any kind of call-and-return, parent/child information

37 Create Probe Source

38 Fragment Types entry exit run upon method entry. Will not run for methods that were inserted into the class by Probekit. run upon method exit: a normal exit, when the method throws an exception, or when a thrown exception propagates out of the method. Will not run for methods that were inserted into the class by Probekit. catch run at the beginning of a catch clause in the method, or at the beginning of a finally clause that runs as the result of an exception.

39 Fragment Types staticinitializer run inside the class initializer of every probed class. If the class does not already have a static initializer, one will be created. executableunit run before every executable unit of code in methods that match the probe's target and filter specification, and for which source code is available. If the method does not have source line information, it will appear to have a single executable unit, numbered zero.. beforecall run in the calling method immediately before the target method is called. Not valid for method probes. aftercall run in the calling method immediately after the target method exits: either a normal exit, or when the target method throws an exception. Not valid for method probes.

40 Creating Probes

41 Apply Probe

42 Probe results