inspectit...because performance matters! The free Performance Management Solution

inspectit...because performance matters! The free Performance Management Solution

Free application performance management Happy Ending? After two years of developing an improved version of the internal business system based on the newest technology and integrating the gathered requirements of the employees, the development lead was happy to announce: The application was deployed successfully and starting this second we are live. We have done it, folks, the hard part is over. After the last few months of working long hours to keep the fixed deadline, the development team was relieved that the hard part seemed to be over. The next day the employees would use the new system and benefit from the added functionality and the improved throughput. The functionality had been tested thoroughly, so what could possibly go wrong? The next morning, after the first server crashed due to memory problems and a number of employees complained about the poor interface response times and the very long time it took for every single business transaction, the development team realized that the hard part in fact had just began. Until lunch, three additional servers broke down and the flood of complaints grew. Nobody could Performance and stability You can call yourself lucky if you never experienced a scenario like the one just mentioned, but it is well possible that you already have. Application performance and application stability is growing to become the project killer number one of today s software projects. The same as for functional problems applies to performance and stability issues: The earlier the problem is detected, the cheaper it will be to fix (with respect to time, complexity and money). Unsatisfactory performance often makes it necessary to revise and change the entire architecture - as a result, whole components need to be rewritten and optimized. Thus performance problems usually greatly outpace functional problems in their impact. What complicates matters is that the solution of performance problems usually affects many more components of the software than common functional defects. Developing and deploying software without adequate performance management processes can be a considerable risk: maybe it will work, maybe and more probably it will not. During the last few years the importance of application performance management is growing steadily, but many projects are still not implementing an adequate environment to detect performance problems early during development. They are still not creating an adequate load test environment and they are still not setting up tool support to gather information about their application during a load test or during production. With the effect, that if a performance problem arises, first the entire supporting environment needs to be set up, before it is possible explain the bad performance and stability problems of the new software. The pressure grew from hour to hour, management complained that the employees could not work as expected. Wild guesses were being issued by the stakeholders, each pointing their finger at other departments. After two days during which no work could be done on the system, management decided to switch back to the old system. Thorough performance analysis of the developed solution showed that the planned architecture was and would never be able to handle the number of requests issued by the users. The whole architecture would need to be revised and re-implemented. Most of the code would have to be adapted. Another year passed until the software was deployed again - this time thoroughly tested for stability and performance. Apart from several small migration issues, the system was running smoothly and the the employees were able to benefit from the new functionality and better performance. to find and fix the problem. Without a suitable performance setup, the entire performance test environment needs to be built if a problem arises before the problem can be analyzed. During our consulting assignments we met a number of developers desperately in need of tool support to analyze their components from a performance point of view without too much over- head. Often requests for performance tools or performance testing is denied by management due to the expected high costs. Managers and developers can now breathe a sigh of relief: NovaTec s inspectit will solve their problems for free. Safeguarding application performance is not expensive The stated reason why no performance safeguards are in place is often high costs. During project planning, performance management activities are often forgotten or postponed shortly before deployment on the production system - a few days prior to going live. Often performance tests are not executed at all at this point since the deadline is close. But safeguarding application performance in fact does not cost, but saves money in the long run. Application performance management is - like functional testing - a purely risk-driven activity. In fact the sole reason why functional tests are implemented and executed is that the costs (with respect to time, money and quality) are higher if these problems first occur during production by the customer. The budget of functional testing is directly derived from the risk and the impact of functional problems. The same holds for performance problems. It is naive to think that complex applications will somehow automatically provide high performance. By comparing the risks with the impact and taking the reduced risk of performance problems into account when integrating performance management, a budget for these activities can easily be granted. Performance management is thus not costly, but in fact presents a cheaper and safer approach as it helps reducing the risk of performance problems and thus the risk of even higher expenses (ranging to and including a premature termination of the project!). There is no one fits all performance management solution and not every project needs a high-standard performance management process. Often performance management is confused with performance testing. Performance management is a process starting with the project initialization, which continues during the complete development lifecycle and supportsthe whole software lifecycle. Performance testing is one step of performance management during which the actual performance tests are carried out. It is invaluable that during the definition of the project goals, performance goals are defined as well. The information gained (number of users on the system, distribution of the load, intents to increase the system size after deployment, detailed information of how long each important use case may take) will help greatly during defining the software architecture and during the implementation. Based on this information tests can be set up meaningfully, ensuring that the performance goals can be reached. For these verifications usually tools are necessary. Available application performance management tools During the last few years suitable costefficient alternatives to expensive load generation tools were provided. Projects that do not need the - no doubt unsurpassed - functionality of the market leaders can still use free, open source tools like JMeter or Grinder to power load tests. But generating load on the application is only the first step. Without the possibility to analyze why and where an application became slow as soon as 50 virtual users perform their predefined work, the root cause of the problem cannot be identified and no optimization can be started. Thus load testing without application monitoring does not give any insights into the application and should be considered as a necessary first step for further analysis. At this point, you know that you will not achieve your goal, but you can still only speculate about the why. We usually see guesswork at this point: It might be this database statement or worse, finger pointing between departments or teams: Our component is running just fine, but component X is always so slow. Integrating monitoring within the application will allow the developer to check why the application is getting slower, he is able to directly pinpoint the problem area and knows which part of the application needs to be reviewed and possibly optimized. Until now, cost-effective application monitoring solutions were not available on the market. Thus the project team had to decide whether to use one of the established tools of one of the market leaders or implement own monitoring logic within the code using log output. Unfortunately projects often went the third way and did not implement any performance safeguarding process at all. The performance testing tools that are currently available can be divided into the following categories: Inhouse-implementations and Professional performance solutions with excess functionality Page 2 Page 3

Why not integrate performance measurement tools into the application code? Important aspects of an application that should be monitored The short answer: Flexibility and maintainability. You will not be as flexible as you will need to be when trying to inspect your application with respect to performance. The more measurement tools you integrate into your software, the less throughput the software will provide. This is due to the fact that collecting the monitoring information also takes time. In fact, including too much monitoring can get you into a situation in which the application performance is so far reduced that the monitoring will not provide accurate performance data. Performance problems can occur anywhere within the entire software stack: In-house components, third-party libraries, the used application server, database queries, the operating system etc. Integrating time measurements only within your application code thus will only cover some of these possible areas. Isolating performance problems usually includes an iterative refining of the gathered monitoring information. The best solution is to first take a look at the functionality that is proven to be slower than expected and add measurement points to the components called within this functionality. If this information is not sufficient, refine the monitoring and include additional measurement points. As soon as the performance problem has been isolated and fixed, at least one monitoring point that easily allows checking for this problem again should be integrated. Then this configuration should be stored and all other measurement points be deleted. With manually integrated code to measure your application components, the time to define, refine and remove Illustration 1 Not recommended: application monitoring by means of logging monitoring is very time-consuming. Aside from that, the integration of non-business-related code always complicates the readability and thus maintainability. Different open source frameworks provide support to integrate monitoring directly with the code and allow basic visualization of the results. The most well-known open source framework is JaMON. We often see the approach to integrate monitoring and/or performance information with the standard logging process (based on tools like the famous log4j). In most cases this approach is chosen to be able to implement some quick and dirty monitoring facility to fix the problems at hand. We have seen multiple projects in which a small performance logging approach like this grew to become the common approach for the entire project. We strongly recommend not to integrate performance monitoring directly within the source code. It is inflexible and cannot provide all the information you need to find performance problems - in addition, usually only time monitoring can be performed. At the other end of the scale are professional solutions to performance management - the commercial tools, like for example CA Wily Introscope, AppDynamics or dynatrace. Nearly all tools provide the performance analyst with the necessary functionality to inspect and test an application with respect to performance. But all software solutions in this field also imply high costs. These tools provide unsurpassed functionality, are thoroughly tested and provide support for nearly all environments (even exotic ones). Still many projects just cannot afford to buy an expensive tool to monitor the application and pinpoint performance problems. Due to this NovaTec GmbH proudly announces the release of inspectit. inspectit is a Java monitoring tool which focuses on the core functionality necessary to cope with performance problems and provides transparent application monitoring for load test support. Using inspectit at least 80 % of all performance management requirements can be easily verified. We strongly need to differentiate between monitoring a productive application and trying to pinpoint performance problems. For monitoring an already productive application which does not impose severe performance problems, a minimal set of monitoring points should be defined, thus allowing for trend analysis. The focus of this method is to detect deviations from the usual behavior and fix them before the users of the application are affected by the problem. Less is more is the key. The optimal monitoring set contains as little information as possible to still provide all the necessary information to monitor the application and to find the problem at hand. For pinpointing existing performance problems or in-depth performance analysis, the amount of measurement points can be increased. For this scenario the tool should help to provide as much information as possible about the internal processes of the application and their performance data - still without causing too much overhead. Gathering all available information is also contra-productive as this will cause too much overhead and falsifes the gathered information too much to be of any help (this is the common problem of profiler tools). There is certain essential information that should always be gathered while testing an application for performance: Inspect the resources All applications need resources during runtime. Increasing the load of an application will most likely also increase the load on the resources (or increase the amount of necessary resources). As soon as the request for a specific resource exceeds its availability, the request will be put on Illustration 2 Monitoring of system resources (CPU and memory) with inspectit hold - thus decreasing performance of the application. Due to this close interdependency between resource and application performance, significant resources need to be monitored. As soon as monitoring shows that resources get scarce, adequate actions need to be taken. The most evident resources are system resources like the CPU and memory utilization or database resources like a database connection. Resources are also often pooled (e.g. in an application server), thus these pools need to be monitored as well. With inspectit the monitoring of system resources will be provided directly out of the box. The inspectit agent directly accesses the platform it is running on and gathers information about the resources of the operating system and about the Java virtual machine. Inspect the application With application monitoring you are able to open the blackbox of the Java virtual machine and collect information Illustration 3 Monitoring of execution times and invocation frequencies of a specific method Page 4 Page 5

about your application during runtime. As each application is different the amount and the locations to monitor differ as well. The most common approach is to monitor at least the business-critical interfaces of your components. Depending on your need for information more monitoring points can be inserted to gather more information about your application. With inspectit you can easily place a time-measuring point on any method of any class within your application (or even all third party libraries as long as they are also written in Java). This measuring point provides you with the information about the frequency of and the execution time of the respected method. Using this approach you can easily monitor the execution count of your components and stay informed about all execution times of your methods. By placing a measuring point on the entry method of an user transaction you can directly monitor the time your application needs to answer the user request. This information will - like all information within inspectit - be stored in a database for further analysis. Based on this information trends can be anticipated and actions can be taken. Inspect the application flow For in-depth analysis of use cases it is necessary to get a precise understanding what the application does and which steps within the application flow are slow. Unfortunately, the application flow is often rather complex and spans a huge tree structure. The flow path through this tree is often closely related to the input variables. This often leads to situations in which certain use-cases are infrequently slow or at least differ greatly in their performance depending on the input. For such situations the best analysis mechanism is to treat each invocation of the use case as a separate set of information. To do so, the tool must provide the unaggregated application flow for each call of the use case. inspectit implements this concept with its invocation sequence tracer. This functionality allows to define the starting point of a use case and will track each invocation of this method independently. This gives you the option of performing an analysis with a focus on user transactions. The measuring points within an invocation sequence do not represent average values, making them Illustration 5 Navigation from the database overview to the invocation sequence view An invocation sequence is created for every invocation of ordercart- ForCustomer perfectly suited for analysis. The fact that the invocation sequence shows you the specific flow through your application lets you see precisely and in detail at which point your application was slowed down by a specific action. Inspect the database queries Current estimates put unnecessary or inefficient database queries at one of the top-most positions of performance problems. This usually occurs as the database is abstracted by additional frameworks. The developers will often never see the actual query executed by the framework. Another frequent scenario is based on the fact that defining new database queries for each and can be easily analyzed in the detailed tree view. inspectit sensors know in what invocation sequence they were invoked. That s why inspectit can offer you extensive navigation options to and from invocation sequences for analyzing performance problems. You start for example in the database view that shows all SQL queries that have been run. In this view (average values) you find a database query, which is very slow (high maximum value) at times. Based on this query you can now easily navigate to the invocation sequence (that is, the specific user action) in which this database query was run. Illustration 4 Invocation sequences enable a transaction-based view of requests Illustration 6 Monitoring of database queries with invocation parameters Page 6 Page 7

persistence framework takes up time. Developers thus tend to re-use existing queries even if these queries return far too much information. But writing new queries results in implementing new code, ranging from new interfaces to new data transfer objects. inspectit allows to track all database queries that were created and executed within your application. By default the database tracing is activated out of the box with no manual configuration being necessary. The provided information allows to quickly inspect all queries with respect to their execution frequency and their respective execution time. Since all data is stored within a database, trends become visible. In addition inspectit allows to display SQL information directly within invocation sequences easily showing if a method was slow due to complex and inefficient algorithm or due to invoked database queries. Based on the specific structure of the SQL which also lets you see concrete bind values for prepared statements you can now analyze why this query presents a problem. You can further copy this query and send it to your database administrator together with all the required information. Inspect the architecture Another significant use of monitoring tools is to evaluate the adherence to architecture specification. Instrumenting the entry methods of the layers and components of the application with an invocation sensor allows to easily check which layers and components were used by each use case. This information is invaluable for architects to check whether the dependencies between applications are suitable and to understand which components are currently used to implement a use case. Inspect exceptions Exceptions that occur in your own program code or in a library in use are an important indicator of the status of your application. It is tedious and errorprone to compile the error codes from a variety of log files in order to analyze Illustration 7 Monitoring of exceptions problems with your application. That s why inspectit handles the tracking of exceptions for you. The exception sensor is activated the moment errors cases occur in the program code. In this way errors are made visible directly even if the application falsely ignores them or handles them incorrectly. Identifying these errors provides the basis for improving the application s stability. The exception sensor also shows you precisely which components of your application triggered the exception. It thus indicates whether the exception was for instance intercepted at a specific location or whether it was relayed further. Of course inspectit also provides you with the entire stack trace to let you analyze the errors in detail. For added convenience you can also navigate easily and quickly from the exceptions overview of the application to the corresponding invocation sequences. Inspect HTTP requests Many of today s business applications offer access to a web-based user interface via a browser. Knowing which action triggered a specific HTTP request at the back end is of key importance for analyzing a web-based application. For this reason inspectit offers a fully configured HTTP sensor that groups HTTP requests by URIs and depicts them in an URI-based aggregated view. This makes it easy to associate specific HTTP requests with the corresponding invoca tion sequences and to analyze them. Because the URI of an HTTP request cannot always be associated one-toone with a specific use case, inspectit offers the option of setting a specific use case s identification flag in the header of the HTTP request. This can for example be configured in a load testing tool such as JMeter to make it possible to associate specific HTTP requests with a use case. inspectit compiles these HTTP requests in a use-case-based aggregated view. Of course it is also possible to return from both the use-case and the URI view to the corresponding invocation sequences for further analysis. inspectit helps you solve your performance problems NovaTec incorporates years of experience in dealing with application performance ranging from the definition and integration of performance processes to safeguard the development of application to firefights in critical projects to improve performance in a short time. We are proud to offer tool independent performance consulting. To offer the best possible performance consulting we have developed a strong relationship with the two big players in today s application performance markets: dynatrace, AppDynamics and CA Wily Introscope. Our engineers provide years of experience in either of these tools and most of the other tools available on the market. Since 2007 NovaTec engineers of the business unit Application Performance Management have been implementing a solution to application performance monitoring and thus we are proud to officially announce the first official release of this application, inspectit. The development of inspectit includes performance monitoring experience gathered from many highly skilled NovaTec engineers to help you solve your performance problems. inspectit works by transparently integrating callback hooks into the application using load-time Java byte code modification. These hooks are then used by inspectit s sensors to collect various monitoring information about the application. This monitoring approach is integrated with a minimum Illustration 8 Grouping of HTTP requests into a use case Page 8 Page 9

of (transparent) code adaptation and thus is the safest possible way to integrate monitoring. All measurement logic is extracted to the sensors that run outside the business relevant logic code and in this way is safely separated from your application. The flexible and solid core framework of inspectit allows easy integration of additional sensors. Hence new sensors can be created to collect the information you need for your application. An inspectit installation consists of at least one agent running in the application JVM and monitoring the application(s), one (or quite rarely multiple) central measurement repositories (CMR) that store the data sent by the agents to a persistent storage and multiple clients that are connected to the CMR and display gathered and aggregated data in a significant way. The inspectit agent is the component that enables measuring points within the monitored application. It must be integrated into the startup script of your application (or application server) and automatically instruments the configured methods with the defined sensors. All configuration settings are defined within a configuration file residing in the agent configuration folder. The integration of the agent is as simple as adding one additional JVM option to the startup script of your application. Setting up the CMR and the GUI is even simpler - just copy the releases to the preferred location and execute the startup scripts. Interested? Contact us at info@inspectit.eu or call us at +49 (711) 22040 700. You may also use the contact information provided below. M.Sc. Stefan Siegl Business Unit Manager Application Performance Management Project lead inspectit stefan.siegl@novatec-gmbh.de M.Sc. Patrice Bouillet Lead architect of inspectit patrice.bouillet@novatec-gmbh.de User Interface Server CMR App 1 Factsheet inspectit App 2 Free performance diagnostics User Interface App 3 Performance analysis doesn t have to be expensive. inspectit gives you everything you need to carry out performance diagnostics and monitoring on your applications? and what s more, it s free. Your Java application need no longer be a mysterious black box. Use inspectit to unlock what s under the lid and give you the upper hand in the struggle for application performance, right from the development phase. With inspectit you can see what s going on inside your application, allowing you to identify and fix performance issues. Win the battle for performance by integrating transparent inspectit sensors, with no need to modify your program code. Integrating measurement points is as simple as adding a single line in a configuration file. Save time, effort and money by identifying potential performance bottlenecks before delivering the application to the client. When it comes to identifying and fixing performance issues, sooner is definitely better. Enjoy improved application performance thanks to inspectit Illustration 9 inspectit components for application monitoring inspectit Agent Features Fast and simple installation and configuration Start implementing your applications without complex and time-consuming tool setup. inspectit can be set up in a matter of an hour. Afterwards you can start defining your measurement points and start the inspection. Page 10 Page 11

Seamless integration inspectit integrates itself seamlessly in any Java application by means of byte code instrumentation. During classloading the measuring code is added, so your business code stays exactly the same, no business logic is altered in any way. State-of-the-art technologies and use of official standards inspectit is written completely in Java and only uses officially released and standardized interfaces. All components of inspectit are based on well-known and standardized libraries like Spring, Eclipse RCP, PicoAgent,... Decentralized architecture The flexible decentralized architecture allows having one (or multiple) central servers collecting, storing and aggregating the data of multiple agents and providing this data to multiple standalone clients for analysis. Exception Sensor The exception sensor is activated the moment error cases occur in the program code. In this way errors are made visible directly even if the application falsely ignores them or handles them incorrectly. Identifying these errors provides the basis for improving the application s stability. HTTP Sensor The HTTP sensor makes it easy to associate HTTP requests and use cases with invocation sequences. This allows a powerful form of analysis of web applications. Optimized for low overhead inspectit is designed to run with as little overhead as possible even in complex clustered environments. Monitoring Functionality inspectit can be used to monitor your live application and provide invaluable information about the current status of your application. Flexible architecture allows fast integration of sensors inspectit is based on an highly flexible agent architecture that allows seamless integration of sensors into the application code. A vast number of supported sensors for instant usage is available out-of-the-box. Additional sensors can easily be integrated. Currently inspectit provides the following sensors: Additional sensors can be integrated easily. Currently inspectit contains the following sensors: Time measurement (Timer sensor) Easily monitor the execution times of methods within your application to see which methods use up the most time. The sensor is also able to provide the pure CPU times which allows checking whether the method took long due to complex logic or if it was merely on hold. Flexible possibilities for sensor configuration inspectit incorporates a flexible configuration of sensors. Architecture Validation Architects can focus the instrumentation to gather architecture validation information, allowing them to verify if the developers use the architecture as the architect designed it to be used. Visualization A highly configurable state-of-the-art user interface allows easy analysis of the gathered data. Support for all Java applications on all platforms inspectit can monitor all Java applications (JVM version > 1.3) and works with all known application servers. The free Java monitoring solution inspectit provides all functionality necessary to implement application monitoring for Java applications but, unlike other other tools, is free. Platform Sensor The platform sensor allows monitoring of system resources like CPU utilization, memory utilization, threading etc. Database Sensor Performance problems are often related to inefficient or unnecessary database queries. inspectit allows to monitor the duration and structure of all database queries and provides the query as it is sent to the database even with all parameters flled in. Invocation Sequence Sensor Use invocation sequences to get unaggregated call invocation trees. Using this sensor you will directly get the information which methods were called by the users and which methods took long within this execution of the logic. This functionality is invaluable for inspecting invocations that are slow sporadically or in-depth analysis of certain functionality. Page 12 Page 13

Page 14 Page 15

inspectit www.inspectit.eu info@inspectit.eu 2009-2013 NovaTec - Consulting GmbH All other brands or product names mentioned here are trademarks or registered trademarks of the respective owners. Version 1.3 / März 2013