ADF Performance Monitor Frank Houweling - ADF Spotlight 2015
Frank Houweling Principal Java / ADF / SOA consultant - AMIS (Netherlands) Focus on performance diagnosis and performance management Lead developer of the ADF Performance Monitor
3 Agenda ADF Performance Monitor Introduction Overview Examples
Instrumentation and Monitoring 4 Airplane pilots depend on instrumentation Crucial information to make decisions
Why Monitoring 5 Applications often contain frustrating blind spots and mysterious, recurring problems that are often difficult to identify, diagnose, and fix. With business-critical applications, it s very important to have a simple and fast way to monitor, diagnose, and resolve application problems before they affect revenue.
HTTP Request/Respons Time is Spent in Which Layer? 6 Request Response Database Application Server Network Client
End User Experience: Call and Ask all Customers? 7
ADF Performance Monitor 8 Advanced tool designed for measuring, analyzing, improving, and checking the performance and health of Oracle ADF applications Tracks and collects crucial production performance information of the application s runtime ADF components that are not standard provided by Oracle Helps development, QA, and operation teams to detect, analyze and resolve common and less common issues in response times, health and resource usage of ADF applications Can be used during development, test and production Is a library attached to your application Less than 1 day for implementation Needs to add a very small amount of code to application to make metrics classes active
Issues are reported in a Dashboard Reporting App and in JDeveloper 9
Month, Week, Day, Hour, 5 Minute Overview 10 Shows real-time critical information about the ADF application's performance Are response times within or outside SLA boundaries? Is immediate action required? What is the error rate? What is the health of the JVM?
Monitor SLA 11 from 15:00 to 16:00 the response times have increased significantly (more red and yellow colored parts in the 15u bar) This should be a trigger to investigate
Monitor Details of All HTTP Requests 12 Client Response Time (including network to client and Browser load time)
Monitor HTTP Request Details of Specific End-Users (1) 13 Select an end-user name in list Performance and problems of specific end-users can now be analyzed
Monitor Requests of Specific End-Users (2) 14 When and where did the end-user click? What were the response times of this end-user?
Request and Response Network Time 15 Request Response Database Application Server Network Client
Browser Load Time 16 Request Response Database Application Server Network Client
Example Slow Network/Browser Load Time (1) - Day Overview 17
Example Slow Network/Browser Load Time (2) Hour Overview 18
Example Slow Network/Browser Load Time (3) 5 Min Overview 19
Example Slow Network/Browser Load Time (4) Request Details 20
Example Network/Browser Load Time Resolved 21 Next day the network problem was resolved - We can see this in the following hour overview graph were there is much less purple colour (right bottom):
Zoom in Into WLS Managed Server (or Monitor All Together) 22 Select / Filter on Managed Server Compare Performance between Managed Servers
JVM Memory Problem Alerts 23
24 Quickly Troubleshoot Customer Complains All errors with all relevant detail information are recorded Error stack traces and call stacks are collected to allow the developer to quickly troubleshoot these errors This information helps to discover, analyze and resolve errors that happen frequently on test / production environments
Troubleshoot Production Errors (1) 25 Top 10 Production Errors / Faults by Occurrences
Trouble shoot Production Errors (2) 26 Solution: increase Maximum Capacity of datasource in WLS console
Trouble shoot Production Errors (3) 27
Very Easy to Find Root Cause of Problems 28 A call stack provides visibility into which ADF methods caused other methods to be executed, organized by the sequence of their execution. A complete breakdown of the ADF request processing, including all ADF method executions, along with elapsed times is printed, organized by lifecycle phase. Inefficiencies can be identified from this report, for example long running ViewObject queries, slow ApplicationModule activations & passivations, e.g.
Troubleshoot Slow ViewObject Queries 29 Analyse runtime SQL executed against the database
ViewObject Query Performance Aggregated 30 AVG, Occurrences and Total execution time
Slow Activation of ApplicationModule 31
Slow Passivation of ApplicationModule 32
Worst ApplicationModule activations & passivations overview 33 Troubleshoot ApplicationModule Pooling problems Set the most optimal ApplicationModule Pooling parameter settings
Worst ApplicationModule activations & passivations overview (2) 34
Occurrences overview 35
Loaded Rows in Memory (1) 36
37 ADF BC Memory Analyzer (2) Example: in this production ADF app, for a single ViewObject instance, more than 900.000 rows were loaded in the ADF app (Blue)!
Loaded Rows in Memory (3)- ADF CallStack 38 In this production ADF app, for a single ViewObject instance, more than 900.000 rows were loaded in the ADF app (!) Loading 900.000 rows takes more than 10 minutes
Warnings and Suggested Solutions 39
ADF Performance Monitor Warnings and Detailed Help 40 Fetching thousands of rows from database Slow passivation and activation of transient attributes Redundant ViewObject query executions Inefficient ViewObject fetchsize Inefficient PageDef iterator rangesize Slow executions of ViewObject queries EntityObject DML operations, database locks ApplicationModule activations / passivations / transactions PageDefinition operations / ApplicationModule custom methods PageDefinition iterators Webservices Java methods More
41 The tool can be dynamically turned on and off (at runtime) at all times The ADF Performance Monitor can be dynamically turned on and off (at runtime) at all times in the Enterprise Manager When the monitor is turned off there is no performance overhead because the metrics classes are not active
Use in JDeveloper during development 42
43 Configuration All configuration steps are well documented in the installation guide An ADF library needs to be added to your ADF application A very small amount of code needs to be added to instrument the ADF application. The metrics classes make use of extension points in the ADF framework in order to measure the time every action/step takes in a http request. Implementation takes less than 1 day, usually no more than a few hours
Overhead 44 Load tests show that the overhead is less than 3 to 4 percent. This overhead is caused by the very detailed collection of performance metrics. The metrics collection can be dynamically turned on and off (at runtime) at all times. When the monitor is turned off there is no performance overhead because the metrics classes are not active.
45 Support Standard Oracle stack ADF Business Components ADF Model ADF Controller ADF Faces JPA/EJB EclipseLink/TopLink POJO JavaBean DataControl JAX-WS Webservices It is possible to instrument (calls to) custom Java methods and 3rd party libraries Application servers: WebLogic Glassfish Tomcat
End-users will be more happy 46
Henri Peters - Project Manager Dutch Ministry of Justice 47 For us it is of great value that we can continuously identify the weakest link, even if there is no actual performance problem (yet)." We can even drill down to the level of an individual user. Sometimes we go so far that we proactively approach end-users: before they complain we already have contacted them. * More quotes of ADF experts and managers at: http://tinyurl.com/nxgkgr6 or http://www.amis.nl/adfperformancemonitor
48 Videos available http://tinyurl.com/npu732q or http://www.amis.nl/adfperformancemonitor Overview ADF Performance Monitor (13 min) Customer case Dutch Ministry of Justice (4 min)
Whitepaper 49 Whitepaper has detailed information about the architecture and implementation http://tinyurl.com/qgfh22u or http://www.amis.nl/adfperformancemonitor Content: Executive overview Introduction Oracle ADF applications and performance ADF Performance Monitor overview Use in JDeveloper Use in test and production environment Dashboard reporting application Summary and details HTTP response times ADF framework call stack Warnings and suggested solutions Worst performing executions in ADF BC and model layer Error stacktraces JVM performance Product architecture Configuration Turn on/off at all times Prerequisites Monitored Events
50 More information on http://www.amis.nl/adfperformancemonitor Contact us for more information and pricing. info@amis.nl or call +31 30 6016000