Winning the J2EE Performance Game Presented to: JAVA User Group-Minnesota Michelle Pregler Ball Emerging Markets Account Executive Shahrukh Niazi Sr.System Consultant Java Solutions
Quest Background Agenda Core Principles of Performance Understanding Production Environment issues Quest APM Approach Quest APM Solutions overview Demo
Company - Quest Software Company Size: $945 Million -- Market Capitalization Past 3 year Revenues: 2003 $304M 19% growth from 2002 2002 $258M - 4.8% growth from 2001 2001 $245M - 48% growth from 2000 Number of APM Customers: 18,000+ Number of Support Team Members: 220 Year Founded: 1985 Corporate Headquarters: Irvine, CA Number of Employees: Over 2,400
Core Principles Understand your goals of Performance Improve response time Improve throughput Ensure scalability Understand your application How does user activity affect each tier? Are there any hotspots? If so, why? i.e Is the application database-intensive? Understand there is no Silver Bullet Be objective Removing one bottleneck will often uncover another
Core Principles Take a System Approach Every layer and subsystem matters Apps Server Tuning Apps Server Test each change individually Document for future analysis
Core Principles Use more powerful machines Vertical scaling Best for CPU-intensive apps Use clustering Horizontal scaling Best in Web and application tiers Increases reliability Use network appliances Specialized hardware devices can increase performance Data storage devices, content caching engines & routers, load balancing hardware, SSL terminators
Production Environment Complexities Hardware Components Software Components
Common Bottlenecks >Firewall Devices 20 SSL handshakes/sec High CPU utilization Heavy I/O (eg. when logging) >Load Balancers Connection imbalance Low throughput >Web Servers Memory: excessive paging High Disk I/O Unbalanced server connections >Application Servers Memory: JVM paging DB connection contention CPU > 80% Execute queue waiting >Database Servers Cache hit ratio causes high disk I/O Memory: excessive paging and high I/O wait time High parse time
End User Suffers / It Cost $$ Triage Network Admin Web Admin App Admin DBA
Quest s APM Application Suite for Performance J2EE Intro Management Suite for the J2EE platform An integrated solution that empowers all the stakeholders in J2EE application performance management to accelerate the detection, diagnosis and resolution of businessthreatening performance issues.
Key features of Quest s APM Solution Service Level management Improve staff efficiency by isolating performance problems based on service models and end user response time Application Server Diagnostics With very low overhead, quickly identify and resolve performance problems caused by resource bottlenecks and server configurations Deep Transactional Diagnostics Perform deep transaction-based diagnostics using Quest's Tag and Follow technology to identify renegade method or SQL causing performance problems Application code profiling Enable developers to quickly resolve the offending line of Java code necessary to improve end user response time
APM Suite: Breadth and Depth Application-level code assurance Certify applications before deployment 24x7 application performance management Find Memory Leaks Eliminate line-level bottlenecks Stop thread deadlocks Ensure performance and scalability under load Certify the application s SLAs Trending and capacity planning avoid future failures Real-time event management SLA reporting Problem detection, diagnosis & resolution
APM Suite: Breadth and Depth
How do we solve them? High Level Systemic View Deep Source-code View Foglight Spotlight Detect, Diagnose, Resolve PerformaSure JProbe TOAD QC Detect Resolve Diagnose Diagnose End-to-End system monitoring 24x7 Live monitoring & alerts Business level reporting Domain/Cell Overview Real-time server diagnostics Expert Advice Distributed system view Transaction execution path Java component diagnostics Low overhead Single VM analysis Memory leaks/inefficient algorithms Developer focused Line-of-code level diagnostics
Complete lifecycle coverage Application-level code assurance JProbe Design and architecture Code optimizations Code enhancements Certify applications before deployment PerformaSure Model application load behavior for performance and scalability Certify for SLA s Baselining / trend analysis Spotlight 24x7 application monitoring and management Foglight Real-time event management SLA measurement Problem management Our solution covers all phases of the custom application lifecycle Benefits: 1.Accelerated diagnosis and root-cause resolution for production problems 2.Application confidence from testing early, testing often to production management with an integrated toolset.
Quest APM -For every Stakeholder What code is behind the symptom? Is the application architecture a problem? Which SQL statements need tuning? Is the DB really the problem? What component is at fault? Who should fix the problem? Is the application available? Is the app server configured correctly?
Quest APM Architecture Quest Management Server
Spotlight for J2EE Spotlight diagnostics on UNIX, Linux, Windows, Oracle, SQL Server, UDB, and WebSphere all provide a dashboard view of the critical metrics for health. Here, Spotlight highlights a problem with our Oracle DB server, showing a bottleneck in the Buffer Cache area. Context-sensitive help and corrective advice is available in all the Spotlights.
Quickly Troubleshoot QA and Production Real-time diagnostics (Spotlight) Contextsensitive expert help suggests solutions End-to-end view includes in depth data on: Web Servers Application Servers Databases Windows Unix ERP, CRM
Foglight Customizable web dashboard may include all the following and more Availability Business Status System Usage Alert Viewing
Architecture Intelligent Agents On-Demand On-Demand Diagnosis Diagnosis Dual-mode Dual-mode agent agent will will send send info info to to both both Foglight Foglight and and PerformaSure PerformaSure Foglight Foglight can can start start PerformaSure PerformaSure session session recording recording based based on on an an Alert Alert What s What s collected collected PerformaSure PerformaSure captures captures application application server server metrics metrics as as well well as as J2EE J2EE component component metrics metrics PerformaSure PerformaSure will will chain chain J2EE J2EE component component calls calls into into transactions, transactions, even even when when the the transaction transaction itself itself spans spans multiple multiple application application servers servers J2EE J2EE components components considered considered by by PerformaSure PerformaSure are: are: JDBC, JDBC, JNDI, JNDI, JMS, JMS, URL, URL, EJB, EJB, servlet, servlet, and and JSP JSP calls calls into into the the application application code. code. Filters Filters can can be be set set for for any any type type of of request request J2EE App Server IBM Websphere v4, 5, 5.1 ++ BEA Weblogic v6.1, 7.0, 8.1 Tomcat v4.1, 5.x JBoss v3.2+ Oracle 9iAS v9.0.2+ Oracle 10G v9.0.4 Foglight Detection, event management JMX Method level alerting Can be run at the same time Foglight will trigger PerformaSure to start recording based on exceed threshold/event PerformaSure Diagnosis -> Resolution Tag-and-Follow instrumentation Low-overhead
Performasure Work Station Request Time Browser SQL Browser Request Tree Browser Metrics Browser Thresholds Browser Network Traffic Browser
Tag and Follow Technology Reconstruction of the true transaction execution path
Deeper resolution at code level PerformaSure PerformaSure can can readily readily export export information information around around the the problem problem method method for for further, further, granular granular profiling. profiling. Go Go to to a a deeper deeper (the (the application application code) code) level level with with JProbe. JProbe. Use Use Quest s Quest s SQL SQL Tuning Tuning component component to to tune tune poorly poorly written written SQL. SQL.
JProbe Architecture Real time Views JProbe Console Snapshots deep diagnostic data JProbe Analysis Engine JVMPI/BCI J2EE App Server Web Server Or Standalone Java App Java Virtual Machine RDBMS - For use in Development and Testing, One to One connection with JVM
Line-of-Code Resolution It s Easy Bottlenecks are automatically highlighted in red It s Fast Find memory leaks quickly with the most detailed object allocation information It s Flexible Line-of-code differencing Reporting in Excel, HTML or text
Detailed JDBC Information Summary of JDBC information collected: Offending SQL Statement or stored procedure Timing Breakdown and Number of Invocations to: Obtain connection Prepare SQL statement & pass parameters Execute statement Manipulate the returned result set Commit or Rollback the database transaction
Integration for DB Diagnostics SQL TUNING: SQLab Vision for Oracle SQL Tuning for DB2 Database Performance Diagnostics: Spotlight on Oracle Spotlight on DB2 Enough information to investigate through QCO!
Customer Results Using Quest Software products AutoDesk saves up to 80% of our time in investigation and diagnosing performance issues in our clustered WebLogic environment, which previously was done through manual log sifting and trial and error techniques. Senior Applications Manager, AutoDesk helped the team narrow down the bottlenecks within our Java code in days versus weeks. J2EE Architect, HSBC Within minutes, we were able to profile two different J2EE applications and get valuable results immediately. Manager of Technical Architecture, UICI
DEMO