Keep It Simple - Common, Overlooked Performance Tuning Tips Paul Jackson Hotsos
Who Am I? Senior Consultant at Hotsos Oracle Ace Co-Author of Oracle Applications DBA Field Guide Co-Author of Oracle R12 Applications DBA Field Guide Vice President of Application Technology Stack SIG Blog: pjacksondba.blogspot.com Twitter: pjackson_dba
Keep It Simple Where are my glasses?
Agenda Database Configuration Patch Levels Database Performance Specifics Purge Options Server Monitoring New Features Continuing Education
Database Configuration Review Initialization Parameters Can be done anytime, especially as part of an upgrade. For EBS customers there are specific My Oracle Support notes : Database Initialization Parameters for Oracle Applications Release 12 (ID 396009.1) bde_chk_cbo.sql - EBS initialization parameters - Healthcheck (ID 174605.1)
Database Configuration Document non-standard settings Utilize the Comment feature to specify reason for setting the value. This column can later be queried from the table. ALTER SYSTEM SET Param_name= Value COMMENT= For SR 123456, set on 03-Mar-12 ;
Patch Levels Staying up to date on patches not only resolves security issues, but can also help to improve performance on the system. For more details see: http://blogs.oracle.com/stevenchan/2010/12/ebs_performanc e_patches.html
Patch Levels Database patches: Applying the latest CPUs or PSU are recommended. In addition, Oracle provides some notes for performance specific patches. Oracle Recommended Patches -- Oracle Database [ID 756671.1]
Patch Levels Application patches: Some additional Database and Application patches are recommended by Oracle. It is also advisable to stay current with the latest Technology Stack patches. Oracle E-Business Suite Recommended Performance Patches [ID 244040.1]
Database Performance Specifics Top SQL Top Segments Top wait events Memory advisors Miscellaneous
Database Performance Specifics Top SQL The Top SQL statements should be reviewed on a regular basis. The filtering criteria should use at least Logical Reads, Elapsed Time, and Executions.
Database Performance Specifics Top SQL If Database Diagnostic pack is not licensed, then query the data dictionary: select buffer_gets, elapsed_time, cpu_time, executions, sql_id, sql_text from (select buffer_gets, elapsed_time, cpu_time, executions, sql_id, sql_text, rank() over (order by buffer_gets desc --Change here for filter ) as rank from v$sqlstats ) where rank <=20;
Database Performance Specifics Top SQL If Database Diagnostic pack is licensed, then query the AWR repository: select b.sql_text, a.snap_id, a.instance_number, a.module, a.action, a.executions_delta, a.buffer_gets_delta, a.cpu_time_delta, a.elapsed_time_delta from ( select snap_id, sql_id, instance_number, module, action, executions_delta, buffer_gets_delta, cpu_time_delta, elapsed_time_delta, rank() over (order by buffer_gets_delta desc) as rank from dba_hist_sqlstat where buffer_gets_delta is not null ) a, dba_hist_sqltext b where a.sql_id=b.sql_id and a.rank <=20;
Database Performance Specifics Top SQL Review the top SQL for tuning opportunities. Identify source for the SQL statements. Using the Program/Module/Action columns from v$sql or awr_hist_sqlstat can help to identify the code. If the code is not well instrumented, then you still may be able to identify the source using the program id and program line in v$sql.
Database Performance Specifics Top SQL Using the following query will return information for a given SQL_ID select parsing_schema_name, service,module,action,program_id,program_line# from v$sql where sql_id=&sql_id; The program_id corresponds to object_id in dba_objects. This can identify what Function, Procedure or Package Body calls the specific SQL statement. The program_line# tells what line the call occurs.
Database Performance Specifics Top SQL If the query you are investigating uses bind variables, you can identify what variables were recently used by the following query: Select child_number, position, name, datatype_string, value_string from v$sql_bind_capture where sql_id=&sql_id order by child_number, position; With this information you can trace the query using the appropriate variable values.
Database Performance Specifics Top SQL Trace either specific statements or slow processes with 10046 level tracing. The trace file can then be analyzed with tkprof or with a more robust tool such as Hotsos Profiler. Analyze the trace file to determine where the system time is being spent. Recommended Method for Obtaining 10046 trace for Tuning [ID 376442.1]
Database Performance Specifics Top SQL Common issue, the Module AUTO_SPACE_ADVISOR_JOB is sometimes at the top of the list of queries sorted by Logical Reads. If the results of the Segment Advisor are not being reviewed, it can be unscheduled and run as needed.
Database Performance Specifics Top Segments Querying the top segments based upon criteria such as Logical Reads, Physical Reads, and Row Lock Waits can help identify busy objects. This can lead to investigations of required indexes, partitioning options, additional SQL tuning or other maintenance activity.
Database Performance Specifics Top Segments Sample query for 11g: with pivot_stats as ( select owner,object_name,statistic_name,value from v$segment_statistics ) select * from pivot_stats PIVOT (sum(value) for statistic_name in ('logical reads', 'physical reads','row lock waits' )) order by 5 desc;
Database Performance Specifics Top Wait Events This is key information to determine where the database is spending its time. The top events can be found in the AWR report or in Statspack reports and querying v$system_event. Common top wait events include CPU Time and DB File Sequential Read. Focusing on tuning the largest Wait Events will yield the most benefit.
Database Performance Specifics Top Wait Events One example was the appearance of enq HW Contention as the second highest wait event on a 10g version of Oracle. Researching this wait event on My Oracle Support returned information about bug 6376915. Additional example will be provided later
Database Performance Specifics Memory Advisors In Enterprise Manager Grid Control or in AWR reports, Oracle provides Memory Advisors that gives recommendations on sizing memory structures in Oracle. Before making any adjustments, ensure there is enough available memory on the server.
Database Performance Specifics Memory Advisors
Database Performance Specifics Miscellaneous Statistics Make sure statistics are up to date. Numerous presentations focus solely on this topic. The last_analyzed column in dba_tables will show when the table was last analyzed. For EBS customers be sure to use FND_STATS calls instead of DBMS_STATS.
Database Performance Specifics Miscellaneous Invalid Objects The number of invalid objects in your system should be zero. If not then the invalid objects should be known and explainable. select owner, object_name, object_type from dba_objects where status='invalid';
Database Performance Specifics Miscellaneous Run Away Sessions Check for long running sessions in the database. Depending on the processing being done, these can cause a lot of overhead. select count(1) from v$session_longops where username not in ('SYS','SYSTEM','DBSNMP') AND time_remaining > 30 AND elapsed_seconds > 30;
Database Performance Specifics Miscellaneous Number of sessions It s useful to track both the active and inactive number of sessions on your system. Some application problems can result in a larger than average number of inactive sessions. If the active number of sessions increases rapidly then it could be a sign that the database has become overloaded.
Database Performance Specifics Miscellaneous Number of sessions by status and total number of sessions. select count(1) from v$session where status='active'; select count(1) from v$session where status='inactive'; select count(1) from v$session;
Database Performance Specifics Miscellaneous Load Profile The AWR report contains a Load Profile near the top of the report. This provides a lot of key information related to number of transactions, number of logical and physical reads, etc for the snapshot period. Having this information available for comparisons with slow time periods will help to troubleshoot issues.
Database Performance Specifics Miscellaneous Load Profile
Purge Options Review largest objects in the database. The following query will show the 50 largest objects. Use OWNER column and segment_name to search for standard purge procedures. select owner, segment_name, segment_type, bytes from (select owner,segment_name, segment_type, bytes, rank() over (order by bytes desc ) as rank from dba_segments ) where rank <=50;
Purge Options Query for any backup tables. If backup tables exist, then they should be investigated to determine if they are still required. select to_char(num_rows,'999,999,999,999'), a.* from dba_tables a where owner not in ('SYS','SYSTEM') and num_rows is not null and table_name like '%BKP%' order by num_rows desc;
Purge Options Query for any materialized view logs. There are known issues where these logs can become excessively large. select * from dba_segments where segment_name like MLOG$_% order by bytes desc;
Purge Options Note, removing objects will not reclaim space without reorganizing the Tablespace. However this will reduce the need for future growth by providing free space within the Tablespace.
Server Monitoring Includes monitoring CPU utilization, Memory usage, and Disk I/O rates. This can be done using Operating System provided tools such as sar, using AWR, Enterprise Manager Host monitoring, or Oracle OS Watcher. OS Watcher User Guide [ID 301137.1]
Server Monitoring Monitoring CPU processing Don t want to exceed around 85% - 90% busy for extended periods of time. This helps to ensure enough CPU time exists for peak periods. Can monitor with vmstat command, CPU id column
Server Monitoring Monitoring Memory usage If the system runs out of active memory then the system will start to Page or Swap. This results in significant overhead on the CPU.
Server Monitoring Monitoring Disk I/O Rate This can result in performance problems across the board. Typically Av Rd(ms) should be less than 10. The following example shows a client whose performance deteriorated over the course of a week.
Server Monitoring Monitoring Disk I/O Example Day with good performance -
Server Monitoring Monitoring Disk I/O Example Day with poorer performance -
Server Monitoring Monitoring Disk I/O Example Day with worst performance -
Server Monitoring Monitoring Disk I/O Does Wait Events show this information too? Yes, compare Top 5 Events for all three days, 1 st day:
Server Monitoring
New Features Paritioning Oracle supports partitioning for EBS tables. This feature has a licensing cost associated. Although partitioning has been available for several releases, Oracle does release new options with this feature. http://blogs.oracle.com/stevenchan/entry/partitioning_and_pu rging_best
New Features Native compiled PL/SQL This feature has been available for several releases. The process to enable this feature is simpler in 11g. For programs that perform a lot of work in the database, native compiled code will run much faster. Time spent waiting for SQL to complete will not be affected.
New Features Advanced Compression This feature has been available since 11g. There is a cost associated with this feature, but it is worth reviewing if you have large amounts of data. It reduces storage requirements and improves performance of SELECT statements. http://blogs.oracle.com/stevenchan/entry/using_advanced_co mpression_with_e-business_suite
New Features Test, Test, Test! With any new feature perform ample testing to validate that functionality works as advertised. For features that have a cost, perform a Cost Benefit Analysis to determine if the feature is worth the cost.
Continuing Education Numerous blogs available. Good place to start: http://blogs.oracle.com/stevenchan/
Continuing Education Oracle provides webcasts available from My Oracle Support. Application Technology Group (ATG) Product Information Center (PIC) [ID 1160285.1] E-Business Suite Applications Technology Group (ATG) Advisor Webcasts [ID 1186338.1] Advisor Webcast Current Schedule [ID 740966.1]
Thank You! Questions? Blog: pjacksondba.blogspot.com Twitter: pjackson_dba Fill out and return surveys!!