Navigating the World of Performance

2011 IBM Power Systems Technical University October 10-14 Fontainebleau Miami Beach Miami, FL IBM Navigating the World of Performance Dawn May - dmmay@us.ibm.com Materials may not be reproduced in whole or in part without the prior written permission of IBM. 5.3

IBM i Performance Tools Suite r o t c o id her atc Job W alyzer n PEX A nalyzer A Heap PDV Gr ee PM for Po we r Pe n Sy rform Scr Ba stem anc een Pe tch M com e Too m rfo l rm odel and s Re an po s ce rts Ex pl o rer Sy st e ms WL E Ma na e m e g Perfo nt Sy Pe stem rf i tra n e C ce D n a rm Collection Services ata l PTDV DB To Na o / l S s Ac QL GU viga t tor Sy ive J Mon I ste ob i m S s tor tatu s Third Party Tools Job Watcher Disk Watcher Performance Explorer We We b Per b P form er f o a rma nce A nce dvi r o Mo sor t a g i nito t s r Inve 2

Where Do I Start When Managing Performance? 1. Collect System-wide Performance Data Collection Services 2. Automated Monitoring Your First Line of Defense System i Navigator System Monitors Guidelines for Setting Monitor Threshold Triggers IBM System Director Monitors 3. Monitor Historical Performance Trends System i Navigator Graph History PM for Power Systems IBM Systems Director 3

Where Do I Start When Analyzing Performance? 4. Basic Performance Analysis Performance Tools System i Navigator Plug-in Performance Data Investigator 5. Advanced Performance Analysis Job Watcher Disk Watcher Performance Explorer idoctor Suite 6. Miscellaneous Performance Tools WebSphere Performance Monitor / Advisor Database Performance Tools 4

Collect System-wide Performance Data 5

Collect System-wide Performance Data Collect Performance Data 24/7 If something goes wrong, you have data that will help analyze the problem, fix it, and prevent it from happening in the future If you can t solve the problem, you have information that makes it easier for IBM Support to solve the problem faster To provide a reliable baseline so you can understand the impact that a software, network, or environmental change had on the performance of your system To provide historical information that enables you to plan for future growth based on real trends, not guesses. 6

What is Collection Services? IBM i function that collects system and job level performance data Collects data from many system resources including: Jobs Disk Units Buses Pools Communication lines Collects data at regular intervals from 15 seconds to 1 hour Produces database files used by Performance Data Investigator Performance Tools for i PM for Power Systems Provides the data for System i Navigator System Monitors IBM Systems Director IBM i Monitors 7

Collection Services Data Categories System Bus Memory Pool and Pool Tuning Hardware Configuration Subsystem System CPU System level data Jobs MI tasks and threads Jobs Operating System Disk Storage IOP Local Response Time Communication (Base, Station, SAP) APPN ARM SNA SNADS Transaction TCP/IP Base TCP/IP Interface Integrated xseries Server (IxS) Extended Adaptive Cache User-defined Transactions Domino HTTP Server (Powered by Apache) Data Port Services LPAR WAS JVM *New in 6.1 Removable Storage *New in 7.1 External Storage *New in 7.1 System Internal Data *New in 7.1 Each category can be turned on/off and interval times can be set individually 8

IBM Systems Director Navigator for i Collection Services Configuration Collection Services runs in a job named QYPSPFRCOL in QSYSWRK. 9

Collection Services Properties Check this box if you plan to use: Performance Data Investigator Performance Tools IBM Systems Director Monitors They all require data in the database files Check this box if you plan to use the Performance Data Investigator. Performance summary data improves the performance of the initial queries when using PDI. 10

Collection Services Customized Data Collection Standard plus protocol is recommended. Use the Customize collection profile if you want to change the default time interval for individual categories. 11

Collection Services Customized Data Collection Select the category you want to change and add it to the customized categories. Then adjust the time. 12

Collection Services Data Retention The collection object has the original data. Standard data is the data in the DB2 files. You can always create the data in the DB2 files if you have the Collection Object. Recommend keeping at least 7 days of detailed data in case a problem occurs. 13

Creating database files with System Director Navigator 14

Creating database files with System Director Navigator Select the categories you want. Files will be created for the duration of the entire collection. Adjust the time if you want files created for a shorter duration. Increase this value if you want data created at a less frequent interval. 15

Collection Services System i Navigator All the prior customization can be done with the System i Navigator client as well. System i Navigator also has Graph History... Check bottom 2 boxes if you plan to use Graph History 16

Commands for Performance Data Collections STRPFRCOL - Start Performance Collection ENDPFRCOL - End Performance Collection CFGPFRCOL - Configure Performance Collection CHKPFRCOL - Check Performance Collection CVTPFRCOL Convert Performance Collection Added in 6.1: DLTPFRCOL Delete Performance Collection SAVPFRCOL Save Performance Collection RSTPFRCOL Restore Performance Collection 17

Monitoring 18

Automated Monitoring with System Monitors System Monitors gather and present real-time performance data Helps monitor the health of your system Helps identify potential performance problems before they become serious issues System Monitors provide multiple levels of performance information Level 1 System wide performance metrics such as CPU Utilization, Disk Utilization, etc. Level 2 A list of items that are contributing most to the Level 1 metric For CPU Utilization, it s a list of jobs that are consuming the most CPU For Disk Utilization, it s a list of disk arms that are the busiest Level 3 A list of performance metrics and properties for the Level 2 items Thresholds can be defined which will trigger an action when a system wide performance metric exceeds the defined comfort level For example, when CPU Utilization exceeds 80%, send a message to notify the operator System Monitors provide powerful capabilities to monitor what is happening on your system BUT finding out what caused the problem often requires other performance analysis tools 19

What Can You Monitor? Over 2 dozen system performance metrics can be monitored CPU Utilization (Average) CPU Utilization (Interactive Jobs) CPU Utilization (Interactive Feature) CPU Utilization Basic (Average) CPU Utilization (Secondary Workloads) CPU Utilization (Database Capability) Interactive Response Time (Average) Interactive Response Time (Maximum) Transaction Rate (Average) Transaction Rate (Interactive) Batch Logical Database I/O Disk Arm Utilization (Average) Disk Arm Utilization (Maximum) Communications IOP Utilization (Average) Communications IOP Utilization (Maximum) Communications Line Utilization (Average) Communications Line Utilization (Maximum) LAN Utilization (Average) LAN Utilization (Maximum) Machine Pool Faults User Pool Faults (Average) User Pool Faults (Maximum) Disk Storage (Average) Disk Storage (Maximum) Disk IOP Utilization (Average) Disk IOP Utilization (Maximum) You can monitor one or many performance metrics in each monitor System Monitors use Collection Services as their data source Collection Services provides only the data that System Monitors need to graph the metrics that are included in the monitor 20

Define A Monitor Select 'New Monitor...' and specify General properties 21

Define A Monitor Select 'Metrics to monitor' and press OK to create What to monitor How often Vertical axis Horizontal axis 22

Start A Monitor Select the monitor, then the start button to select systems/groups 23

View A Monitor Metric data 24

View A Monitor Click on graph point to see upper right pane Click on object graph in upper right pane to see details in lower right 25

Setting Threshold Condition indicating problem exists Condition indicating problem resolved Automation will send a message if condition occurs 26

Viewing Thresholds Threshold Indicators Drill down with Actions 27

Threshold Actions i PC client 28

Monitoring with IBM Systems Director http://ibmsystemsmag.blogs.com/i_can/2010/03/i-can-manage-i-with-ibm-systems-director.html 29

Types of Managed Systems Agentless - SSH (5733-SC1 Option 1) Best for environments that require very small footprints and are used for specific tasks, such as one-time inventory collection, firmware and driver updates and remote deployment Platform Agent - CIMOM (5722-UME, 5770-UME) - IBM Universal Manageability Enablement for i Provides agentless functions and... Common Information Model (CIM) events and monitoring Monitoring message queues (5770-UME V1R3 or later) Inventory Group and cumulative PTF support Common Agent - CAS for IBM i Provides agentless functions and platform agent functions and Additional monitoring features File transfer http://publib.boulder.ibm.com/infocenter/director/pubs/topic/com.ibm.director.tbs.helps.doc/fqm0_tbs_ibm_i_endpoints.html http://www.ibmsystemsmag.com/ibmi/director_agents/35716p1.aspx 30

Common CIM Monitors 31

IBM i Monitors 34 Metrics for common monitoring scenarios 32

Monitor Thresholds 33

Automation Plans - Actions 34

Create Your Own Monitor

Manage Processes - Requires CAS Agent WRKACTJOB via Director 36

IBM Systems Director - Event Filters for IBM i Messages 37

IBM Tivoli Monitoring http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/main_os400.pdf 38

Monitor Historical Performance Trends 39

Monitor Historical Performance Trends The Graph History function in System i Navigator is an extension of the System Monitors support System Monitors allow you to view performance over the last hour Graph History allows you to view performance data over days, weeks, or months to identify trends and help plan for future needs PM for Power Systems needs to be running to view Graph History data that is older than one week You do not need to send PM data to IBM to use Graph History but the collection facility on your system needs to be running If you send PM data to IBM, you will have access to performance graphs via the web to help you analyze performance PM for Power Systems has additional detail not found in Graph History 40

Viewing Data Using Graph History 41

Options What to view Length of time to view Click Refresh to view the data 42

Layout 43

Viewing Data Summarized information Dynamic Help Glide over a graph point More information Zoom (in/out) Slider bar Right-click on graph data point 44

Exporting Data Export to PC format 45

Performance Management for Power Systems PM for Power Systems If you don t want to manage the collection of performance data yourself By default PM will be active and collect performance data PM starts Collection Services on Version 5 releases Beginning with 6.1, Collection Services is started by default without PM starting it PM data can be sent to the IBM Workload Estimator Activating PM does not mean that you have to transmit performance data to IBM The data remains on your system unless you explicitly request that it be sent to IBM There are many good reasons to transmit to IBM even if you don t purchase additional PM services Easy to understand reports that help you manage performance 46 Trending information to help you plan for future upgrades

Starting Performance Management Can be started with System i Navigator or the CFGPM400 CL Command Transmission of data to IBM requires a Service Agent connection Complete instructions located at: http://publib.boulder.ibm.com/infocenter/systems/scope/i5os/topic/rzahx/rzahxplangrow1pm1.htm?tocnode=int_220143 47

Configuring Performance Management IBM Systems Director Navigator for i 48

Performance Management for Power Systems http://www-03.ibm.com/systems/power/support/pm/index.html 49

Accessing PM for Power Systems Reports Sign On https://pmeserver.rochester.ibm.com/pmserverinfo/loginpage.jsp 50

PM for Power Systems Enterprise View Click here for detailed performance reports for this system or partition 51

Viewing PM Reports 52

Basic Performance Analysis 53

Basic Performance Analysis When a performance problem occurs you often need to use performance analysis tools to identify the cause of the problem to correct it Beginning with 6.1, you now have two choices for basic performance analysis: The Performance Tools plug-in in System i Navigator IBM Systems Director Navigator for i Performance tasks Manage performance collections Performance Data Investigator 54

Performance Tools plug-in in System i Navigator The Performance Tools plug-in presents more detail than System Monitors which provides more capability to analyze the cause of a performance issue Graphs are similar to System Monitor graphs Multiple data views allow you to analyze performance in many ways Summary statistics provide an overall view of system performance Drill down to the time interval when a problem occurred and use the power of the GUI to sort performance data by any available metric Beginning with 6.1, it is recommended to use the IBM Systems Director Navigator Performance tasks Wait data included Many more charts to look at the data Can view all charts in table format Extensive customization capabilities 55

Starting the Performance Tools Plug-In 56

Viewing Performance Database Members Display data Convert member to current release Delete member 57

IBM Systems Director Navigator for i Performance Tasks Browser-based performance tasks Manage performance data collections Graphical user interface for collecting performance data Graphically view and analyze performance data. Collection Services Health Indicators Job Watcher Disk Watcher Performance Explorer http://www.ibm.com/developerworks/ibmi/library/i-pdi/index.html 58

Investigate Data 59

Investigate Data Collection Services 60

CPU Utilization and Waits Overview Start with CPU Utilization and Waits Overview Shows CPU Utilization (red line) Shows Wait Information (stacked bars) Green bars are disk time Can easily identify when the CPU utilization dropped that disk time went up Drill down for further analysis Type of disk operations Contributing jobs 61

Interactive Charts Interactive tools Selection Pan Tooltips Zoom Zoom out Reset zoom Drill-down options to look at other views of the data 62

Display Charts as Tables, Tables as Charts 63

Health Indicators 64

Physical System Charts Collection Services has the ability to collect certain high-level cross-partition processor performance metrics for all logical partitions on the same single physical server regardless of operating system. This is available on Power 6 and above servers, with a minimum firmware level xx340_061. When this data is available, it can be viewed via several perspectives found under "Physical System". HMC option to enable performance collection must be turned on for the IBM i partition to collect the data http://ibmsystemsmag.blogs.com/i_can/2009/10/i-can-display-cpu-utilization-for-all-partitions.html

Logical Partitions Overview Requires Power 6 and IBM i 6.1 or later 66

Collection Manager 67

Advanced Performance Analysis 68

Advanced Performance Analysis Advanced and detailed analysis is necessary to fully diagnose some performance problems IBM i has sophisticated tools for this purpose Job Watcher Disk Watcher Performance Explorer idoctor product 69

Job Watcher Job Watcher collects more detailed performance data than collection services Call Stacks Activation Group information SQL Statements Continuous sampling Allows for deep performance diagnostics 70

Disk Watcher Statistical Overview Average response times and total I/Os for the entire collection Good starting point to get an overview of the entire collection Statistical Details Detailed statistics Trace Detailed information on every I/O operation Can identify thread/task associated with the I/O operation Can identify program/ procedure doing the I/O operation 71

Performance Explorer Performance Explorer is the most sophisticated IBM i performance tool Can collect the details of every I/O operation, every task switch Hundreds of events collected Thus, most complex to use Generally used by IBM performance analysis experts 72

IBM idoctor for i Product developed by the IBM Rochester Support Center for deep, detailed performance analysis Three major components Job Watcher Job Watcher Collection Services Investigator Disk Watcher PEX Analyzer Heap Analyzer http://www-912.ibm.com/i_dir/idoctor.nsf 73

Miscellaneous Performance Tools 74

Web Performance Advisor / Monitor 75

Interfaces for the Database Performance Tools System i Navigator Full set of database performance tools IBM Systems Director Navigator for i Subset of database functions

Index Advisor Advisor shows indexes that would help specific query (if any) Use advice along with Index Analyzer for the table to determine if index would be helpful Create button prompts Create Index dialog with recommended columns pre-filled

Live Analysis SQL Plan Cache Easy analysis of current SQL workloads Without overhead of database monitor Advanced filtering to quickly narrow focus to the most interesting SQL By number of executions By DB2 object Easy access to Visual Explain for detailed analysis Archive capability into snapshot for future comparisons SQE-Only Capability to have HVs inserted into statement

SQL Performance Monitor Toolset Have an existing application and want to capture the SQL activity and analyze it? Start / Stop database monitor Pre & Post-Filtering Summary or Detailed Import data from other systems Areas to watch: Index advised Full opens vs. pseudo opens Access plan rebuilds Index creates Table scans on big tables Index Scans over large number of keys Recommended during application tuning & problem determination Keep a baseline monitor run for comparison

Visual Explain Active window - Look at DB objects (indexes, constraints, etc.) involved Attributes show all the settings that impact query optimizer Labels include optimizer s estimates SQE plans show decomposed SQL statement for each operation Icon Highlighting to speed analysis Expensive nodes Index Advised nodes Debug messages help bridge to picture

Web-based Solution Collection Services PM for Power Systems Collection Services Job Watcher Disk Watcher Disk Watcher Collector IBM Systems Director Navigator for i WorkLoad Estimator Green Screen Job Watcher Collector Performance Explorer Performance Tools Reports System commands Batch Model Performance Explorer 81

A Redbooks publication! http://www.redbooks.ibm.com/redbooks/pdfs/sg247808.pdf 82

First Place To Go For Performance Tools Info http://www-03.ibm.com/systems/i/advantages/perfmgmt/ 83

Performance and Scalability Services The IBM i Performance and Scalability Services Center can provide facilities and hardware IN ROCHESTER to assist you in testing hardware or software changes Traditional benchmarks Release-to-release upgrades Assess application performance when migrating to a new release of IBM I Stress test your system Determine impact of application changes Proofs of Concept (e.g. HA alternatives; SSD analysis, external storage, etc.) Evaluate application scalability Capacity planning all with the availability of Lab Services IBM i experts and development personnel To request any of these services, submit at: http://www-03.ibm.com/systems/services/labservices/psscontact.html 84

IBM Systems Lab Services and Training Power Services Key Offerings Americas, WW Contacts High Availability Services on Power Systems (including Advanced Copy Services for PowerHA on IBM i) Systems Director Services PowerCare Services Frank Kriss Performance and Scalability services (including system, application, and database tuning) kriss@us.ibm.com, 507-253-1354 IBM i, High Availability Virtualization Services for AIX on Power Systems Karen Anderson Application and database modernization consulting (SOA implementation) kanders@us.ibm.com, 972-561-6337 IBM i Vouchers Linux on Power consulting, custom application development, implementation, and optimization services Stephen Brandenburg Security on Power consulting and implementation services System consolidation and migration service High Performance Computing consulting and implementation services SAP on IBM i consulting Power Blades on BladeCenter (including VIOS on i and blades running IBM i implementation) Smart Analytics services (including DB2 Web Query implementation and consulting) Public, private, customized and self-paced virtual training Power Systems Technical University www.ibm.com/systems/services/labservices stgls@us.ibm.com Mark Even even@us.ibm.com, 507-253-1313 IBM i sbranden@us.ibm.com, 301-803-6199 PowerVouchers, Virtualization Program, AIX SWOT/SWAT, AIX Allen Johnston allenrj@us.ibm.com, 704-340-9165 PowerCare Cindy Mestad cindysm@us.ibm.com, 507-253-6143 Power Performance and Scalability Center 2010 IBM Corporation

i Can... Blogs on Performance Topics Main i Can page http://ibmsystemsmag.blogs.com/i_can/ Performance blogs http://ibmsystemsmag.blogs.com/i_can/performance/ Systems Management blogs http://ibmsystemsmag.blogs.com/i_can/systems-management/ 86

Blog Posts http://ibmsystemsmag.blogs.com/i_can/ http://ibmsystemsmag.blogs.com/i_can/performance/ http://ibmsystemsmag.blogs.com/i_can/2012/10/performance-data-investigatorbetter-than-ever.html http://ibmsystemsmag.blogs.com/i_can/2012/09/job-performance-information.html http://ibmsystemsmag.blogs.com/i_can/2011/08/customizing-a-perspective-in-pdi.html http://ibmsystemsmag.blogs.com/i_can/2011/05/new-systems-director-navigator-service-packs.html http://ibmsystemsmag.blogs.com/i_can/2011/02/ibm-systems-director-navigator-for-i-work-management-integration-with-performance-tasks.html http://ibmsystemsmag.blogs.com/i_can/2010/07/i-can-collect-more-performance-data-in-71.html http://ibmsystemsmag.blogs.com/i_can/2010/05/-i-can-measure-disk-response-times.html http://ibmsystemsmag.blogs.com/i_can/2010/03/i-can-understand-scaled-cpu-time.html http://ibmsystemsmag.blogs.com/i_can/2010/02/i-can-use-power7-features-with-ibm-i-611.html http://ibmsystemsmag.blogs.com/i_can/2010/01/i-can-analyze-command-performance.html http://ibmsystemsmag.blogs.com/i_can/2009/11/i-can-tell-you-why-youre-waiting.html http://ibmsystemsmag.blogs.com/i_can/2009/10/i-can-display-cpu-utilization-for-all-partitions.html http://ibmsystemsmag.blogs.com/i_can/2009/10/i-can-investigate-performance-data.html 87

IBM i Web Sites with Performance Information IBM i Information Center http://publib.boulder.ibm.com/iseries/ Systems Management Performance IBM i Performance Management http://www-03.ibm.com/systems/power/software/i/management/performance/index.html Performance Management for Power Systems http://www-03.ibm.com/systems/power/support/pm/index.html IBM Workload Estimator http://www.ibm.com/systems/support/tools/estimator DeveloperWorks - http://www.ibm.com/developerworks/ibmi/ IBM i Performance Data Investigator http://www.ibm.com/developerworks/ibmi/library/i-pdi/index.html IBM i Performance Data Investigator Edit Perspectives http://www.ibm.com/developerworks/ibmi/library/i-pdiedit/index.html Forum http://www.ibm.com/developerworks/forums/forum.jspa?forumid=2751&cat=493 idoctor http://www-912.ibm.com/i_dir/idoctor.nsf Job Waits Whitepaper https://www-912.ibm.com/i_dir/idoctor.nsf/3b3c112f7fbe774c86256f4000757a8f/$file/job_waits_white_paper_61_71.pdf IBM Systems Director http://www-03.ibm.com/systems/management/director/index.html IBM Tivoli Monitoring http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/welcome.htm IBM Tivoli Monitoring Agent for i5/os http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/main_os400.pdf 88

Redbooks and Redpapers on IBM i Performance Tools IBM eserver iseries Performance Management Tools http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/redp4026.html?open A Systems Management Guide to Performance Management for System i and System p servers http://www.redbooks.ibm.com/abstracts/sg247122.html?open Sizing IBM i5/os Work on IBM System i5 Partitions http://www.redbooks.ibm.com/abstracts/sg246656.html?open Application and Program Performance Analysis Using PEX Statistics http://www.redbooks.ibm.com/abstracts/sg247457.html?open Managing OS/400 with Operations Navigator V5R1 Volume 5: Performance Management http://www.redbooks.ibm.com/abstracts/sg246565.html?open IBM idoctor iseries Job Watcher: Advanced Performance Tool http://www.redbooks.ibm.com/abstracts/sg246474.html?open Best Practices for Managing IBM i Jobs and Output (and a few other special tips) http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/redp4454.html?open i5/os Diagnostic Tools for System Administrators: An A to Z Reference for Problem Determination http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/sg248253.html?open IBM eserver iseries Systems Management Handbook http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/redp4070.html?open 89

Articles IBM Systems Magazine, IBM i - Sky High Performance, Aug 2009 http://www.ibmsystemsmag.com/ibmi/august09/coverstory/26021p1.aspx SystemiNetwork - Performance Data Investigator Consolidates Functions in One Place, June 2009 http://systeminetwork.com/article/performance-data-investigator-consolidates-functions-one-place SystemiNetwork - IBM Systems Director Navigator for i: Performance Tasks Overview, June 2009 http://systeminetwork.com/article/ibm-systems-director-navigator-i-performance-tasks-overview IBM Systems Magazine, IBM i A Command Performance, Nov 2008 http://www.ibmsystemsmag.com/ibmi/november08/administrator/22426p1.aspx IBM Systems Magazine, IBM i - Introducing IBM Systems Director Navigator for i5/os, Aug 2008 http://www.ibmsystemsmag.com/ibmi/august08/administrator/21503p1.aspx IBM Systems Magazine, IBM i A Collective Effort, Nov 2006 http://www.ibmsystemsmag.com/ibmi/november06/trends/7201p1.aspx IBM Systems Magazine, IBM i - Mission: Performance Management, Nov 2006 http://www.ibmsystemsmag.com/ibmi/november06/features/7129p1.aspx 90

Articles on Job Watcher idoctor versus IBM i 6.1 Performance Tools http://www.ibmsystemsmag.com/i5/october08/trends/21990p1.aspx Web Power http://www.ibmsystemsmag.com/i5/november08/administrator/22431p1.aspx Introduction to Job Watcher Green Screen Commands http://www.ibmsystemsmag.com/i5/november08/tipstechniques/22521p1.aspx Top 10 Hidden idoctor Gems http://www.ibmsystemsmag.com/ibmi/enewsletterexclusive/23868p1.aspx Using idoctor for iseries Job Watcher to Determine Why Jobs Wait http://www.ibmsystemsmag.com/ibmi/october05/technicalcorner/8896p1.aspx 91

Articles on Disk Performance A New Way to Look at Disk Performance http://www.ibmsystemsmag.com/i5/may07/administrator/15631p1.aspx Analyzing Disk Watcher Data http://www.ibmsystemsmag.com/i5/may08/tipstechniques/20662p1.aspx Using Wait State Accounting to Determine Disk Performance http://www.systeminetwork.com/artarchive/20700/index.html Understanding Disk Performance, Part 2: Disk Operation on i5/os http://www.systeminetwork.com/artarchive/20870/understanding_disk_performance Part_2 Disk_Operation_on_i5_OS.html Understanding Disk Performance, Part 3: Metrics of Disk Performance http://systeminetwork.com/article/understanding-disk-performance-metrics A Look at System i Integrated DASD Configuration and Performance under i5/os Redpaper REDP-3919-00 92

Special notices This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area. Information in this document concerning non-ibm products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of nonibm products should be addressed to the suppliers of those products. IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied. All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions. IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice. IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies. All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment. Revised September 26, 2006 93

Special notices (cont.) IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 5L, AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/os, i5/os (logo), IBM Business Partner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pseries, Rational, RISC System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xseries, z/os, zseries, Active Memory, Balanced Warehouse, CacheFlow, Cool Blue, IBM Systems Director VMControl, purescale, TurboCore, Chiphopper, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Parallel File System,, GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager, iseries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, POWER7, System i, System p, System p5, System Storage, System z, TME 10, Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. AltiVec is a trademark of Freescale Semiconductor, Inc. AMD Opteron is a trademark of Advanced Micro Devices, Inc. InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries or both. Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both. NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both. SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC). The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org. TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC). UNIX is a registered trademark of The Open Group in the United States, other countries or both. Other company, product and service names may be trademarks or service marks of others. Revised December 2, 2010 94

October 3, 2012 2011 IBM Power Systems Technical University October 10-14 Fontainebleau Miami Beach Miami, FL IBM Navigating the World of Performance Dawn May - dmmay@us.ibm.com Materials may not be reproduced in whole or in part without the prior written permission of IBM. 5.3

October 3, 2012 IBM i Performance Tools Suite idoctortcher a Job W alyzer n PEX A nalyzer A Heap PDV Gr PM for Pe e e n Sy rform Scr Ba stem anc een Pe tch M com e Too rf o m l rm odel and s Re an po s ce rt s Ex plo re r Po we rs yst em s WL E e nag Ma nt me rm Perfo C anc e Collection Services PTDV Sy r al ent D at a Pe stem DB rf To i N Ac /SQL ols G avig ato Sy tive J Mon UI r s te o b i m S s tor tatu s Third Party Tools Job Watcher Disk Watcher Performance Explorer We We b Per b P form erfo a rma nce A nce dvis r o Mo or t a ig nito t r Inves 2 -We have a lot of performance management capability -It s spread over many different user interfaces with no linkage between any of them -We have a tremendous amount of system instrumentation and several different mechanisms for collecting performance metrics 2

October 3, 2012 Where Do I Start When Managing Performance? 1. Collect System-wide Performance Data Collection Services 2. Automated Monitoring Your First Line of Defense System i Navigator System Monitors Guidelines for Setting Monitor Threshold Triggers IBM System Director Monitors 3. Monitor Historical Performance Trends System i Navigator Graph History PM for Power Systems IBM Systems Director 3

October 3, 2012 Where Do I Start When Analyzing Performance? 4. Basic Performance Analysis Performance Tools System i Navigator Plug-in Performance Data Investigator 5. Advanced Performance Analysis Job Watcher Disk Watcher Performance Explorer idoctor Suite 6. Miscellaneous Performance Tools WebSphere Performance Monitor / Advisor Database Performance Tools 4

October 3, 2012 Collect System-wide Performance Data 5

October 3, 2012 Collect System-wide Performance Data Collect Performance Data 24/7 If something goes wrong, you have data that will help analyze the problem, fix it, and prevent it from happening in the future If you can t solve the problem, you have information that makes it easier for IBM Support to solve the problem faster To provide a reliable baseline so you can understand the impact that a software, network, or environmental change had on the performance of your system To provide historical information that enables you to plan for future growth based on real trends, not guesses. 6

October 3, 2012 What is Collection Services? IBM i function that collects system and job level performance data Collects data from many system resources including: Jobs Disk Units Buses Pools Communication lines Collects data at regular intervals from 15 seconds to 1 hour Produces database files used by Performance Data Investigator Performance Tools for i PM for Power Systems Provides the data for System i Navigator System Monitors IBM Systems Director IBM i Monitors 7

October 3, 2012 Collection Services Data Categories System Bus Memory Pool and Pool Tuning Hardware Configuration Subsystem System CPU System level data Jobs MI tasks and threads Jobs Operating System Disk Storage IOP Local Response Time Communication (Base, Station, SAP) APPN ARM SNA SNADS Transaction TCP/IP Base TCP/IP Interface Integrated xseries Server (IxS) Extended Adaptive Cache User-defined Transactions Domino HTTP Server (Powered by Apache) Data Port Services LPAR WAS JVM *New in 6.1 Removable Storage *New in 7.1 External Storage *New in 7.1 System Internal Data *New in 7.1 Each category can be turned on/off and interval times can be set individually 8

October 3, 2012 IBM Systems Director Navigator for i Collection Services Configuration Collection Services runs in a job named QYPSPFRCOL in QSYSWRK. 9

October 3, 2012 Collection Services Properties Check this box if you plan to use: Performance Data Investigator Performance Tools IBM Systems Director Monitors They all require data in the database files Check this box if you plan to use the Performance Data Investigator. Performance summary data improves the performance of the initial queries when using PDI. 10

October 3, 2012 Collection Services Customized Data Collection Standard plus protocol is recommended. Use the Customize collection profile if you want to change the default time interval for individual categories. 11

October 3, 2012 Collection Services Customized Data Collection Select the category you want to change and add it to the customized categories. Then adjust the time. 12

October 3, 2012 Collection Services Data Retention The collection object has the original data. Standard data is the data in the DB2 files. You can always create the data in the DB2 files if you have the Collection Object. Recommend keeping at least 7 days of detailed data in case a problem occurs. 13

October 3, 2012 Creating database files with System Director Navigator 14

October 3, 2012 Creating database files with System Director Navigator Select the categories you want. Files will be created for the duration of the entire collection. Adjust the time if you want files created for a shorter duration. Increase this value if you want data created at a less frequent interval. 15

October 3, 2012 Collection Services System i Navigator All the prior customization can be done with the System i Navigator client as well. System i Navigator also has Graph History... Check bottom 2 boxes if you plan to use Graph History 16

October 3, 2012 Commands for Performance Data Collections STRPFRCOL - Start Performance Collection ENDPFRCOL - End Performance Collection CFGPFRCOL - Configure Performance Collection CHKPFRCOL - Check Performance Collection CVTPFRCOL Convert Performance Collection Added in 6.1: DLTPFRCOL Delete Performance Collection SAVPFRCOL Save Performance Collection RSTPFRCOL Restore Performance Collection 17

October 3, 2012 Monitoring 18

October 3, 2012 Automated Monitoring with System Monitors System Monitors gather and present real-time performance data Helps monitor the health of your system Helps identify potential performance problems before they become serious issues System Monitors provide multiple levels of performance information Level 1 System wide performance metrics such as CPU Utilization, Disk Utilization, etc. Level 2 A list of items that are contributing most to the Level 1 metric For CPU Utilization, it s a list of jobs that are consuming the most CPU For Disk Utilization, it s a list of disk arms that are the busiest Level 3 A list of performance metrics and properties for the Level 2 items Thresholds can be defined which will trigger an action when a system wide performance metric exceeds the defined comfort level For example, when CPU Utilization exceeds 80%, send a message to notify the operator System Monitors provide powerful capabilities to monitor what is happening on your system BUT finding out what caused the problem often requires other performance analysis tools 19

October 3, 2012 What Can You Monitor? Over 2 dozen system performance metrics can be monitored CPU Utilization (Average) CPU Utilization (Interactive Jobs) CPU Utilization (Interactive Feature) CPU Utilization Basic (Average) CPU Utilization (Secondary Workloads) CPU Utilization (Database Capability) Interactive Response Time (Average) Interactive Response Time (Maximum) Transaction Rate (Average) Transaction Rate (Interactive) Batch Logical Database I/O Disk Arm Utilization (Average) Disk Arm Utilization (Maximum) Communications IOP Utilization (Average) Communications IOP Utilization (Maximum) Communications Line Utilization (Average) Communications Line Utilization (Maximum) LAN Utilization (Average) LAN Utilization (Maximum) Machine Pool Faults User Pool Faults (Average) User Pool Faults (Maximum) Disk Storage (Average) Disk Storage (Maximum) Disk IOP Utilization (Average) Disk IOP Utilization (Maximum) You can monitor one or many performance metrics in each monitor System Monitors use Collection Services as their data source Collection Services provides only the data that System Monitors need to graph the metrics that are included in the monitor 20

October 3, 2012 Define A Monitor Select 'New Monitor...' and specify General properties 21

October 3, 2012 Define A Monitor Select 'Metrics to monitor' and press OK to create What to monitor How often Vertical axis Horizontal axis 22

October 3, 2012 Start A Monitor Select the monitor, then the start button to select systems/groups 23

October 3, 2012 View A Monitor Metric data 24

October 3, 2012 View A Monitor Click on graph point to see upper right pane Click on object graph in upper right pane to see details in lower right 25

October 3, 2012 Setting Threshold Condition indicating problem exists Condition indicating problem resolved Automation will send a message if condition occurs 26

October 3, 2012 Viewing Thresholds Threshold Indicators Drill down with Actions 27

October 3, 2012 Threshold Actions i PC client 28

October 3, 2012 Monitoring with IBM Systems Director http://ibmsystemsmag.blogs.com/i_can/2010/03/i-can-manage-i-with-ibm-systems-director.html 29

Types of Managed Systems Agentless - SSH (5733-SC1 Option 1) Best for environments that require very small footprints and are used for specific tasks, such as one-time inventory collection, firmware and driver updates and remote deployment Platform Agent - CIMOM (5722-UME, 5770-UME) - IBM Universal Manageability Enablement for i Provides agentless functions and... Common Information Model (CIM) events and monitoring Monitoring message queues (5770-UME V1R3 or later) Inventory Group and cumulative PTF support Common Agent - CAS for IBM i Provides agentless functions and platform agent functions and Additional monitoring features File transfer http://publib.boulder.ibm.com/infocenter/director/pubs/topic/com.ibm.director.tbs.helps.doc/fqm0_tbs_ibm_i_endpoints.html http://www.ibmsystemsmag.com/ibmi/director_agents/35716p1.aspx 30 The overall structure of IBM Systems Director 6.1 has a web-based console, a server with a data base, and managed resources. Manage systems can be of one of these three types. The following charts describe the protocols and functions for each of these types.

October 3, 2012 Common CIM Monitors 31

October 3, 2012 IBM i Monitors 34 Metrics for common monitoring scenarios 32

October 3, 2012 Monitor Thresholds 33

October 3, 2012 Automation Plans - Actions 34

October 3, 2012 Create Your Own Monitor

October 3, 2012 Manage Processes - Requires CAS Agent WRKACTJOB via Director 36 36

October 3, 2012 IBM Systems Director - Event Filters for IBM i Messages 37 37

October 3, 2012 IBM Tivoli Monitoring http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/main_os400.pdf 38

October 3, 2012 Monitor Historical Performance Trends 39

October 3, 2012 Monitor Historical Performance Trends The Graph History function in System i Navigator is an extension of the System Monitors support System Monitors allow you to view performance over the last hour Graph History allows you to view performance data over days, weeks, or months to identify trends and help plan for future needs PM for Power Systems needs to be running to view Graph History data that is older than one week You do not need to send PM data to IBM to use Graph History but the collection facility on your system needs to be running If you send PM data to IBM, you will have access to performance graphs via the web to help you analyze performance PM for Power Systems has additional detail not found in Graph History 40

October 3, 2012 Viewing Data Using Graph History 41

October 3, 2012 Options What to view Length of time to view Click Refresh to view the data 42

October 3, 2012 Layout 43

October 3, 2012 Viewing Data Summarized information Dynamic Help Glide over a graph point More information Zoom (in/out) Slider bar Right-click on graph data point 44

October 3, 2012 Exporting Data Export to PC format 45

October 3, 2012 Performance Management for Power Systems PM for Power Systems If you don t want to manage the collection of performance data yourself By default PM will be active and collect performance data PM starts Collection Services on Version 5 releases Beginning with 6.1, Collection Services is started by default without PM starting it PM data can be sent to the IBM Workload Estimator Activating PM does not mean that you have to transmit performance data to IBM The data remains on your system unless you explicitly request that it be sent to IBM There are many good reasons to transmit to IBM even if you don t purchase additional PM services Easy to understand reports that help you manage performance 46 Trending information to help you plan for future upgrades

October 3, 2012 Starting Performance Management Can be started with System i Navigator or the CFGPM400 CL Command Transmission of data to IBM requires a Service Agent connection Complete instructions located at: http://publib.boulder.ibm.com/infocenter/systems/scope/i5os/topic/rzahx/rzahxplangrow1pm1.htm?tocnode=int_220143 47 IBM recommends you never end the PM agent. 47

October 3, 2012 Configuring Performance Management IBM Systems Director Navigator for i 48 48

October 3, 2012 Performance Management for Power Systems http://www-03.ibm.com/systems/power/support/pm/index.html 49

October 3, 2012 Accessing PM for Power Systems Reports Sign On https://pmeserver.rochester.ibm.com/pmserverinfo/loginpage.jsp 50

October 3, 2012 PM for Power Systems Enterprise View Click here for detailed performance reports for this system or partition 51

October 3, 2012 Viewing PM Reports 52

October 3, 2012 Basic Performance Analysis 53

October 3, 2012 Basic Performance Analysis When a performance problem occurs you often need to use performance analysis tools to identify the cause of the problem to correct it Beginning with 6.1, you now have two choices for basic performance analysis: The Performance Tools plug-in in System i Navigator IBM Systems Director Navigator for i Performance tasks Manage performance collections Performance Data Investigator 54

October 3, 2012 Performance Tools plug-in in System i Navigator The Performance Tools plug-in presents more detail than System Monitors which provides more capability to analyze the cause of a performance issue Graphs are similar to System Monitor graphs Multiple data views allow you to analyze performance in many ways Summary statistics provide an overall view of system performance Drill down to the time interval when a problem occurred and use the power of the GUI to sort performance data by any available metric Beginning with 6.1, it is recommended to use the IBM Systems Director Navigator Performance tasks Wait data included Many more charts to look at the data Can view all charts in table format Extensive customization capabilities 55

October 3, 2012 Starting the Performance Tools Plug-In 56

October 3, 2012 Viewing Performance Database Members Display data Convert member to current release Delete member 57

October 3, 2012 IBM Systems Director Navigator for i Performance Tasks Browser-based performance tasks Manage performance data collections Graphical user interface for collecting performance data Graphically view and analyze performance data. Collection Services Health Indicators Job Watcher Disk Watcher Performance Explorer http://www.ibm.com/developerworks/ibmi/library/i-pdi/index.html 58 Limited information on the V6R1 Performance tasks in this presentation separate sessions on what s new in V6R1 for IBM Performance tools and a whole session on Performance Data Investigator. 58

October 3, 2012 Investigate Data 59

October 3, 2012 Investigate Data Collection Services 60

October 3, 2012 CPU Utilization and Waits Overview Start with CPU Utilization and Waits Overview Shows CPU Utilization (red line) Shows Wait Information (stacked bars) Green bars are disk time Can easily identify when the CPU utilization dropped that disk time went up Drill down for further analysis Type of disk operations Contributing jobs 61

October 3, 2012 Interactive Charts Interactive tools Selection Pan Tooltips Zoom Zoom out Reset zoom Drill-down options to look at other views of the data 62

October 3, 2012 Display Charts as Tables, Tables as Charts 63

October 3, 2012 Health Indicators 64

October 3, 2012

October 3, 2012

October 3, 2012 Collection Manager 67 67

October 3, 2012 Advanced Performance Analysis 68

October 3, 2012 Advanced Performance Analysis Advanced and detailed analysis is necessary to fully diagnose some performance problems IBM i has sophisticated tools for this purpose Job Watcher Disk Watcher Performance Explorer idoctor product 69

October 3, 2012 Job Watcher Job Watcher collects more detailed performance data than collection services Call Stacks Activation Group information SQL Statements Continuous sampling Allows for deep performance diagnostics 70

October 3, 2012 Disk Watcher Statistical Overview Average response times and total I/Os for the entire collection Good starting point to get an overview of the entire collection Statistical Details Detailed statistics Trace Detailed information on every I/O operation Can identify thread/task associated with the I/O operation Can identify program/ procedure doing the I/O operation 71

October 3, 2012

October 3, 2012 IBM idoctor for i Product developed by the IBM Rochester Support Center for deep, detailed performance analysis Three major components Job Watcher Job Watcher Collection Services Investigator Disk Watcher PEX Analyzer Heap Analyzer http://www-912.ibm.com/i_dir/idoctor.nsf 73

October 3, 2012 Miscellaneous Performance Tools 74

October 3, 2012 Web Performance Advisor / Monitor 75

Interfaces for the Database Performance Tools System i Navigator Full set of database performance tools IBM Systems Director Navigator for i Subset of database functions The main ideas here: System i Navigator in V6R1 contains the full set of database tools. This interface has been around for several releases. System i Navigator tasks have been available since V5R3, and have been added to each release. Uses the same dialogs as the installed client. IBM Systems Director Navigator for i5/os is new in V6R1. Also uses the same dialogs as the installed client, but does not support the full database functionality.

Index Advisor Advisor shows indexes that would help specific query (if any) Use advice along with Index Analyzer for the table to determine if index would be helpful Create button prompts Create Index dialog with recommended columns pre-filled

Live Analysis SQL Plan Cache Easy analysis of current SQL workloads Without overhead of database monitor Advanced filtering to quickly narrow focus to the most interesting SQL By number of executions By DB2 object Easy access to Visual Explain for detailed analysis Archive capability into snapshot for future comparisons SQE-Only Capability to have HVs inserted into statement Adding in stmt with variable is a big deal Column customization also included

SQL Performance Monitor Toolset Have an existing application and want to capture the SQL activity and analyze it? Start / Stop database monitor Pre & Post-Filtering Summary or Detailed Import data from other systems Areas to watch: Index advised Full opens vs. pseudo opens Access plan rebuilds Index creates Table scans on big tables Index Scans over large number of keys Recommended during application tuning & problem determination Keep a baseline monitor run for comparison application performance tuning. overview of your whole application so you can narrow it down queries help to summarize and help you understand the application detailed results are ordered by the most important then go to VE on that statement

Visual Explain Active window - Look at DB objects (indexes, constraints, etc.) involved Attributes show all the settings that impact query optimizer Labels include optimizer s estimates SQE plans show decomposed SQL statement for each operation Icon Highlighting to speed analysis Expensive nodes Index Advised nodes Debug messages help bridge to picture I know which statement is the problem

October 3, 2012 Web-based Solution Collection Services PM for Power Systems Collection Services Job Watcher Disk Watcher Disk Watcher Collector IBM Systems Director Navigator for i WorkLoad Estimator Green Screen Job Watcher Collector Performance Explorer Performance Tools Reports System commands Batch Model Performance Explorer 81 81

October 3, 2012 A Redbooks publication! http://www.redbooks.ibm.com/redbooks/pdfs/sg247808.pdf 82

October 3, 2012 First Place To Go For Performance Tools Info http://www-03.ibm.com/systems/i/advantages/perfmgmt/ 83

October 3, 2012 84

IBM Systems Lab Services and Training Power Services Key Offerings Americas, WW Contacts High Availability Services on Power Systems (including Advanced Copy Services for PowerHA on IBM i) Systems Director Services PowerCare Services Frank Kriss Performance and Scalability services (including system, application, and database tuning) kriss@us.ibm.com, 507-253-1354 IBM i, High Availability Virtualization Services for AIX on Power Systems Karen Anderson Application and database modernization consulting (SOA implementation) kanders@us.ibm.com, 972-561-6337 IBM i Vouchers Linux on Power consulting, custom application development, implementation, and optimization services Stephen Brandenburg Security on Power consulting and implementation services System consolidation and migration service High Performance Computing consulting and implementation services SAP on IBM i consulting Power Blades on BladeCenter (including VIOS on i and blades running IBM i implementation) Smart Analytics services (including DB2 Web Query implementation and consulting) Public, private, customized and self-paced virtual training Power Systems Technical University www.ibm.com/systems/services/labservices stgls@us.ibm.com Mark Even even@us.ibm.com, 507-253-1313 IBM i sbranden@us.ibm.com, 301-803-6199 PowerVouchers, Virtualization Program, AIX SWOT/SWAT, AIX Allen Johnston allenrj@us.ibm.com, 704-340-9165 PowerCare Cindy Mestad cindysm@us.ibm.com, 507-253-6143 Power Performance and Scalability Center 2010 IBM Corporation

October 3, 2012 i Can... Blogs on Performance Topics Main i Can page http://ibmsystemsmag.blogs.com/i_can/ Performance blogs http://ibmsystemsmag.blogs.com/i_can/performance/ Systems Management blogs http://ibmsystemsmag.blogs.com/i_can/systems-management/ 86 86

October 3, 2012 Blog Posts http://ibmsystemsmag.blogs.com/i_can/ http://ibmsystemsmag.blogs.com/i_can/performance/ http://ibmsystemsmag.blogs.com/i_can/2012/10/performance-data-investigatorbetter-than-ever.html http://ibmsystemsmag.blogs.com/i_can/2012/09/job-performance-information.html http://ibmsystemsmag.blogs.com/i_can/2011/08/customizing-a-perspective-in-pdi.html http://ibmsystemsmag.blogs.com/i_can/2011/05/new-systems-director-navigator-service-packs.html http://ibmsystemsmag.blogs.com/i_can/2011/02/ibm-systems-director-navigator-for-i-work-management-integration-with-performance-tasks.html http://ibmsystemsmag.blogs.com/i_can/2010/07/i-can-collect-more-performance-data-in-71.html http://ibmsystemsmag.blogs.com/i_can/2010/05/-i-can-measure-disk-response-times.html http://ibmsystemsmag.blogs.com/i_can/2010/03/i-can-understand-scaled-cpu-time.html http://ibmsystemsmag.blogs.com/i_can/2010/02/i-can-use-power7-features-with-ibm-i-611.html http://ibmsystemsmag.blogs.com/i_can/2010/01/i-can-analyze-command-performance.html http://ibmsystemsmag.blogs.com/i_can/2009/11/i-can-tell-you-why-youre-waiting.html http://ibmsystemsmag.blogs.com/i_can/2009/10/i-can-display-cpu-utilization-for-all-partitions.html http://ibmsystemsmag.blogs.com/i_can/2009/10/i-can-investigate-performance-data.html 87 87

October 3, 2012 IBM i Web Sites with Performance Information IBM i Information Center http://publib.boulder.ibm.com/iseries/ Systems Management Performance IBM i Performance Management http://www-03.ibm.com/systems/power/software/i/management/performance/index.html Performance Management for Power Systems http://www-03.ibm.com/systems/power/support/pm/index.html IBM Workload Estimator http://www.ibm.com/systems/support/tools/estimator DeveloperWorks - http://www.ibm.com/developerworks/ibmi/ IBM i Performance Data Investigator http://www.ibm.com/developerworks/ibmi/library/i-pdi/index.html IBM i Performance Data Investigator Edit Perspectives http://www.ibm.com/developerworks/ibmi/library/i-pdiedit/index.html Forum http://www.ibm.com/developerworks/forums/forum.jspa?forumid=2751&cat=493 idoctor http://www-912.ibm.com/i_dir/idoctor.nsf Job Waits Whitepaper https://www-912.ibm.com/i_dir/idoctor.nsf/3b3c112f7fbe774c86256f4000757a8f/$file/job_waits_white_paper_61_71.pdf IBM Systems Director http://www-03.ibm.com/systems/management/director/index.html IBM Tivoli Monitoring http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/welcome.htm IBM Tivoli Monitoring Agent for i5/os http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/main_os400.pdf 88 88

October 3, 2012 Redbooks and Redpapers on IBM i Performance Tools IBM eserver iseries Performance Management Tools http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/redp4026.html?open A Systems Management Guide to Performance Management for System i and System p servers http://www.redbooks.ibm.com/abstracts/sg247122.html?open Sizing IBM i5/os Work on IBM System i5 Partitions http://www.redbooks.ibm.com/abstracts/sg246656.html?open Application and Program Performance Analysis Using PEX Statistics http://www.redbooks.ibm.com/abstracts/sg247457.html?open Managing OS/400 with Operations Navigator V5R1 Volume 5: Performance Management http://www.redbooks.ibm.com/abstracts/sg246565.html?open IBM idoctor iseries Job Watcher: Advanced Performance Tool http://www.redbooks.ibm.com/abstracts/sg246474.html?open Best Practices for Managing IBM i Jobs and Output (and a few other special tips) http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/redp4454.html?open i5/os Diagnostic Tools for System Administrators: An A to Z Reference for Problem Determination http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/sg248253.html?open IBM eserver iseries Systems Management Handbook http://www.redbooks.ibm.com/redbooks.nsf/redbookabstracts/redp4070.html?open 89 89

October 3, 2012 Articles IBM Systems Magazine, IBM i - Sky High Performance, Aug 2009 http://www.ibmsystemsmag.com/ibmi/august09/coverstory/26021p1.aspx SystemiNetwork - Performance Data Investigator Consolidates Functions in One Place, June 2009 http://systeminetwork.com/article/performance-data-investigator-consolidates-functions-one-place SystemiNetwork - IBM Systems Director Navigator for i: Performance Tasks Overview, June 2009 http://systeminetwork.com/article/ibm-systems-director-navigator-i-performance-tasks-overview IBM Systems Magazine, IBM i A Command Performance, Nov 2008 http://www.ibmsystemsmag.com/ibmi/november08/administrator/22426p1.aspx IBM Systems Magazine, IBM i - Introducing IBM Systems Director Navigator for i5/os, Aug 2008 http://www.ibmsystemsmag.com/ibmi/august08/administrator/21503p1.aspx IBM Systems Magazine, IBM i A Collective Effort, Nov 2006 http://www.ibmsystemsmag.com/ibmi/november06/trends/7201p1.aspx IBM Systems Magazine, IBM i - Mission: Performance Management, Nov 2006 http://www.ibmsystemsmag.com/ibmi/november06/features/7129p1.aspx 90 90

October 3, 2012 Articles on Job Watcher idoctor versus IBM i 6.1 Performance Tools http://www.ibmsystemsmag.com/i5/october08/trends/21990p1.aspx Web Power http://www.ibmsystemsmag.com/i5/november08/administrator/22431p1.aspx Introduction to Job Watcher Green Screen Commands http://www.ibmsystemsmag.com/i5/november08/tipstechniques/22521p1.aspx Top 10 Hidden idoctor Gems http://www.ibmsystemsmag.com/ibmi/enewsletterexclusive/23868p1.aspx Using idoctor for iseries Job Watcher to Determine Why Jobs Wait http://www.ibmsystemsmag.com/ibmi/october05/technicalcorner/8896p1.aspx 91 91

October 3, 2012 Articles on Disk Performance A New Way to Look at Disk Performance http://www.ibmsystemsmag.com/i5/may07/administrator/15631p1.aspx Analyzing Disk Watcher Data http://www.ibmsystemsmag.com/i5/may08/tipstechniques/20662p1.aspx Using Wait State Accounting to Determine Disk Performance http://www.systeminetwork.com/artarchive/20700/index.html Understanding Disk Performance, Part 2: Disk Operation on i5/os http://www.systeminetwork.com/artarchive/20870/understanding_disk_performance Part_2 Disk_Operation_on_i5_OS.html Understanding Disk Performance, Part 3: Metrics of Disk Performance http://systeminetwork.com/article/understanding-disk-performance-metrics A Look at System i Integrated DASD Configuration and Performance under i5/os Redpaper REDP-3919-00 92 92

October 3, 2012 Special notices This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area. Information in this document concerning non-ibm products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of nonibm products should be addressed to the suppliers of those products. IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied. All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions. IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice. IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies. All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment. Revised September 26, 2006 93 93

October 3, 2012 Special notices (cont.) IBM, the IBM logo, ibm.com AIX, AIX (logo), AIX 5L, AIX 6 (logo), AS/400, BladeCenter, Blue Gene, ClusterProven, DB2, ESCON, i5/os, i5/os (logo), IBM Business Partner (logo), IntelliStation, LoadLeveler, Lotus, Lotus Notes, Notes, Operating System/400, OS/400, PartnerLink, PartnerWorld, PowerPC, pseries, Rational, RISC System/6000, RS/6000, THINK, Tivoli, Tivoli (logo), Tivoli Management Environment, WebSphere, xseries, z/os, zseries, Active Memory, Balanced Warehouse, CacheFlow, Cool Blue, IBM Systems Director VMControl, purescale, TurboCore, Chiphopper, Cloudscape, DB2 Universal Database, DS4000, DS6000, DS8000, EnergyScale, Enterprise Workload Manager, General Parallel File System,, GPFS, HACMP, HACMP/6000, HASM, IBM Systems Director Active Energy Manager, iseries, Micro-Partitioning, POWER, PowerExecutive, PowerVM, PowerVM (logo), PowerHA, Power Architecture, Power Everywhere, Power Family, POWER Hypervisor, Power Systems, Power Systems (logo), Power Systems Software, Power Systems Software (logo), POWER2, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, POWER6+, POWER7, System i, System p, System p5, System Storage, System z, TME 10, Workload Partitions Manager and X-Architecture are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. AltiVec is a trademark of Freescale Semiconductor, Inc. AMD Opteron is a trademark of Advanced Micro Devices, Inc. InfiniBand, InfiniBand Trade Association and the InfiniBand design marks are trademarks and/or service marks of the InfiniBand Trade Association. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries or both. Microsoft, Windows and the Windows logo are registered trademarks of Microsoft Corporation in the United States, other countries or both. NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both. SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC). The Power Architecture and Power.org wordmarks and the Power and Power.org logos and related marks are trademarks and service marks licensed by Power.org. TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC). UNIX is a registered trademark of The Open Group in the United States, other countries or both. Other company, product and service names may be trademarks or service marks of others. Revised December 2, 2010 94 94