Capacity Planning: Where the Mistakes Are Session: 11598

Transcription

1 Capacity Planning: Where the Mistakes Are Session: Kathy Walsh IBM Washington Systems Center 2011 IBM Corporation

2 Trademarks ATS - Washington Systems Center The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both. Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market. Those trademarks followed by are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States. For a complete list of IBM Trademarks, see *, AS/400, e business(logo), DBE, ESCO, eserver, FICON, IBM, IBM (logo), iseries, MVS, OS/390, pseries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xseries, z/os, zseries, z/vm, System i, System i5, System p, System p5, System x, System z, System z9, BladeCenter The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce. * All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O conf iguration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM repres entative or Business Partner for the most current pricing in your geography. 2

3 Agenda Measurement Intervals LSPR Service Units vs MIPS vs MSU vs zpcr Hiperdispatch and IRD Low Utilization Specialty Engines 3

4 Evaluate Current Performance Data Capacity Planning ASSUMES the system is well tuned Generally SMF Records 70:78, 30 are used for Analysis A good planning process will still make some rudimentary checks to evaluate the performance of the system Good z/os capture ratio Latent demand in an LPAR Latent demand in a CP (single TCB architectures) Latent demand in Job queues Consistently high utilization Well-running I/O subsystem No processor storage contention Evaluate the WLM setup to ensure the workloads have enough granularity to get a reasonable view of the system Need to look at the report class granularity 4 4

5 Picking the Timeframe for Analysis Pick the period which drives your capacity Understand the business cycle Know when there are software problems which distort the typical capacity Review capacity based on importance Percent CP B A T C H T S O S T C O P E N D B A S E O N L IN E S S Y S T E M H o u r 5

6 Intervals, Amount of Data, and Confidence Don t use a single period to do capacity planning Review multiple days, month end, other peak periods When using tooling ensure you look across time to ensure the period being fed to the tool is representative Must ensure the period represents valid data Order the capacity charts based on WLM importance Make sure the correct workloads disappear when the CEC becomes busy If not, examine the WLM policy 6

7 LSPR: The heart of the data LSPR Old/new processors measured at high utilization with same workload Workloads and environments updated to stay current Presents capacity ratios among processors Source for Single Number Metrics - MIPS, MSU, SU/SEC Based on: Average Workload Median LPAR Config z L2 L1 CPU1 L3 Cache... LSPR 1.11 Table LOW RNI Memory L4 Cache L2 L1 CPU4... AVERAGE RNI L2 L CPU1 L3 Cache... L2 L1 CPU4 HIGH RNI Nest 7

8 Picking a Workload s Capacity Curve Many factors influence a workload's capacity curve What they are actually affecting is the workload's Relative Nest Intensity The net effect of the interaction of all these factors determines the capacity curve The chart below indicates the trend of the effect of each factor but is not absolute Some batch will have high RNI while some transactional workloads will have low Some low IO rate workloads will have high RNI, while some high IO rates will have low Low Relative Nest Intensity High Batch Application Type Transactional Low IO Rate High Single Application Mix Many Intensive CPU Usage Light High Locality Data Reference Pattern Diverse Simple LPAR Configuration Complex Extensive Software Configuration Tuning Limited 8

9 Using MIPS Tables Don t Get Crossed Up MUST know which LSPR table version the values are derived from NEVER mix MIPS values from different LSPR tables Mistakes happen when processors are in one LSPR table but not the other Contact IBM and ask for help [email protected] MIPS tables are only valid for general positioning NOT for capacity planning z114 zpcr zpcr CPs CPs R11mlt A A A A A B B B B B Z9-BC zpcr zpcr R07 CPs R9mlt R6mlt A A A B B B

10 Service Units Overview Unitless number, used by z/os (SRM) to determine amount of service a transaction is receiving Service units are accumulated for CPU consumed (TCB and SRB), I/O activity (IO), and processor storage (MSO) Recommend MSO values be set to SU/SEC is used to set the MSU value which is used to give a single price for software System z or z/os can change performance of processor via the service stream and the SU/SEC value is not updated Does not include the impacts of: Actual LPAR Configuration Actual Workloads HiperDispatch Park/UnPark of Logical CPs Specialty CPs SU/SEC or MSU values should not be used to determine the relative capacity of processors 10

11 Calculating MSUs CPU Factor Provided by hardware vendor CPU Service Definition Coefficient Provided by Installation Typically CPU = 1.0, SRB=1.0 Service units are derived from CPU seconds CPU SU = (TCB seconds + SRB Seconds * CPU Factor) * CPU SDC MSU Million Service Units CPU Factor * # of GCPs * 3600 secs / 1,000,000 CALC PUBLISH MSU MSU CALC PUBLISH MSU MSU

12 Calculating MSUs MSU values are single number metrics which are the same regardless of LPAR configuration, workload, or operating system In certain environments like the z9 and the z10 the published MSU values contained a technology dividend to provide reduced software prices With LSPR v1.11 the MSU value is set by the Average RNI workload using the LSPR Multi-Image table z9 vs z10 z10 vs z196 CALC PUBLISH LOW AVERAGE HIGH MSU MSU MIPS MIPS MIPS CALC PUBLISH LOW AVERAGE HIGH MSU MSU MIPS MIPS MIPS

13 Processors are not single speeds Example Three different LPAR configurations and their impact on capacity z9 vs z10 ZPCR ZPCR ZPCR ZPCR CALC PUBLISH AVERAGE 3 LPAR 6 LPAR 12 LPAR MSU MSU MIPS MIPS MIPS MIPS z10 vs z196 ZPCR ZPCR ZPCR ZPCR CALC PUBLISH AVERAGE 3 LPAR 6 LPAR 12 LPAR MSU MSU MIPS MIPS MIPS MIPS

14 Capacity Planning Needs to be aware of Actual Capacity Example Take two z processors with different LPAR configurations and consolidate to a single larger z196 Compare using LSPR Multi Image Average Workload Table vs zpcr Info ZPCR ZPCR ZPCR ZPCR CALC PUBLISH AVERAGE 3 LPAR 6 LPAR 12 LPAR MSU MSU MIPS MIPS MIPS MIPS SCP (System Control Program) is operating system level z/os 1.10* - indicates 1.11 LSPR tables are being used, z/os 1.10 software constraints 14

15 Capacity Planning Needs to Be Aware of Actual Capacity Guideline: The new CEC should be around 90% busy Z # / 9630 Z # / PCP too Big AVG ORIGINAL zpcr NEW CEC Weight MIPS LPAR WEIGHT MIPS W eight LCP AVG AVG zpcr zpcr MSU MIPS MIPS % MIPS MIPS %

16 Server Consolidations The capacity estimates need to be made using the same LSPR tables Capacity needs to be measured at end points, not intermediate stages Example: 9 LPARs from 2 different CEC need to migrate to new footprint Measure all 9 LPARs at one point in time (BEFORE) and project capacity (AFTER) Migrate 3 LPARs to new CEC can t measure at this point and compare to AFTER AFTER expectation was set for 9 LPARs not 3 Migrate 3 of 6 LPARs on CEC2 to new CEC can t measure at this point AFTER expectation was set for 9 LPARs Can t use current utilization of remaining 3 LPARs on old CEC and project forward since the MIPS rating of old has changed Need to use zpcr to do System z Capacity Planning 16

17 Hiperdispatch z/os exclusive: ATS - Washington Systems Center Parks and unparks logical CPs based on capacity demands Works to (re)dispatch work to same set of physical CPs Heuristic Sensitivities Processor cache technology Number of physical processors Size of the z/os partition Logical : Physical processor ratio Memory reference pattern Exploitation of IRD Vary CPU Management LSPR data for z10 and z196 assumes Hiperdispatch=YES 17

18 Hiperdispatch Capacity Guidelines z10 1-2% for a 1 book environment - less than 12 purchased CPs/zIIPs/zAAPs 2-4% for a 2 book environment - less than 26 purchased CPs/zIIPs/zAAPs 4-7% for a 3 book environment - less than 40 purchased CPs/zIIPs/zAAPs 7-10% for a 4 book environment - less than 64 purchased CPs/zIIPs/zAAPs z196 Share of the partition - assumes 1.5 logical to physical ratio Number of Physical CPs + ziips + zaaps <= <= share in processors < 1.5 0% 0% 0% 0% 1.5 <= share in processors < 3 2-5% 3-6% 3-6% 3-6% 3 <= share in processors < 6 4-8% 5-9% 6-10% 6-10% 6 <= share in processors < % 7-13% 8-14% 8-16% 12 <= share in processors < % 10-18% 11-21% 24 <= share in processors < % 12-24% 48 <= share in processors <= % 18

19 Latent Demand: LPAR Busy vs MVS Busy CPU 2097 CPC CAPACITY 1451 MODEL 719 CHANGE REASON=N/A HIPERDISPATCH=YES ---CPU TIME % LOG PROC NUM TYPE ONLINE LPAR BUSY MVS BUSY PARKED SHARE % 0 CP HIGH 1 CP HIGH 2 CP HIGH 3 CP HIGH 4 CP MED 5 CP MED 6 CP LOW 7 CP LOW 8 CP LOW 9 CP LOW A CP LOW B CP LOW C CP LOW D CP LOW E CP LOW F CP LOW 10 CP LOW TOTAL/AVERAGE CEC Busy = * 19 CP =.22 CPs available Weight: 5.32 CPs Using: 42.47/100 * 17 LCP = 7.22 CPs 19

20 Understanding the Numbers CEC is 98.85% busy LCP LOG PROC POLARITY UNPARKED LPAR UNPARKED ONLINE LPAR BUSY MVS BUSY PARKED SHARE % CPs MVS BUSY EFF HIGH HIGH HIGH HIGH MED MED LOW LOW LOW LOW LOW LOW LOW LOW LOW LOW LOW Unparked Effective Percent of time dispatched by LPAR when not parked (LPAR Busy / 100) 100 Parked Time /

21 CPU to Dispatch Ratio Interval CP 0 CP 1 CP 2 CP 3 1 CICS,STC,Batch,Batch CICS L=P BATCH L=P STC L=P BATCH L=P 2 CICS,STC,Batch CICS L BATCH L=P STC L=P 0 3 CICS,Batch,Batch,Batch CICS L=P BATCH L=P BATCH L=P BATCH L=P 4 CICS CICS L CICS Active 4:4 = 100% CICS Dispatched 2:4 = 50% LPAR BUSY 10:16 = 63% MVS BUSY 12:16 = 75% 21

22 Understanding the Numbers Next Interval CEC is 97.85% busy LCP LOG PROC POLARITY UNPARKED LPAR UNPARKED ONLINE LPAR BUSY MVS BUSY PARKED SHARE % CPs MVS BUSY EFF HIGH HIGH HIGH HIGH MED MED LOW LOW LOW LOW LOW LOW LOW LOW LOW LOW LOW MVS Busy: Online Time - (Wait Time + Parked Time) Online Time - Parked Time LPAR MVS Busy: Unparked Time * (LCP MVS BUSY) Unparked CPs) 22

23 Hiperdispatch Summary Important to ensure LPAR weights are close to actual LPAR usage Drives better allocation of Vertical Highs Still be realistic in number of logical CPs assigned to an LPAR e.g. if using 7.7 LCPs at max specify 9-11 not 17 LPAR MVS Busy is key metric driving unparking Calculate Unparked Effectiveness and evaluate workload delays Impacts are very workload specific Check CPU to Dispatch ratios Latent Demand indicators now need to include knowledge of: Parked CPs over time Unparked Effectiveness Watch LPAR weights for small LPARs with low utilization Weight = 1.98 CPs then 1 VH, 1 VM (2 LCPs) Change Weight to: 2.01 then 1 VH, 2 VM (3 LCPs) 23

24 Don t get Button Happy in zpcr zpcr provides a lot of productivity enhancements Automatically reading in your RMF data Adjusting logical CPs Make sure the changes requested are representative of the environment If not, then need to adjust the zpcr study Example: RMF partition data report shows the intended LPAR configuration but actual usage can differ from intended IRD, Hiperdispatch Use of Whitespace 24

25 LPAR Share ATS - Washington Systems Center % Share WSC1 WSC2 Share Hour 25

26 Typical Partition Data Report Used in zpcr PARTITION DATA LOGICAL P ----MSU---- -CAPPING-- PROCESSOR- NAME S WGT DEF ACT DEF WLM% NUM TYPE WSC1 A NO CP WSC2 A NO CP WSC3 A NO CP WSC4 A NO CP *PHYSICAL* TOTAL -- AVERAGE PROCESSOR UTILIZATION PERCENTAGES -- LOGICAL PROCESSORS --- PHYSICAL PROCESSORS --- EFFECTIVE TOTAL LPAR MGMT EFFECTIVE TOTAL LPAR definitions state the WSC1 LPAR gets 55% of the environment Overachieving the weight this needs to be reflected in zpcr 26

27 Get the right capacity relationships Unused LPARs should be unchecked Change causes the Partitions Weight in zpcr to change This indicates WSC1 gets 64.71% of the CEC 27

28 Get the right capacity relationships With Hiperdispatch the logicals in RMF PDR report don t match what is really running This LPAR gets.6471 * 10 = 6.47 CPs or 5 VH, 2 VM, 3 VL At capacity the LPAR would run with 7 LCPs, not 10 28

29 Get the right capacity relationships But it is really using 83% of the environment so it has more LCPs running RMF report would give 9739 MIPS but actual configuration gives 10,452 MIPS 29

30 New Version of zpcr will Provide Assistance 30

31 When Determining Capacity When using automated input be sure to adjust zpcr weights and number of logical CPs to reflect actual capacity Want to designate a zpcr model which gives the best view of typical peak capacity Want to see the capacity available at contention given most likely conditions 31

32 CPU Utilization ATS - Washington Systems Center Processor is a with 3 LPARs but is running only 50% busy zpcr Multi-Image Table places this at 17,171 MIPS, or 859 MIPS per CP Processor is actually running faster than this Utilization A u g 1 5 A u g 1 6 A u g H o u r 32

33 CPU Utilization ATS - Washington Systems Center Impact to capacity planning comes in two flavors May have less headroom on processor than expected When moving a workload, it may not fit in the new container Example Assume a workload is running at 50% busy on a 2000 MIPS box without factoring in utilization effect, it will be called a 1000 MIPS workload in fact, it may be an 1100 MIPS workload when running at the efficiency of a 90% busy box Caution #1: There is NOT room to double this workload on the current box Caution #2: If moved to a new box or LPAR, it will likely need a 1100 MIPS container (not 1000 MIPS) to fit ROT: CPU per tran will vary 3-5% for every 10% change in utilization 33

34 How to Handle ATS - Washington Systems Center Build zpcr model for current processor, e.g. z Per CP speed is: / 20 = 931 MIPS But running 50% busy the CEC really looks like.5 * 20 = 10 CPs /.9 = 11 CPs 34

35 How to Handle ATS - Washington Systems Center Change the host to be a z adjusting the CPs as needed for the smaller n- way 711 delivers 11,104 MIPS / 11 = 1009 MIPS Actual capacity being delivered is more like 1009 MIPS CPs not 931 MIPS per CP 35

36 Impact of Specialty CPs on GCP Capacity Slight impact on capacity when running multiple books and specialty CPs Busy of the specialty CPs will govern the extent of the impact zpcr assumes the specialty CPs are fully utilized (90%) and so gives a conservative view of capacity **** Environment MSU GCP MIPS PER CP ziip MIPS TOTAL 2817 M15 with 4 GCPs M15 with 8 GCPs M15 with 4 GCPs, 4 ziips Per CP Speed MIPS Per MSU GCP 15 8GCP 4GCP, 4ZIIP GCP 8GCP 4GCP, 4ZIIP 36

37 ziip Busy Impacts on Capacity zpcr estimate assumes the ziip is 90% busy Impacts of IIPHONORPRIORITY limits ziip busy Workload eligibility requirements limits ziip busy Environment MSU GCP MIPS PER CP ziip MIPS TOTAL 2817 M15 with 4 GCPs M15 with 8 GCPs E12 with 4 GCPs, 4 ziips Difference between 4 GCP + 0 ziip and 4 GCP + 4 ziip is 359 MIPS ziip is 30% busy, true impact to GCP is 359 *.3 = 108 MIPS So GCP MIPS are = 4562 MIPS 37

38 Specialty CP Capacity Planning Mistake Need to look across time for zxxp eligible work, and understand interval which drives need for processor capacity Make sure the peak which drives capacity can use the zxxp CPs to reduce costs CEC Capacity Potential zxxp Capacity Rolling 4hr zxxp CPU

39 Summary ATS - Washington Systems Center Ensure period used for capacity analysis reflects representative time periods Ensure WLM policy is prioritizing work correctly Eliminate events which impact hardware or software capacity Use LSPR (zpcr) when calculating capacity relationships Don t use MSUs, or Service Units Always ensure a single LSPR table is used when building capacity Understand Hiperdispatch latent demand indicators Unparked Effectiveness and Number of Parked CPs over time Check CPU:Dispatch ratios and adjust weights or config off logicals if there are delays Understand the capacity aspects of parallel sysplex Understand impacts of Low Utilization on CPU per tran Ensure zpcr models reflect actual capacity and not defined capacity Understand impacts of Specialty CPs on GCP capacity 39

40 z/os Capture Ratio CPU time used by the system to do processing which cannot be related to a specific user ƒcapture ratios in z/os have improved over time ƒ88-95% capture ratios are "common" ƒindicator of overall system health Calculation Capture Ratio = ( Service Class APPL%) / # Logical CPs LPAR Busy Should be a concern if capture ratio varies widely across time 40

41 Capture Ratio Data Sources SERVICE POLICY -TRANSACTIONS- TRANS-TIME HHH.MM.SS.TTT --DASD I/O-- ---SERVICE--- SERVICE TIME ---APPL %--- --PROMOTED STORAGE---- AVG ACTUAL SSCHRT 8208 IOC 38141K CPU CP BLK AVG MPL EXECUTION RESP 8.8 CPU K SRB AAPCP 0.00 ENQ TOTAL ENDED QUEUED 867 CONN 6.4 MSO 0 RCT IIPCP CRM SHARED END/S R/S AFFIN DISC 0.1 SRB 21293K IIT LCK #SWAPS 2796 INELIGIBLE 198 Q+PEND 2.3 TOT K HST AAP N/A -PAGE-IN RATES- EXCTD 0 CONVERSION 1 IOSQ 0.0 /SEC AAP N/A IIP 0.00 SINGLE 0.0 AVG ENC STD DEV IIP BLOCK 0.0 REM ENC 0.00 ABSRPTN 950 SHARED 0.0 MS ENC 0.00 TRX SERV 950 HSP 0.0 CPU 2094 CPC CAPACITY N/A MODEL 712 CHANGE REASON=N/A H/W MODEL S38 ---CPU TIME % NUM TYPE ONLINE LPAR BUSY MVS BU 0 CP CP CP CP CP CP CP CP CP CP A CP B CP TOTAL/AVERAGE RMF Monitor 1 ƒrmf CPU Activity Report and use LPAR Busy from CPU Activity ƒuse RMF Workload Activity with control card SYSRPTS(WLMGL(POLICY)) and get a single report per interval /100 = 7.38 CPs /100 * 12 CPs = 7.72 CPs Capture Ratio = 7.38 / 7.72 = 96% 41

42 Subcapacity GCPs and Specialty CPs Specialty CPs always run at full speed of processor model Same z/os image has CPs running at different speeds Requires CPU seconds to be normalized Example: zaap is 8 times the speed of the GCP Execution Time = GCP seconds + (zaap seconds * normalization factor) Execution Time = 1 second + (1 second * (8)) = 8 seconds = 9 seconds GCP 1 Sec zaap 1 Sec Normalization factor used is in RMF 72 subtype 3 record, R723NFFI (zaap) or R791NFFS (ziip) Normalization factor used is in the SMF 30 record, SMF30ZNF (zaap) or SMF30SNF (ziip) When zaap/ziip and GCP are the same speed the normalization factor resolves to 1 42