Capacity Management Improvement - From Assessment to Benefits

Transcription

1 Capacity Improvement - From Assessment to Benefits Neil McMenemy, Principal Consultant, Capacitas Agenda Reasons for engagement Structure of the improvement program Progress to date Capability of team Further benefits and how they will be measured Next Steps Summary 2

2 Reason for engagement The existing staff have no training in formal Capacity techniques There is a gap between the Capacity Planning team s capability and the customers requirements There are no formal relationships with other related disciplines leading to customers requirements not being met There is little or no direct contact with the relevant customers The existing Capacity Planning team have no direct involvement in the application development lifecycle leading to late involvement with new or changed applications and not enough time to: deliver new capacity plans or forecasts size the changes before live influence performance 3 Reason for engagement The Capacity Planning team spend a lot of time building and maintaining systems and routines to collect system resource data There is only a partial Capacity Database (CDB)/Capacity Information System (CMIS) There are few business forecasts leading to speculative capacity plans The Capacity Planning team do not employ scenario planning or sensitivity analysis to deal with the lack of business forecasts Neither the Capacity Planning team leader, nor any other single person, appears to understand the overall approach to Capacity 4

3 Increasing Complexity More Reactive More Proactive Capacity Maturity Model Tier 5 Tier 4 Tier 3 Forecasting based on a blend of resource util. and demand planning Forecasting based on business demand planning methodology Capacity supporting and driving business decisions Tier 2 Forecasting based on resource utilisation trends Tier 1 Reactive Capacity Tier 0 No Capacity 5 Issues to address 6

4 Structure of the Improvement Program : Scope of deliverables ICT Services Processes Proactive Capacity Planning Reactive Capacity Capacity Planning Processes Modelling Workload Characterisation Capacity Plan Demand Forecast Intelligent Growth Threshold Alerting Capacity Reporting Business Capacity Service Capacity Demand Ministry of Justice Connecting for Health Network Rail Performance Testing Interface VOSA Demand Forecasting Processes Demand Forecasting UK Borders Agency First Tier Services Service A Service B Staff National Farmers Union Training & Mentoring Second Tier Services Service C Service D Formal Capacity Planning Skills Other Customers 7 Structure of the Improvement Program : Deliverables Phase I Training: Introduction to Capacity Capacity Process/Standard Approach Monitoring Tools Review & Recommendation Report Centralised Data Storage Advice & Assistance with Automation Pilot Project to Demonstrate Approach Phase II Integration of Capacity Process into Dev. Lifecycle Performance Testing Strategy Training: Tools & Methods Training: Capacity & Perf. Mgt. using MS-Excel: Analysis & Graphing Solutions Post Implementation Capacity Service Audit Refresh 8

5 In parallel Alongside the improvement programme we were engaged in separate pieces of work This was due to the customer s temporary lack of resources and skills Involved Capacitas taking our own medicine Following process Building capacity and performance models Maintaining CSIPs 9 Intermediate Progress Team Organisation Application Lifecycle Automation Capacity Database Scope Position in Organisation Capacity Planning Process Deliverables Configuration CPU I/O Memory Performance Testing Disk Space Sensitivity Analysis and Network Response Time Analysis Common Data Analysis Business Service Objectives Business Volume Business Volume Forecasts Service Before During 10

6 Capability of team Objective measurement Tools and methods examination Pass mark 26/40 Overall 9 candidates/6 passes/3 failures Rate of 1 failure per 3 sittings c.f. previous rate of 1 failure per 14 sittings 11 Capability of team Subjective opinion Technical potential is good Existing technical experience is below average Existing process experience is poor Little demonstration of leadership 12

7 Further benefits Many benefits won t be realised until the team actively put in place what they have learned For each customer a Continuous Service Improvement Plan was developed (CSIP) The CSIP lists: Actions Targets s Risks Mitigating actions High priority customers are targeted first 13 Example CSIP extract ID Category Target Action Risk Mitigating steps Owner Priority Date for completion Notes 10 Resource monitoring 1 100% 80% Implement monitoring of all relevant Incomplete data for forecasting, capacity Dependent on 2 to ensure scope is fully understood Steve Jordan M platforms and components modelling and planning activities 2 100% 0% Map services to specific hardware Hardware within scope of application not Service delivery managers to confirm mapping of services to hardware Sameena Hassam M 30/04/2010 understood 3 100% 80% Implementation of CMIS Inappropriate/inconsistent data Ensure data is collected centrally and data aggregation/retention policies are defined M Refer to Monitoring aggregation/retention policies Tools and Data Collection Executive Summary 4 100% 90% Define thresholds Capacity/performance exceptions are not Determine thresholds based on SLAs or platform specification H recognised 5 100% 0% Validate forecasts Capacity forecasts are incorrect Build capability to compare resource data against resource forecasts M Dependent on 10 Business and service demand 6 100% 20% Identify key BVIs and FDUs Resource models are not driven by business Discuss business drivers and functional demand units with appropriate personnel H Some low level monitoring demands service metrics such as logins/sec 7 100% 20% Implementation of CMIS CMIS does not contain BVIs Ensure data is collected centrally M 8 100% 0% Validate forecasts Capacity forecasts are incorrect Build capability to compare business and service data against forecasts M Dependent on % 0% Obtain business plans Capacity forecasts are incorrect Use business plans as input into capacity model H Forecasts of business metrics 10 Forecast of capacity 100% 0% Build capacity model Capacity management will not be Identify relationship between BVIs, FDUs and SWVs H pro-active Dependent on 9 requirements Measure service performance % 0% Understand historical performance Service performance is not tracked Collect performance related data and incidents L % 0% Validate forecasts Performance forecasts are incorrect Build capability to compare performance data against performance forecasts L Dependent on 14 Impact of service changes are not understood Performance of specific workloads is not % 0% Obtain test results for key transactions Define and execute performance tests to measure transaction service time and response L understood times Performance degradation is not identified Capacity forecasts are incorrect 14 Forecast of performance 100% 0% Build performance model Capacity management will not be pro-active Build capability to determine response times for key transactions L Dependent on Reduce capacity related 0% Unknown Monitor frequency of capacity Baseline will not be accurate Monitor last-minute capacity-related purchases and reported performance H issues issues issues Capacity issues of new releases Ensure capacity model is kept up to date and major changes are modelled H % 0% Model application changes will not be identified Ensure cost-effective resource FTEs 0.5 FTEs Monitor time spent on manual/reactive No time available for pro-active capacity planning Automate tasks as much as possible H 01/01/2011 provisioning capacity monitoring FTEs 0.0 FTEs Monitor time spent on demand forecasting Cost of capacity plans are too high Automate tasks as much as possible H 01/01/ Formal capacity management 100% 100% Attend appropriate capacity management Capacity management techniques in - - use are not - Complete training training events best-practice 20 Improve customer satisfaction 100% 0% Collect customer opinions Incomplete customer requirements Complete customer satisfaction survey H Awaiting response from Simon Thurlby 14

8 s Time Recording Customer Surveys Self Assessments Process s (KPIs) External Audit More work coming their way 15 Time Recording Target Capacity Team Estimated Time Percentage of Time spent in activity category 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Actual Target Reactive/Manual Neutral Proactive/Automatic 16

9 Prioritised Services Customer Tier Time Recording Baseline Customer Survey Baseline Self- Assessment Baseline CSIP Created Customer 01 1 No No No No Customer 02 1 N/A No No N/A Customer 03 1 No Yes Yes Yes Customer 04 1 Yes No Yes No Customer 05 1 Yes No Yes Yes Customer 06 1 Yes Yes Yes No Customer 07 1 Yes Yes Yes Yes Customer 08 1 Yes Yes No No 71% 50% 63% 43% Customer Tier Time Recording Baseline Customer Survey Baseline Self- Assessment Baseline CSIP Created Customer 09 2 No Yes No No Customer 10 2 Yes No No No Customer 11 2 Yes No Yes No Customer 12 2 No No Yes No Customer 13 2 No No No No Customer 14 2 Yes No No No Customer 15 2 No No No No Customer 16 2 Yes No No No Customer 17 2 Yes No Yes No Customer 18 2 Yes No No No Customer 19 2 Yes Yes No No Customer 20 2 Yes No No No Customer 21 2 Yes No No No Customer 22 2 Yes No No No 71% 15% 21% 0% 17 Example Customer Survey Questions How do you rate the service you receive from Capacity? How do you rate the timeliness of the delivery of the deliverables? Does the reporting received from Capacity meet your/the customer s contractual requirements? How do you rate the accuracy of reports you receive from Capacity? How do you rate the Capacity process documentation? 18

10 Example Process s (KPIs) For each account/service the number of key business and service drivers that are monitored and reported on For each account/service the timeliness of justification and implementation of hardware to meet business demand For each account/service the amount of unused capacity For each account/service the number of problems caused by capacity and performance For each account/service the proportion of time by capacity management staff spent Planning and modelling Fire-fighting For each account/service the cost of preparing capacity plans (and other deliverables) In total, the number of account/services for which the following are produced: Business demand driven capacity plans Feedback of plan v actual data to the business Capacity and performance models throughout the development lifecycle 19 Sell improvement to customers Alongiside the mentoring steps we helped sell the improvements to customers Included a presentation explaining the overall improvement program Specific plans for the customer Also put together an example model (specific to the customer) that did not rely on the customer giving accurate forecasts that did not rely on a consistent relationship between driver and resource but focussed on forecasting and included multiple business scenarios (see example in MS Excel) 20

11 Team Organisation Application Lifecycle Performance Testing Capacity Database Anticipated Progress Automation Position in Organisation Scope Capacity Planning Process Deliverables Configuration CPU I/O Memory Sensitivity Analysis and Disk Space Response Time Analysis Network Common Data Analysis Business Service Objectives Business Volume Business Volume Forecasts Service Before During After 21 Benefits still to come Deliverables More capacity plans and plan versus actual analysis Matches customers needs Business Volume Supports capacity plans and plan v actuals Service Measures service rather than just modelling it Common Data Analysis Allows cross-skilling Supports other activities Sensitivity Analysis and Scenario Planning Measures impact of potential changes 22

12 Next steps Complete integration of process with development lifecycle Complete CSIPs for each customer Complete actions in CSIPs Maintain process measurements and continue to improve 23 Summary It is important to set strategy and have a documented process Baseline measurements are not trivial to take if there has been no tradition of measurement Mentoring and leading by example is a necessary partner to formalised training It is as important to know and document what you don t do so that it can be handled elsewhere Sell to customers (have good example materials) Listen to customers (but their point of view is not the only one) Have a plan to improve and measure it 24