CUSTOMER FOCUSED BUSINESS METRICS THROUGHOUT THE SDLC

Transcription

1 BIO PRESENTATION W6 September 29, :00 PM CUSTOMER FOCUSED BUSINESS METRICS THROUGHOUT THE SDLC Steve Wrenn Liberty Mutual Insurance Information Systems Better Software Conference & EXPO September 27-30, 2004 San Jose, CA USA

2 STEPHEN WRENN Stephen Wrenn is currently the Sr. Director of IT Service & Quality Assurance at Liberty Mutual Insurance Group. There he is responsible for bringing people, process and technology together to solve insurance business problems on a global basis. Prior to Liberty Mutual, Stephen was the VP of Global Service & Support for Enterasys Networks, where he led a team of service professionals in delivering industry leading network quality initiatives, and global expansion of Enterasys professional services. Stephen has over 20 years experience in IT company s, holding leadership positions at Hewlett-Packard, Cabletron, and Cisco Systems. He has built technical support organizations in both the US and abroad, most recently encompassing offices in London, Singapore and Hong Kong. Stephen holds a B.S. in Electrical Engineering from the University of Massachusetts and an MBA from the Whittemore School of Business at the University of New Hampshire.

3 Customer Focused Business Metrics Throughout the SDLC Better Software Conference & Expo 04 Measuring and Reporting Track Steve Wrenn, Director IT Service & Quality Assurance Liberty Mutual Insurance Group, Information Systems

4 Introduction A quick overview of Liberty Mutual IS and it s culture. What we set out to do. Our process or journey Constraints SDLC vs. PLC Quality Framework Development Focus Outcomes What we learned

5 Where We Were at Liberty First of all you have to understand a few things about Liberty Mutual Insurance Business is inherently Risk Adverse Private Company with, until recently, mostly internal customer s for IT Currently profitable in a very stable and long-life business After that very Much Like a Lot of IT Development Shops Not enough time to do everything Lack of controls after all, Coding is an art right?

6 What We Set Out To do Run IT Like a Business. Increase Level of Service to our customers by delivering high quality products and services on time and on budget. Get it right the first time One and Done!

7 Our Process to Run IT Like A business First decide where we want to go. Strategic objectives Business objectives Competitive advantage Application Development Focus Getting the Voice of the Customer (VOC). Who s the customer? What do customers want/need Operational Focus TQM, Six-Sigma, CMMI Level?

8 Customer Focus Faster CUSTOMER DELIGHT! Better Cheaper Creativity, Flexibility and Responsiveness are the Keys!

9 Our Process.continued Then evaluate where we currently stand. (Nothing new here, a simple GAP analysis should suffice nicely yeah right!) Baseline metrics.what metrics? Our work is so unique you can t measure it Evaluation of current processes Things are working fine, we re making money, why do we want to change things Processes were all different, so any metrics collected could not be compared. All hard to do without knowing current output (systems thinking). We finally settled on quality control data gathering for 6 months.

10 Our Constraints The old project management triple constraint of scope, schedule and budget, and also a mindset that said get it done on time, at all costs. What do you think was the main cause of this?

11 It All Starts With Requirements Or does it?

12 SDLC vs. PLC Mindset The software development life cycle of most companies are very well thought out and focused on getting a project done on time, on schedule and on scope. How many of them are actually focused on delivering the customer what they want and need to be successful? How do they know if they are?

13 The PLC Feedback-Loop Think customer input, which in most companies comes from the users of your current products and services. (VOC) How many of you proactively use this information to feedback to your development teams the fruits of their labors, or to design your next product? The past is a pretty good predictor of the future

14 LMIG IT Quality Framework / Methodology Product Life Cycle (PLC) Plan Build Run Retire Software Development Life Cycle (SDLC) Release/Acceptance Pre-Production Production PDCA PDCA PDCA Customer (End-User) Quality Assurance Requirements SQA/Test Plan QA Deliverables Stress Testing Final SLA System Burn-in SLA Monitoring Support Training EOL Strategy B.U. Voice of the Customer (Requests/Concerns) Change Management (For New & Production) Problem Management (Restore & Eliminate) Tools to enable Effectiveness & Efficiency Metrics for Success and Adjustments (Targets)

15 Our Development Focus Plan and Build Define, Design, Develop & Test Before you jump into this, there is a little education that is extremely valuable. How do you keep the team focused on getting the customer s what they need to be successful?

16 A Focused Development Team Help them start to think like business managers Take the time to explain to them what the fruits of their labors are. They are not developing code, they are building pieces to a product that fit into a system or systems that somehow make money for the company! Show them how their applications contribute, they may surprise you with their ideas here. Show them how quality, both high and low, effects the outcome

17 From the Quality/Test Perspective Worked with the development team to put metrics in place that constantly let them know how they and their processes are doing. Plan, Do, Check, Act.make it self-managing by building a metrics program or dashboard Closed-loop, systems thinking and training There can be a lot of cultural shock here.

18 Liberty IT Continuous Process Improvement People Process Optimum Productivity! Technology Knowledge Sharing, Customer Focus and Leadership are Key!

19 Our Outcomes We developed a quality process and measurement program that collected few, but very specific measurements at each stage of our development cycle. We improved our first release quality by 40% in the first 6 months of our initiative How did we do it?

20 Project Name: Project Code: Scope Definition Requirements Definition Requirements Definition LMGIT - SDLC Key Quality Deliverables - Worksheet Analysis Architecture && Design IT Project Mgr : Bus Project Mgr : Construction & Test Implementation PMO Required Request for Project Funding Report Create Project Charter * Create Initial IT Project Plan (006) with Key Milestones* PMWB Forecast Risk /Issues Log Change Management Process QA Best Practices BISAT/ Opportunity Evaluation (100)* /** Create Initial SQA Plan* Review with Technology Mgt Board (TMB)* Review with Enterprise Portfolio Review Board (EPRB) * Obtain Key Stakeholder Decision for Go / No Go for proceeding with Bus. Requirements * Statement of Work / Contract (if applicable) Sarbanes Oxley * Key Stakeholder Sign-Offs ** (NNN) Refers to the LMSDP Process Artifacts PMO Required First cut SLA Performance * Create Initial Test Strategy Document QA Best Practices Business Alternatives (125)* Bus Objectives (130)* Proposed Workflow (132)* System Context Diagram - To Be (140)* Requirements Management Plan Glossary of Terms QA Best Practices Updated Updated SQA Plan Updated IT Project Plan PMO Required Freeze Detailed Business Requirements (135s) - OO Projects = Use Case Scenarios, Class Diagrams, Activity Diagrams, etc. * (all changes now go through the Change Management Process)*** Technical Architecture Preliminary Review Report * Logical Data Model * Completed Disaster Recovery Requirements * QA Best Practices Security Requirements * System Interface Requirements * Create Test Plan Create Test Cases - Functional, System, Integration, & UAT Baseline Schedule/ Budget Business Rules Data Conversion Rules Archive Purge Requirements Updated Glossary of Terms QA Best Practices Updated Updated / Test Strategy - Finalized * Baselined IT Project Plan PMO Required Functional Specs (490)* Infrastructure Recovery Requirements (IRR) - 1 st Cut QA Best Practices Technical Alternatives (260) Detailed Technical & Architec. Design Review * User Interface Specifications System Storyboard/ Flow (250/255) HTML / Word Page Mock- Ups (495) Non Functional / Supplementary Specs - Performance Reqmnts, Archive /Purge, Security, Disaster Recovery, etc. OO / Baselined Object Model Finalizd Funct l Data Model * Desktop QA Plan * Program Specs (570) End User Training Strategy Implementation Strategy QA Best Practices Updated Finalized Test Cases - Functional, Systems Integration, and UAT Finalized Test Plan * Updated Project Plan PMO Required User Acceptance Testing QA Best Practices Source Code Mgt Plan/ Configuration Mgt Service Center Change Ticket Define/Create Testing Environments Software Code Created Conduct Peer Code Reviews Unit Testing Functional Testing Systems Integrated Testing Performance Testing Volume Testing Stress Testing Desktop QA Certification Notification UAT Testing Sign-Off * / **** Risk Assessment to Prod* Software Code Frozen * (all coding changes now go thru Change Mgt Process) Baselined Phys Data Model Test Communication Plan for Teams/ Management QA Best Practices Updated Finalized IRR Document * Finalized SLA * Updated IT Project Plan PMO Required Implementation Plan Including Implementation Checklist (back out plan and Signoff (800) Lessons Learned and Action Plan Document (060) QA Best Practices Resolved Requirements Change Management Issue Production Support Plans* On-line Help System: User Guide (700); Help Desk (701); Training (710); Installation Instructions (720) Release Notes - Business End User and IS Release Notes Perform Post Implementation Verification Testing Obtain Client Sign Off for Go/ No Go to deploy to Production * IT - QA Sign Off per SQA Plan Monitor System Performance for 90 Days Published Re-Usable Assets in Component Re-Use System Maintenance Support Plans *** Date Business Requirements Frozen: / / Proj Mgr: QA Mgr LMGIT - SDLC - Key Quality Deliverables Worksheet - V1.03 Project Management IT Project Plan PMWB (Project Managers Workbench) Risk/ Issues Log Change Management Process/ Log (Mantis) -> ServiceCenter SQA Plan **** Date User Acceptance Testing Completed : / / Proj Mgr QA Mgr David H. Maurer / June 15, 2004

21 SDLC Quality Gates / Metrics Initiation / Scope Definition Instituted a rigorous project qualifying review ROI or competitive advantage Strategic fit or else! Requirements Definition Do we know what this requirement is for? Has it been verified for testability and signed of by the Test or QA manager? Have the Requirements been signed off by all parties?

22 Requirements Definition Customer Needs / Wants Analysis

23 Requirements Definition Quality Requirements 5 Major Issues Preventing High Quality Requirements (and thus setting the project up for failure from the start) Requirements not complete Requirements not testable (to gray) Requirements developed without end-user input Requirements thought to be non-negotiable Requirements not signed of by all parties.

24 SDLC Quality Gates / Metrics Analysis Test plans completed Requirements freeze in place (change mgt) Architecture and Design Test cases completed (100%) Requirements Freeze in place

25 SDLC Quality Metrics Gates Construction and Test Changes in this stage require higher level of sign-off SLA developed Implementation SLA signed-off by all parties 30, 60, 90 day monitoring of quality Project estimation chart completed

26 What We Learned Culture change is more difficult than any technical challenge you will encounter The more time spent in planning the less time spent in other phases or An ounce of planning is worth a pound of development. Signatures are very powerful in setting and keeping expectations We very often take too much for granted when it comes to business / technical understanding. Metrics need to be SMART

27 Liberty IT Monthly or Quarterly Report Card COMPLETE ON-TARGET CONCERNED FY 04 Q1 Deliverables Strategy #1: Engage proactively and regularly with customers, partners and sales organizations, and technology development stakeholders, to ensure delivery of consistent, high quality, and relevant service and support worldwide. >/= 4.6 out of 5.0 on WW customer satisfaction survey >15% Increase in Productivity <99% of budgeted expenses Strategy #2: Grow current service offerings in order to enable the business groups to deliver better service to their customers, and thus further differentiate the total Enterasys value proposition from the competition. >110% WW Bookings targets (15% growth) >65% Contribution Margin (cost down 10%, rev up 15%) Strategy #3: Develop and deliver new services that add value to our products and solutions, and differentiate them in the marketplace. >50% of services developed in last 3 years Key Monthly Accomplishments Measurable accomplishments towards your business plans Productivity, costs, customer satisfaction, etc. Examples. Completed Design Stage on project XX % of projects in development that are on time Change management measures Quality before release measures Quality in production measures Status FY'01 Q1 $50,000 $45,000 $40,000 $35,000 $30,000 $25,000 $20,000 $15,000 $10,000 $5,000 $0 Customer Satisfaction Customer Satisfaction Rating Rating US Canada LA EMEA AP Q2 Q3 Q4 FY'02 Q1 Bkgs Actual F'cst Q2 Fcst Q3 Fcst FY'01 Q1 Q2 Q3 Q4 FY'02 Q1 Q2 Q3 Q4 Fcst Q4

28 SDLC Quality Gates / Metrics Data vs. Information Data is IMPORTANT! Without Data....Opinion Prevails Data + Analysis = Intelligent Information Data by itself is the library without a librarian.

29 The Importance of Proper Measurement Positive or Negative?

30 The Importance of Proper Measurement Positive or Negative?

31 The Importance of Proper Measurement Positive or Negative?

32 LMIG IT Quality Framework / Methodology Thoughts Don t Tell Me Prove It! Close-The-Loop Systems Thinking Product Life Cycle (PLC) Plan Build Run Retire Software Development Life Cycle (SDLC) Release/Acceptance Pre-Production Production PDCA PDCA PDCA Customer (End-User) Quality Assurance Requirements SQA/Test Plan QA Deliverables Stress Testing Final SLA System Burn-in SLA Monitoring Support Training EOL Strategy B.U. Voice of the Customer (Requests/Concerns) Change Management (For New & Production Apps) Problem Management (Restore & Eliminate) Tools to enable Effectiveness & Efficiency Metrics for Success and Adjustments (Targets)