Cost of Poor Quality: Analysis for IT Service Management Software Software Concurrent Session: ISE 09 Wed. May 23, 8:00 AM 9:00 AM Presenter: Daniel Zrymiak
Key Points: Use the Cost of Poor Quality: Failure Dimensions to address and resolve problems originating from an inadequate IT Service Management application. Define Factors and Dimensions of Failure by Type Define Factors and Dimensions by Correction Type Define Failures by Phase Track specified outcomes for Continual Improvement Show the Cost-Benefit advantages of improvements
Presentation Benefits (1 of 2): Learning Objectives: Demonstrate how a detailed classification of Failure Costs and Resolution Methods can be applied to support the development and maintenance of IT Service Management software. Can you think of one example of a Failure Cost or Cost of Poor Quality related to the development, deployment, and maintenance of a software system? How would this example be categorized?
Presentation Benefits (2 of 2): Career Benefits: Align quality initiatives to financial realities and priorities Expand analysis to include financial tracking of failures and corrections for future reference Technical Content: Software Quality Engineering/Management, Lean Six Sigma Professional Development: Cost of Poor Quality is pertinent for Managers of Quality and Organizational Excellence, also Lean Six Sigma Black Belts. Automation software is relevant for Testing, IT Governance.
Scenario: Deficient IT application Canadian-based Energy company used an IT Service Management application from an external vendor. Development, training, and deployment did not occur as scheduled causing schedule delays and cost overruns. Eventual implementation and go-live was characterized by frequent outages, user dissatisfaction, and interruption of regular business processes. Transition budget was depleted before the application was stabilized.
Factors and Dimensions Factors: Broad categories or segments to address the failures at a more granular level Dimensions: Elements of analysis within a factor to determine appropriate resolution and corrective action
Factors of Failure Costs Failure Type: People, Product, Process, Transaction, General Correction Type: Failure Phase: Resolution, Restoration Expectations, Commitment, Realization, Delivery, Support
People (Capability vs. Engagement) For failures related to people performing the work, two dimensions should be considered. Capability refers to the qualifications and general competency. Engagement considers the overall motivation and willingness to perform the work to the required standard. Due to inadequate training and communication, the individuals using the tool were not adequately capable nor engaged to use the tool correctly.
Product (Meeting Specs vs. Fitness For Use) For failures related to products, two dimensions should be considered. Meeting Specifications can be demonstrated by objective verifications. Fitness For Use can be observed through subjective validations (i.e. look and feel) The tool was ITIL compliant and demonstrated correct functionality in controlled tests, but was not robust to handle normal volumes, nor could it be adapted to fit the client s normal business processes or expected workflows.
Transaction (Accuracy vs. Timeliness) For failures related to transactions, two dimensions should be considered. Transactions can fail at the point of interaction or discovery can occur later through reporting. Accuracy can relate to the correctness of quantity, suitability, and thoroughness. Timeliness refers to tracking delays and lateness of the deliverable. The tool was difficult to configure, and configuration problems affected accuracy. Timeliness was impaired by performance problems, causing delays to handle requests.
Process (Effectiveness vs. Adherence) For failures related to processes, two dimensions should be considered. Process failures can lead to subsequent failures in other areas. Effectiveness determines whether the process was suitable and applicable. Adherence analysis reveals points where a valid process is being disregarded in favor of ad-hoc or tailored solutions. The tool had a distinct workflow which was different from the client s prior patterns, requiring adjustment. To bypass restrictions, tailoring and shortcuts were required.
General (Explicit vs. Implicit) For failures that do not align with a single category, a general analysis should occur. Explicit failures reveal shortcomings relative to an approved standard or benchmark. Implicit failures diverge from expected outcomes and contribute more significantly to overall dissatisfaction. There was a subjective sense of frustration and disappointment with the tool, which diminished the overall effectiveness of the transition.
Cost of Poor Quality Measurements People: Losses are measured in hours and days delayed per person, and Escalation/Specialist resource time and materials to resolve. Product: Losses are measured in the variable costs of system impairments and downtime arising from the deficiencies. Process: Losses are measured in the frequency of redundant or abandoned processes needed to complete transactions. General: There were indirect outcomes assignable to the low quality tools including team member attrition and project deferral due to lack of confidence with the tool.
Factors of Failure Costs Failure Type: People, Product, Process, Transaction, General Correction Type: Failure Phase: Resolution, Restoration Expectations, Commitment, Realization, Delivery, Support
Resolution (Contingency vs. Correction) All failures must be resolved to move from a failed error state to a stabilized recovery. Contingency programs involve different initiatives to compensate for the error condition. Corrective actions require the root causes to be identified and resolved. As a contingency, departments revert to tracking information on their MS Office applications and updating by email. To correct the problems, technical teams investigated based on priorities reported by the post-go Live Command Center.
Restoration (Mitigation vs. Accommodation) Following the interruption and loss arising from failures, efforts are needed to restore the trust and confidence in the relationship. Mitigation includes collaboration through awareness and vigilance, and applied controls and standards to entities. Accommodation refers to the concessions and compensations to address losses and penalties. Mitigation for future releases could be established through a more rigorous approval and release process (on next slide). Accommodation was made through financial concessions.
Establish Mandatory Release Criteria Given the error conditions realized from a poor IT Service Management application, releases to client production systems should be restricted until objectively proven adequate in the following categories of system assurance: PRODUCT SYSTEM COMPLIANCE Functionality Availability Security Reliability Configurability Environmental Controls (i.e. SOX) Usability Performance Verification and Validation Assurance Maintainability Efficiency Data Privacy
Cost of Correction Measurements Contingency: These costs are measured in hours and days of just in case plans to continue in the event of system failure or data corruption. Corrective Actions: These costs reflected the rework, redesign, and incremental variable costs associated with maintenance and support releases and retraining of personnel. Mitigation: These costs reflected the additional design and process steps needed in response to identified risks and hazards of software failure. Accommodations: These costs include the various penalties and concessions which were needed to comply with contractual service level expectations and preserve the business relationship.
Factors of Failure Costs Failure Type: People, Product, Process, Transaction, General Correction Type: Failure Phase: Resolution, Restoration Expectations, Commitment, Realization, Delivery, Maintenance and Support
Need to Distinguish Failures by Phase CoPQ programs experience a disproportionate clustering of failures within the client-facing areas of the organization. The true source and timing of failures actually occurred earlier in the business cycle. Distinguishing failures by phase increases transparency and accountability across the entire enterprise. Early-stage failures can affect acceptance at later stages.
Phase 1: Expectations Prior to engagement, the organization establishes expectations as the basis for comparisons and standards (i.e. boutique shop vs. factory outlet merchandise). Failure points are established by the overall impression of quality as communicated by marketing and positioning strategies. The vendor promoted itself as a reliable provider of IT Service Management applications, and sold itself being capable of delivering a quality product.
Phase 2: Commitment and Planning In this phase, specific promises and delivery terms are established. Based on these agreements, the details for fulfillment are created. Fine print needs to be carefully reviewed to avoid potential problems. Risks and contingency scenarios should be considered in anticipation of probable potential failures. When committing and planning for the development and deployment of the IT Service Management tool, there were inadequate resources applied to testing, training, and deployment, creating opportunities for failures.
Phase 3: Realization and Fulfillment In this phase, actual work is performed to bring the solution from concept to readiness. Depending on the complexity, this is the phase where most failures are injected. Without the necessary controls, failures from this phase can propagate and taint the overall solution. With respect to the IT Service Management tool, inadequate configurations were pushed to the client system by the vendor, without having had the key assurance attributes (i.e. reliability, usability, system performance, etc.) properly verified and validated. False Green indicators were given.
Phase 4: Delivery In this phase, the product, service, or solution is provided to the customer or stakeholder. This is where evident failures could be rejected, causing delays and dissatisfaction. This is the go-live portion where failures combine technical internal failure characteristics and supplemental external failure business losses. With the vendor, there was inadequate preparation for the go-live, and the resulting change to the business was met with surprise and frustration. Damage control was needed with the involvement of senior technical expertise.
Phase 5: Maintenance and Support In this final phase, constant interaction is performed to address the correction, adaptation, or prevention of identified or potential shortcomings. Support can be scheduled or immediate, depending on the scope and importance of the failure. After applying the corrections, organizations should communicate causes and actions through a Failure Tracking System to reduce overall failures. Maintenance and Support are provided by the vendor through a responsive Command Center to expedite fixes.
Continual Improvement through Tracking Failure Analysis includes the categorization and discovery of the modes and effects of different failure types. Failure Tracking links this information to specific actions and an ongoing information system. SLACK model: Summary, Lessons Learned, Action, Commitment, Knowledge Base
Summary Failures can be summarized and categorized by factor and dimension (i.e. People-based, Capability). From a simple description, quick analysis and rapid responses can be consistently supported across the organization. Common failure identifications can support trend analysis and statistical process controls. The IT Service Management tool failures reflected failures across all categories (i.e. people, product, process) and in almost every dimension (i.e. capability, engagement).
Lessons Learned This instructive portion captures the discovery of the information while it is current and available. Insights on the impacts and downstream effects of particular failures will prompt the establishment and application of appropriate controls and standards. Real-life examples will reinforce training and awareness, and leverage common experiences across the enterprise. The lessons learned from the deficient IT Service Management tools are shown on the next slide.
Lessons Learned People: Ensure intended users are proficient and engaged with the product prior to release, and are positive advocates. Product: Restrict releases until they have met the mandatory assurance criteria for Product, System and Compliance. Process: Align the IT Service Management tools with the established processes agreed by client, business, and stakeholders; preventing workarounds and ad hoc solutions. General: Forcing a deficient application upon a client and delivery team will deplete time and resources, causing losses. It is better to stabilize the system prior to release.
Action The correction is outlined by factor and dimension (i.e.: Resolution, Contingency). By recording the actions, this provides a ready reference to support recurrence and rapid turnaround of corrections. There are many actions required to correct and mitigate the risks arising from the deployment of a deficient IT Service Management System. These should be noted in order to be consistently leveraged and replicated across different engagements.
Commitment Every action requires a commitment which includes the resource owning the solution and the completion date. Tracking will increase transparency and accurate computation of costs for failure response. CoPQ can be approximated and used as a baseline for cost reduction initiatives including Variance Analysis, Activity Based Costing, and Lean Six Sigma programs. Compressing transition costs comes from increasing preparations and preventive measures, increasing levels of assurance, and controlling the deployment of applications.
Knowledge Base All failure information should be aggregated into a common knowledge base. Objective information can be used as inputs for verification, validation and future planning activities. Build a continual learning organization which updates knowledge for continual improvement. Knowledge of IT Service Management applications currently in use can be captured and made available for future reference.
Benefits to CoPQ Programs Increased accuracy of failure tracking and diagnosis. Improved application of corrective methods to target nature and timing of failure type. Diligent tracking of failures and response activities will generate more accurate cost information. Combination of approaches will objectively reduce Cost of Poor Quality and reduce turnaround time for solutions.
Cost-Benefit Rationale for Improvements False savings to reduce costs by streamlining development and deployment of IT Service Management applications. Actual Experience from Canadian Energy company demonstrates that a deficient solution will create delays and disruptions, requiring rework from technical resources, and deplete transition budgets without client satisfaction. Incremental improvements to enhance assurance, user preparations, and release controls will result in a more correct and usable solution, reducing the need for expensive restoration and stabilization, saving time and money.
Learning Confirmation: Demonstrate Demonstrate how a detailed classification of Failure Costs and Resolution Methods can be applied to support the development and maintenance of IT Service Management software. Can you now apply Cost of Poor Quality analysis methods, using factors and dimensions, to demonstrate the expected benefits and mitigations resulting from adopting increased quality approaches?
Presentation Business Context What is the business context and how does it improve upon the existing knowledge base? This presentation extends the analysis of Failure Costs by defining categories of failure and resolution, and specifying phases for appropriate correction.
Presentation Practical Applications What are the practical applications towards the participants jobs, functions, projects, or departments? This presentation defined the cost descriptions to support diagnosis and the establishment of an ongoing knowledge base to support more rapid resolutions.
Presentation Integration or Connection How can this knowledge be integrated or connected to other areas? This presentation is tied to overall software quality management and continual improvement. Lean and Six Sigma practitioners will benefit from obtaining the cost measures and deriving the value streams for software.
Presentation Business Context What are the expected gains, and the risks and implications of omission? This presentation provides awareness of potential software quality problems and a structured approach to correction, with the risks being expanded losses and Cost of Poor Quality from undesirable outcomes.
Comments/ Questions: Daniel Zrymiak Accenture: Mobilization/ Outsourcing Excellence dzss@shaw.ca daniel.zrymiak@accenture.com (604) 575-3269 or (604) 862-4808 16721 62A Ave., Surrey, B.C., Canada