Transformation: Corporate Development and IT Part 5 Quality Management in Large Scale Projects Thomas Gutzwiller July 28, 2009
How do we define project quality? 2
Determinants of project quality the magic triangle Project costs Project time Benefit of design products (scope of benefits) It is normal, that projects cause quality problems, that means that costs, time or benefits are not fully reachable. This must be managed accordingly. 3
What is quality assurance? 4
Basic concepts Categories of quality assurance Constructive quality assurance Constructive quality assurance means, that project process is clearly defined, in terms of A specific project process A general project management method for an overall steering of the projects Analytical quality assurance Analytical quality assurance means review and testing of design products and auditing of the project processes Process quality determines product quality 5
Basic concepts Categories of quality assurance Quality assurance Constructive Analytical Project process model General project management process Test Audit Quality control Quality test 6
First diagnosis in a bad going project When the project team must develop the project process itself or when there are differences about the right procedure... When there are Stars in a project, who manage the project alone... When employees permanently make long hours... then... the project normally doesn t have a defined, accepted and validated project process. With project management discipline alone no project can be managed. There must be additionally a specific, on the problem matched, project process model. Without defined procedure we normally don t reach the objectives. 7
Projects in a two-party contract relationship Often we have situations in major projects, where the management engages external suppliers (e.g. consulting companies or outsourced IT). In this case responsibility between the parties has to be regulated. The ISO 9001 Norm offers an instrument for that. Basically responsibility is defined as follows: Client: Product responsibility is responsible for costs, deadlines and benefits (scope of benefits) Project control on the level of external project management Supplier: Process responsibility is responsible for a state of the art project process, which he is able to handle (ability of repetition) Project control on the level internal project management 8
External and internal project control External project control (client) Planning categories are Milestones per phase (deadlines) documents Ready for user test Total expenditures, total costs of projects Viable benefits (scope of benefits) Risk factors Internal project control (supplier) Planning categories are Project organization Activity planning (per phase) Project documentation, result structuring Available resources (in particular employees, client and supplier) Allocation of employees to activities/results with detailed effort planning (work packages) 9
Client organization Project owner (client) Project committee team (experts) Client project leader Client organization has product responsibility 10
Supplier organization Supplier Supplier project leader Team 1 Team 2... Team n Supplier organization has process responsibility 11
Supplier organization The teams vary depending on project type. Typically the following team types occur in projects: Project office teams Change management teams Analysis teams Design teams Front-end-construction teams Component development teams Cross function teams Method specialist teams Tools specialist teams Data base administration teams Test teams Technical infrastructure teams Acceptance test team Organizational implementation teams 12
Team composition in a two-party contract relationship Involvement of employees out of the client organization Involvement of employees out of the supplier organization Teams: Project office teams Change management teams Analysis teams Design teams Front-end-construction teams Component development teams Cross function teams Method specialist teams Tools specialist teams Data base administration teams Test teams Technical infrastructure teams Acceptance test team Organizational implementation teams 13
Organization of phase sign-off as central element of the two-party contract relationship Projects are spitted in phases. At the phase end the client organization approves the phase results, in order that the next phase can be started under stable conditions. The projects phases are predefined through the project process model. Depending on project type a different project process model is taken and therefore a different phase structure occurs as well. At the phase end there is a review respectively a test (at the end when it comes to information systems). 14
Organization of phase sign-off as central element of the two-party contract relationship Client project leader organizes a review (respectively test) team. The review (respectively test) team (composed of experts which are not involved in the project team) checks the design product and gives recommendation to the project committee. The project committee signs off the phases respectively decides, what has to be changed on the design product. Through this practice it is ensured, that the benefit of the design product is checked by independent experts, which are not involved daily in the project. 15
Organization of phase end Sign off for a strategy project Project setup Vision and mission statement External analysis Gap Analysis Internal analysis Formulation of strategic options Selection of strategic options Implementation planning Implementation controlling 16
Organization of phase end Sign off for a process development project 17
Organization of phase end Sign off for an information system planning project 18
Organization of phase end Sign off for a standard application software implementation project Migration planning To-be. bus. processes To-be activities Parameterization Integration tests Go-live plan Ref. org. concept Parameterization Organization structure Conceptual design Detailed design Interface concept Data migration plan Implementation Fail. planning Org. document. Interface program Data input and migration System documentation Test develop. system /Test 19
Organization of phase end Sign off for a process and systems integration project IT szenario Data reqs. Apps. scenario Applications list. Preliminary study Function reqs. Interface desc.. Workstep list Workflow list Infrastructure plan Task list Implement. plan Tasks chains Task description Work step desc.. Concept Test plan Authorization policy Dialog flow diag. Program design WF control Tests Task prog. Test Implementation /Test 20
Organization of phase end Sign off for a in-house development project IS project (Application development) General framework CIO: Development of IS concept ( IS concept). CIO: Development of information system architecture ( Information system architecture). Organizer/ business manager: Identification of process outputs (CRI s), processes and definition of process flows ( Process development). Business analyst design of information relevant aspects of to-be-business-system Modeling of information relevant aspects, which are to be covered by the to be applications (essential data and transactions) Business analyst design of external application model Identification and detailed model of all transactions (incl. dialog-sequences), which should be provided by the new applications, identification and detailed model of data bases and data structures. Software engineer design of internal application model Construction of transactions and databases with specific developed tools and information technology, testing of the system. /Test 21
Organization of phase end Sign off for implementation Implementation Planning Fall-back Plan Resources concept Training Concept Data Transfer Concept Install Set up Test preproduction System system Setup Syst. Parameter Create Interface Progr. Create Data Migr. Progr. Finalize Organizational Design Processes Org. structure/ Capacity plan Activities Forms Downtime plan Create Batchjobs Setup authorizations Integration testing In test system Implementation Plan Plan Master Data Transfer Plan Transaction Data Transfer Finalize Documentation System docu. Org. Documentation Test Cases Test Master Data Collection Training Install Resources Test Results Final Fall-back Plan Backup Concept 22
Organization of phase end Sign off for implementation Implementation Plan Final check Resources Backup Concept Simulation Implementation Downtime Plan Fall-back Plan System Implementation Install Productive System Interfaces Tables Data Migration Integrationstest Transaction Data Master Data Authorizations Control Master Data Transfer Transaction Data Transfer Last integration test in productive system Test Cases Test Results System Transfer to an Business (RUN) System Doc. Org. Documentation. System Release /Test Productive System Operating/ Monitoring Hotline (Onsite)- Support Backup Life Test Incremental Enhancement 23
Control tasks in the context of analytical quality assurance Tasks of quality assurance Test Audit Internal review External review Internal test External test Method audit Project leadership audit SU CL SU CL CL CL CL = Client, SU = Supplier 24
Control tasks in the context of analytical quality assurance SU Supplier review/test It must be checked, If the design product is formally consistent If the design product is materially adequate The check is conducted on the design products, which are required for each project phase according to the project process model Before each external review/test (with client) there has to be a internal review test (on the level of supplier) 25
Control tasks in the context of analytical quality assurance CL Client review/test It must be checked, If the design product is materially adequate according to the needs. That implies formal consistence, for which the supplier is responsible. The review/test is conducted on design products, which are required at the extenal review for each project phase according to the project process model. The review/test is conducted by an independent review team, which is not involved in the project (not part of the project team but part of the client project organization). This review team is organized by the client project leader. 26
Control tasks in the context of analytical quality assurance General test levels CL 6 5 Acceptance ( Operations ) User test ( Acceptance test ) First of all without interfaces; at the end with all interfaces System test 4 ( Integration test ) to third-party system SU 3 2 1 Application test ( Chain test ) Transactions tests ( Single test f. logical transaction) Components test ( Single test for service function) Customizing / Programming Runs in parallel 27
Control tasks in the context of analytical quality assurance CL Client audit Method audit Spontaneous control through external instance, which assesses, if the project process model is applied in the project. Objective of the audit is project process conformity. Project leadership audit Spontaneous control through external instance, which evaluates, how the project can minimize deviations in the area of overdrafts regarding Costs Deadlines Benefits (scope of benefits) Project costs Objective of the audit are corrective measures to close deviations. Project time Benefits of design product (Scope of benefits) 28
Control tasks in the context of analytical quality assurance CL Client audit The audit is normally not announced, that means spontaneously. It is initiated through the client organization. Mostly it is executed by a third-party (e.g. consulting company). Each major project is audited regularly (every 6 to 12 months). Each audit analyzes and evaluates the project situation and suggests corrective measures, which are to be executed by project management and overlooked by the project committee. No large scale project without audits. 29
Constructive quality assurance ISO 9001 ISO 9000 series is a quality system standard of the International Standards Organization, Geneva, which can be applied generally to all products, services or processes. ISO 9001 regulates specifically the product and service production in a two-partycontract relationship: ISO 9001 defines the model for a quality system when a contractor demonstrates the capability to design, produce, and install products or services. 30
Constructive quality assurance ISO 9001 Target: - Regulation of the Client-Supplier-relationship (Two-party contract relationship) - Self enforcement of the supplier to quality orientation Method: Definition of project processes (project activities, results, roles etc.) Definition of configuration management for the purpose of evidence documentation (Management of versions and change requests regarding results) Definition of quality management system (institutional basis in the company) Certification after ISO 9000 Series Publication to clients Regular method audits 31
Constructive quality assurance SEI-Process-Maturity-Schema* Target: Evaluation of performance of suppliers in information system development. Originally initiated by the Department of Defense of the United States. Method: Analysis and classification of suppliers in the following 5 stages of maturity 1: Chaotic process 2: Repeatable process 3: Defined process 4: Controlled process 5: Optimized process * Humphrey, SEI, Carnegie Mellon University 32
Constructive quality assurance SEI-Process-Maturity-Schema Stage of maturity 1 Chaotic process : Chaotic ad hoc development process; Over-Commitment of supplier; Stars dominate project Stage of maturity 2 Repeatable process : Repeatable development process; not documented; by tradition; based on experience know how ( We have done it always like this ) Stage of maturity 3 Defined process : Development process is fully and consistently documented and is applied like this. 33
Constructive quality assurance SEI-Process-Maturity-Schema Stage of maturity 4 Controlled process : The development process is permanently measured in projects for gathering quantitative figures, which will be used in following projects as basis for planning. Stage of maturity 5 Optimized process : Defined, controlled development process is permanently optimized (between projects). If optimization was successful the results can be measured on the quantitative figures. 34
Constructive quality assurance Evolution of the SEI-Maturity Model: Project Management Maturity Model (PMMM) Level 5 Continuous Improvement Level 4 Benchmarking Level 2 Common Processes Level 3 Singular Methodology Feedback Level 1 Common Language Source: Kerzner, Project Management 35
Constructive quality assurance Evolution of the SEI-Maturity Model: Project Management Maturity Model (PMMM) Level 1 - Common Language: In this level, the organization recognizes the importance of project management and the need for a good understanding of the basic knowledge on project management, along with the accompanying language/terminology Level 2 - Common Processes: In this level, the organization recognizes that common processes need to be defined and developed such that successes on one project can be repeated on other projects. Also included in this level is the recognition that project management principles can be applied to and support other methodologies employed by the company. Level 3 - Singular Methodology: In this level, the organization recognizes the synergistic effect of combining all corporate methodologies into a singular methodology, the center of which is project management. The synergistic effects also make process control easier with a single methodology than with multiple methodologies. 36
Constructive quality assurance Evolution of the SEI-Maturity Model: Project Management Maturity Model (PMMM) Level 4 - Benchmarking: This level contains the recognition that process improvement is necessary to maintain a competitive advantage. Benchmarking must be performed on a continuous basis. The company must decide whom to benchmark and what to benchmark. Level 5 - Continuous Improvement: In this level, the organization evaluates the information obtained through benchmarking and must then decide whether or not this information will enhance the singular methodology. 37