Project Cost Overrun Review Process Improvement xxx, IUPUI TECH 581 Quality and Productivity in Industry and Technology Abstract: This project used Lean Six Sigma tools to review the process performed by the Engineering Department to review, document and submit approvals for project cost overruns. Engineering spends approximately 800 hours annually performing these reviews, costing the Company over $68,000 a year. This work activity is not standardized across engineering and no process documentation or tools exist to ensure Company objectives are met to complete these reviews within 30 days. In this study key stakeholders and customers involved in the process were interviewed, process observations were performed and measurement data was collected and analyzed to determine where improvements could be made. Baseline measurements indicated high variability in process times ranging from 5 minutes to 120 minutes and analysis of the data collected found the Key Process Input Variable was the "type" of project being reviewed. The team developed process improvement documentation which included process flowcharts, job aids, and spreadsheet tools and kicked off a Pilot of the new process. During the Pilot the process documentation and tools will be refined and sustained improvement measured and documented prior to a full implementation. The initial results from the Pilot indicate reduced variability in process times and lower average process times are possible. Upon full implementation measurement data will be collected quarterly to ensure process improvement times are sustained. The key to implementing process improvements is have a standard process in place upon which to base those improvements. Having established this documented process, future mechanization efforts or process enhancements will have a basis for measuring improvements. Second, having well defined performance measures in place is key to understanding the cost and time impacts of process changes. Implementing the key measures through this project will position the company for making further improvements, possibly through mechanization efforts, by providing a basis for cost benefit studies. 1
Background: Each year Engineering creates and obtains approval for over 5,000 jobs. Approximately 20% of these jobs require supplemental approvals due to revisions, cost overruns and unforeseen field conditions. This process consumes approximately 800 engineering hours a year. The process to complete these reviews is not standardized and no standard process documentation exists within engineering to review, document and process these approvals. In previous years, this process received little attention but in 2009 managers were given an objective to complete their reviews and submit all supplemental approvals within 30 days. This objective has highlighted the need to improve the process to ensure these timeframes can consistently be met by the Engineering organization. Experimental Method: The Lean Six Sigma Methodology: The Six Sigma DMAIC Methodology was utilized for this investigation and is summarized below: Define a problem, set a goal striving for customer satisfaction and aligning business objectives. Measure the process by collecting relevant data to realize issues and for future comparison. Analyze to verify connection and cause of problems. Improve or the process by reducing variation, based upon the analysis. Control the process and maintain the reduction of variation. Define Phase: A project charter was completed by the project team that defined the business case, problem statement and goal statement for the Project Cost Overrun Review Process Improvement project. The project charter is shown in figure 1. 2
Lean Six Sigma Project Charter Project Name: Reduce Engineering Time to Prepare Cost Overrun Approvals Date Chartered Start Date: Target Completion Date: 10-7-09 10-7-09 12-16-09 Project Team Phone/email Title Engineering Manager Process Owner Phone Title Engineering Manager Problem Statement The purpose of the project is to reduce the Engineering time required to review, document and submit supplemental approvals for job cost overruns. This process consumes approximately 800 Engineering hours annually in Indiana. Goal Statement Reduce the Engineering time required to prepare job cost overrun documentation by 25% (to 30 minutes or less for each submittal), saving approximately 200 engineering hours per year. Project Scope This process begins when a cost overrun on a project occurs which requires engineering to review the reasons for the job cost overrun and ends when the supplemental job approval is submitted and approved by management. Cost overruns are defined by Operating Procedure #46 which requires supplemental approval for jobs that exceed their current approved amount by >10%. In order to minimize disruption in the Engineering organization, only one Indiana Design Center will have engineers interviewed, observed, and process improvements implemented during the course of this study Deliverables 1. Documented process for efficiently reviewing, documenting and approving job cost overruns. 2. Consistent engineering time savings of 25% per job over current methods when preparing job supplemental approvals. Figure 1 Project Charter A Voice of the Customer (VOC) analysis was performed by interviewing engineers performing the process today along with Planning, Finance and other managers not on the project team and asking four key questions: 1. What do you like about the existing processes related to the OP46 process? 3
2. What are the weaknesses within these processes? 3. What are the opportunities for improvement within these processes? 4. What could potentially threaten the success of this project team? VOC analysis indicated that while a strength of the current process is that the system does provide proactive notification to the engineers that a review needs to be performed, the current process was not well understood by the engineers and they had no documentation developed specifically for these reviews. As a result, no single process was being used to complete these reviews. Instead, each engineer pulled different reports and looked at different information as they worked the process according to individual preferences. The VOC analysis also revealed there were opportunities to improve the process through proactive documentation, improved reports and more feedback from Construction on cost overruns. The analysis also found that the process was threatened by poor understanding of the financial systems and what steps needed to be performed to correctly complete the review requirements. Measure Phase: Baseline measurement of the following Key Process Input Variables (KPIVs) was performed: Number of Op46 notifications received each week by each engineer Number of Op46 notifications received each week that require a handoff to Planning Size of the project with an Op46 notification (dollars) Size of the project overrun with an Op46 notification (dollars) Project Type The overall process time was gathered using two methods to track the Key Process Output Variables: 1. Engineers self reported their total process time along with the error notification and approval submission dates on a data collection form placed in a shared data area; 2. Direct observations of engineers performing the process were performed to gather the same information in additional to other process steps that self reporting may have missed such as the number of systems accessed and reports pulled during the review. 4
This data was gathered over a period of 45 days and included 50 self reported observations and 15 direct observations of the process. A process map was created that documented how the engineers generally performed the process today as shown in Figure 2. However, since no standard process existed, each engineer performed the process slightly different from the one shown. The largest variation was related to the specific reports pulled during the review and how the information on those reports was used. Start Process: Engineer receives op46 error notification from CMC report Access CFAS to determine the size and scope of the error Engineer pulls Jam status & estimate reports and compares the forecast hours and costs to actuals Engineer reviews job file for revisions, change orders, or other items that increased cost after initial approval If Necessary, Engineer talks to Construction, the Field Coordinator, or accesses other systems concerning the cost overrun Planning project? Yes Provide Planning with documentation on their findings related to the project cost overrun Follow up to ensure Planning submits approval within 30 day objective. No Engineer documents the reason for cost overrun in CFAS Project Description section Update CFAS budget lines to meet or exceed current costs (if project not complete) Prepare and submit MIC addendum to supervisor End Process Figure 2 Process Map Results and Discussion: Analyze Phase: Multiple direct process observation studies were conducted by team members using process observation worksheets and spaghetti diagrams to quantitatively assess the impact of the primary operational barriers. Examples of process observation worksheets, spaghetti diagrams and other data collection and analysis tools for the Project Cost Overrun Review Process Improvement process are shown in the following figures: 5
Date Notification occurred Engineer ID Project Number Time Spent Preparing Supplemental Submittal Planning Project? (YES/NO) If Planning Project, Date sent to Planning Date Supplemental Submittal sent Figure 3 Observation Worksheet #1 (Engineers self-reported this data) Date Cost overrun notification received Date Supplemental Submittal sent Engineer Original cost of project Amount of cost overrun Project Type Job # Step Description Clock Time Task Time (min) Wait Time (min) # System Loads or Logins # Rpts or System Inquries Number of Interuptions Interuption time Observations 1 Review OP46 error report 2 Review Project in CFAS 3 Pull & Review Jam reports 4 Review paper job files 5 Calls to Construction, FC, etc? 6 Other systems? 7 Planning Project? (Y/N) 8 Email to Planning (process stops) 9 IF Not Planning Project, THEN 10 Document overrun reason in CFAS 11 Update CFAS budget lines 12 Prepare MIC addendum & Submit Total Time Figure 4 Observation Worksheet #2 (Used in direct observations by Process Team) 6
Figure 5 Spaghetti Diagrams The primary operational barrier to improving process times and ensuing that cycle time objectives were consistently met was the KPIV of Project Type. Stratification of the data by Project type found that some types of projects consistently required higher average process times than others (see Figure 6). Average minutes 250 200 150 100 50 0 Time to Compete Process by Project Type 91% 77% 61% 43% 60 58 50 22% 43 38 25 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Figure 6 - Time to Complete Process by Project Type 7
Data gathered during the direct process observation found that some Project Types required more reports, system queries and longer review times than other Project Types. A detailed analysis found that some projects had a higher percentage of material, utilized contractor labor in addition to internal construction labor, and often involved equipment placements (Figure 7 & Figure 8). % Costs FTTN F2 Projects % Costs Other Project Types 0% 18% 7% 0% 75% Construction Labor Materials Engineering Equipment Exempt Materials 6% 27% 16% 7% 23% 21% Construction labor Materials Engineering Equipment Exempt Materials Contract Labor Figure 7 - FTTN Project Costs Figure 8 - Other Project Costs The key findings of the Analyze phase were: The key process input variable driving process time is Project Type or Project Complexity (meaning more materials and equipment in addition to construction and engineering labor). FTTN F2 projects are less complex with few materials and mostly consist of internal construction labor and as a result it is easier to determine the cause of the cost overrun. Investigating overruns on the more complex project types of OSPRP, Legal Mandate and others requires 57% more reports and 122% more system inquiries. The engineers had some uncertainty on which reports or system queries will provide the best information resulted in pulling several unnecessary reports. The increased complexity of these projects also increases documentation time. Process times for complex projects are double that of the less complex FTTN F2 type projects. Improve Phase: Multiple solutions were generated to reduce or eliminate the impact of operational barriers: 1. Establish a documented process flow and create a job aid for Engineers to standardize the process. 8
2. Establish a documented process flow, job aid and request IT create customized reports that summarize and compare initial costs to actual costs along with details on key cost overrun areas. 3. Establish a documented process flow, job aid and Request IT automatically pull and send the existing JAM report required for every review to the engineer with the OP46 notification report. Solutions #2 and #3 build upon solution #1 by adding some system automation to the process review to reduce the manual work effort required to gather and evaluate the information. Each solution was evaluated on the basis of cost, likelihood of success, and ease of implementation using the solution matrix shown in Figure 9. Solution #1 Solution #2 Solution #3 KPIV: Establish a documented Process Flow and create a Job Aid for Engineers Establish documented process flow and Job Aid and request IT Establish a documented Process Flow create customized cost analysis and Job Aid and have IT send JAM reports report Complex Project Types* 7 10 8 Total Impact* 7 10 8 Success Criteria Likelihood of Success 9 1 3 Cost ($$$) note: 10 = lowest cost 10 1 6 Engineer Satisfaction 7 10 9 Engineering Supervisor Satisfaction 7 10 9 Finance Satisfaction 10 5 10 Planning Satisfaction 7 10 9 Consistent Quality of Supplemental Reviews 10 10 10 Success Criteria Total 60 47 59 Total Score 67 57 64 Figure 9 - Solution Matrix 9
The team determined that while solution #3 had a strong potential of process time reduction, the timeframe required to obtain IT approval and development of the automatic report delivery impacted the likelihood of a successful Pilot in 2009. Instead, solution #1 was selected to be piloted beginning in December 2009 to ensure a consistent well documented process is in place. Process times will continue to be collected using Observation Worksheet #1 as shown in Figure 3 above. A Future State process map was developed to reflect the process changes developed by the project team. While the changes are not dramatic, it reflects the focus on pulling a single JAM status report (in process step 3) and eliminating the step where the engineer makes calls to the field for additional information. These calls were rare, did not provide any additional information during observations, and added unnecessary process delays. Start Process Engineer receives op46 error notification from CMC report Access CFAS to determine the size and scope of the error Engineer pulls Jam status report and compares the forecast hours and costs to actuals Engineer reviews job file for revisions, change orders, or other items that increased cost after initial approval Planning project? Yes Provide Planning with documentation on their findings related to the project cost overrun Follow up to ensure Planning submits approval within 30 day objective. No Engineer documents the reason for cost overrun in CFAS Project Description section Update CFAS budget lines to meet or exceed current costs (if project not complete) Prepare and submit MIC addendum to supervisor End Process Figure 10 - Future State Process Map Control Phase Pilot Implementation: Pilot implementation will be conducted using a staggered implementation strategy, utilizing small, incremental tests of the process change. One engineering office began the Pilot process 10
after a 12/11/2009 kick-off and a follow-up review on 12/15/09 to ensure the process was understood by the participants. This Pilot will run for at least 4 weeks or until at least 20 observations of both FTTN F2 and the other more complex Project Types can be collected. During this time refinements to the documentation will be made based on the feedback received during the Pilot. While the pilot is still in progress, the initial data gathered on 16 observations of the FTTN F2 projects was promising. Graphs of the results from the pilot phase as compared to pre-pilot results are shown in Figures 11 and 12. Average process times for even these project types declined but more importantly, process time variation was reduced. 5 15 25 35 45 55 Figure 11 Pre-Pilot Results FTTN Projects BoxPlot BoxPlot 10 12 14 16 18 20 Figure 12 Pilot Phase Results FTTN Projects To date only one observation of the "Other Project Types" has been made so that comparisons to the Pre-Pilot process times cannot be made at this time. 11
Control Phase Full Implementation Plan: If the initial pilots are successful, the project team will host training for the other engineers in the N Central and South Bend Design centers. They will be trained using the updated flowcharts and documentation based on the Pilot feedback. Diffusion of the process to the rest of the engineering organization in Indiana will begin during this phase of the implementation. Supervisors and engineers from other parts of the State will be invited to observe the process in action and the flowcharts and documentation will be presented to them during staff meetings. Standardization of the process will be ensured by placing the documentation in a shared data documentation folder for reference by all Stakeholders and engineers. The documentation will be reviewed annually and updates made as required. To ensure process improvements are sustained the total process time will be collected for all projects with OP46 cost reviews for one week every quarter using the Pilot Plan Data collection tracking form. These results will be summarized for each Design area in the chart shown in Figure 13. OP46 Project Cost Review Data Collection (week of 2-8-2010 thru 2-12-2010) Engineer # Op46 Min Time Max Time Avg Time area reviews (minutes) (minutes) (minutes) Min Days Max Days Avg Days South Indianapolis N Central North Total Figure 13- Cost Review Summary Report This information will then be charted on a quarterly basis to ensure the process is in control. Figure 14 illustrates the quarterly tracking sheet that will be used to chart the Average process time for each Design area and for the State as a whole. 12
Average Time For OP46 Project Cost Reviews (minutes) Engineer area 1Q10 2Q10 3Q10 4Q10 South Indianapolis N Central North Total Figure 14- Cost Review Quarterly Tracking Report Comparisons to prior quarterly results will be performed by the Engineer Managers and posted on Engineering web for review. The existing OP46 tracking reports posted on the Results Web will be monitored monthly to ensure key objectives for the overall process continue to be met (15 day handoff to Planning and the 30 day overall submittal objective). Updated Cost/Benefit Analysis The current process is estimated to cost approximately $68,000 annually and require over 800 engineering hours to complete. The cost of implementing this project can be summarized as: Training the engineers on the process is expected to cost $4,420 in lost engineering time (1 hour * 52 engineers * $85/hour = $4,420) The documentation was developed off line and required no company costs to develop other than printing copies for the engineers to reference. The benefits of standardizing the process are consistent methods and the establishment of a platform for future process improvements. If the Pilot successfully delivers the 25% time savings objective, the project will save the company $17,000/year (800*25%*$85/hour = $17,000). Conclusions: The key to implementing process improvements is have a standard process in place upon which to base those improvements. Having established this documented process, future mechanization 13
efforts or process enhancements will have a basis for measuring improvements. Second, having well defined performance measures in place is key to understanding the cost and time impacts of process changes. Implementing the key measures through this project will position the company for making further improvements, possibly through mechanization efforts, by providing a basis for cost benefit studies. The initial Pilot results, while incomplete, indicate process time improvements and reduced process variation is possible through this project implementation. Appendices: Measurement Phase Data Checksheet #1 Measurement Phase Data Checksheet #2 Improve Phase Pilot Data Checksheet OP46 Baseline Process Job Aid OP46 Expenditure Inquiry Job Aid OP46 Process Flowchart Job Aid (Excel File) CFAS Cost Summary Template (Excel File) 14