Assessment of NCTD Program Management Framework for Positive Train Control Program

Transcription

1 Assessment of NCTD Program Management Framework for Positive Train Control Program Subtask 2: Analysis Gap Analysis Prepared for: Brad Hansen, M.S., PMP Director, PMO Capital Projects May icfi.com/transportation

2 EXECUTIVE SUMMARY Subtask Objective Identify gaps in the PTC Program project management processes, artifacts, tools and staff resources required for effective management Assess the PTC Program s Program Management Plan (PMP) compliance with: PMI PMBOK 49 Code of Federal Regulation (CFR) Contents of a project management plan ICF s experience and lessons learned from analogous programs, tailored to the NCTD PTC Program environment Review priority components of the PMP, as requested by NCTD: Program Management Organization the organizational structure and associated resources Requirements Management the processes and tools NCTD employs to manage requirements Risk Management the process and tools employed by NCTD to manage programmatic risk Recommend prioritized improvement areas to NCTD s program management approach based on expected benefit to the NCTD organization 1 icfi.com/transportation

3 EXECUTIVE SUMMARY Key Themes The implementation of the PTC System is complex and dynamic, placing stress on NCTD s organizational capabilities In many instances, the PTC Program s current state activities are inconsistent with the Program Management Plan s (PMP) documented processes, tools, and structures Formally documented program-level controls, integrating all PTCrelated activities and components, do not exist or are not being exercised The program-level controls that do exist are largely specific to the HTI PTC System contract which represents only 40% of the total program budget 2 icfi.com/transportation

4 EXECUTIVE SUMMARY Gap Analysis Results All Most Some Few The color scale reflects the degree of compliance with (i.e. all, most, some, few) the processes, artifacts and tools prescribed within the PMBOK knowledge areas How well does the PTC PMP stack up to PMBOK? How well does the Current State of the Program follow PMBOK? PMBOK Knowledge Area Integration Management Scope Management Cost Management Schedule Management Risk Management Quality Management Human Resource Management Procurement Management Stakeholder Management Communications Management PMP NCTD PTC Ratings Current State Summary Little evidence of integration processes or tools were observed The entirety of the PTC Program scope is not defined and documented No cost management baseline exists and inconsistencies between PTC Program and NCTD cost management processes are present Lack of a schedule baseline or a single integrated master schedule Undefined risk management process, but recently instituted project-level processes will enhance Program level risk management An organizational quality capability does not exist, nor is there a Quality Management System The current organizational structure is not the most effective approach Mature organizational procurement capabilities with few needed improvements Level of stakeholder identification, analysis and engagement for a Program of this complexity is not evident Strong organizational capability, though ineffectively integrated into the PTC Program 3 icfi.com/transportation

5 EXECUTIVE SUMMARY Recommended Improvements 1 Baseline the Program 2 Enhance program control 3 Re-align program organization 4 Implement a coordinated stakeholder strategy 5 Dedicate resources to prepare for postimplementation operations and maintenance 4 icfi.com/transportation

6 Methodology 5 icfi.com/transportation 5

7 METHODOLOGY Scope of Gap Analysis Evaluation of PTC Program Management Plan Assessment of Numerous Program artifacts, Program staff interviews, PTC Program organization charts, and observations all constitute the current state Standards used : A Guide to the Project Management Body of Knowledge (PMBOK) Fifth edition (with emphasis on the ten Knowledge Areas) The Standard for Program Management Third Edition 6 icfi.com/transportation

8 METHODOLOGY FRA PMP Content Requirements are addressed by PMI s PMBOK Knowledge Areas PMBOK Knowledge Area Integration Management Scope Management Cost Management Schedule Management Risk Management PMP Content Requirement (a) description of adequate recipient staff organization, complete with well-defined reporting relationships, statements of functional responsibilities, job descriptions, and job qualifications (d) document control procedure and recordkeeping system (i) plan for internal reporting requirements including cost and schedule control procedures (e) change order procedure which includes a documented, systematic approach to the handling of construction change orders (b) budget covering the project management organization, appropriate consultants, property acquisition, utility relocation, systems demonstration staff, audits, and such miscellaneous costs as the recipient may be prepared to justify (c) construction schedule Not required (but recommended, by the assessment team) (g) quality control and quality assurance programs which define functions, procedures, and responsibilities for construction and for system installation and integration of system components Quality Management (h) material testing policies and procedures (j) criteria and procedures to be used for testing the operational system or its major components Human Resource Management Procurement Management Stakeholder Management Communications Management (f) description of organizational structures, management skills, and staffing levels required throughout the construction phase Not required (but recommended, by the assessment team) Not required (but recommended, by the assessment team) Not required (but recommended, by the assessment team) 7 icfi.com/transportation

9 METHODOLOGY Rating Our color coded scale provides a rating for each knowledge area: All Most Some Few The color scale reflects the degree of compliance with (i.e. all, most, some, few) the processes, artifacts and tools prescribed within the PMBOK knowledge areas Ratings are provided for the PMP and the current state PMP Assesses the PMP against PMBOK Knowledge Areas Current State Assesses the Program s current state processes, artifacts and activities against PMBOK Knowledge Areas 8 icfi.com/transportation

10 Gap Analysis Results 9 icfi.com/transportation 9

11 GAP ANALYSIS RESULTS Integration Management Rating: PMP Current State includes the activities needed to identify, define, combine, unify, and coordinate multiple components within the program. Analysis Several key integration-related artifacts exist PMP, last revised on March 22, 2012 Project charter Organizational charts Program management information systems Recent project-level standard processes, templates, and reporting requirements have been implemented and are contributing to integration management Inconsistencies between the current state organizational structure and the structure defined in the PMP Identified Gaps Documented process and tools that integrate Program-level scope, schedule, schedule and budget (i.e., no Integrated Master Schedule, Program WBS, or time-phased budget) Many current state activities do not follow processes documented in the PMP Process for maintaining the PMP Process for program-level monitoring and control Program Charter lacks clear articulation of the purpose, scope, budget, schedule and roles/responsibilities Process for integrated change control (scope, schedule, cost, requirements, system configuration 10 icfi.com/transportation

12 GAP ANALYSIS RESULTS Scope Management Rating: PMP Current State defines the work required to deliver a benefit (major product, service, or result with specified features and functions) [and] includes all of the activities involved in planning and managing the program s scope. Analysis System requirements for PTC are provided in 49 Code of Federal Regulation (CFR) Requirements for Positive Train Control systems Additional due diligence is required to ensure PTC Program requirements are compliant with this CFR The PMP contains elements of scope management including, but are delegated to HTI: Contract WBS Scope change control process Requirements management and traceability Identified Gaps A scope statement that defines the entire PTC Program (note: an original Scope Statement was referenced in the materials reviewed during the Discovery subtask, but not found) A Program-level Work Breakdown Structure (WBS) to include ALL Program work (vendorimplemented and NCTD-implemented projects and tasks) Control of Program scope Program-level requirements management and traceability for all PTC Program projects 11 icfi.com/transportation

13 GAP ANALYSIS RESULTS Cost Management Rating: PMP Current State identifying the program s financial sources and resources, integrating the budgets of the program components, developing the overall budget for the program, and controlling costs throughout the duration of both the components and the program. Analysis Total program budget of $87.3M is funded through nine sources The program s contingency fund decreased by 70% during the period from July 2011 through January It is not clear whether the remaining contingency funds will be sufficient There are differences in the budget management practices used by the Program and the NCTD Finance Division, resulting in at least $21.2M (~24% of the total program budget) of funds that cannot be currently reconciled by the Finance Division Identified Gaps Program cost management processes do not reconcile with organizational cost management practices (15 accounts versus 5 accounts) Post-implementation operations and maintenance cost estimates have not been documented - current estimates appear to be anecdotal and not analytically based Cost management process 12 icfi.com/transportation

14 GAP ANALYSIS RESULTS Time Management Rating: PMP Current State includes determining the order in which the individual components are to be implemented, the roadmap for the program, and the milestones to be measured to keep the overall program on track and within defined constraints. Analysis Activities of individual contracts and certain Program elements appear in multiple schedules Project Managers are assigned to monitor contractor progress against project-level schedule baselines The PMP defines an approach to schedule management using a three-level hierarchy, each with a different level of detail - NCTD is not executing the approach Identified Gaps A Program-level schedule baseline A single integrated master schedule that encompasses ALL Program activities Dependencies Predecessor/successor relationships Critical Path Schedule hierarchy Schedule management process Objective basis for NCTD s ability to achieve the December 31, 2015 completion date 13 icfi.com/transportation

15 GAP ANALYSIS RESULTS Risk Management Rating: PMP Current State includes the processes of conducting risk identification, analysis, and response planning in order to reduce the likelihood of occurrence and/or impact an individual, or series of risks, have on a program. Analysis There are currently four versions of risk registers within the program, some more current than others Risk management discussion in the PMP and the Program Risk Management Plan have major inconsistencies Project-level risk management plans and processes have recently been implemented Identified Gaps A documented risk management process to ensure risk management consistency within all Program elements A single integrated risk register Risks are currently assessed and prioritized using only qualitative measures 14 icfi.com/transportation

16 GAP ANALYSIS RESULTS Quality Management Rating: PMP Current State includes the activities of the performing organization that determines program quality policies, objectives, and responsibilities so that the program will be successful. Analysis Current quality management processes in the PMP are specific to the HTI contract NCTD is required to document a complete description of the safety assessment and Verification and Validation processes applied to the PTC system in an FRA-mandated PTC Safety Plan (PTCSP) prior to FRA-testing of the system Rail Safety Consulting LLC is developing NCTD s PTCSP Identified Gaps Program-level quality management processes as part of a Quality Management System Clarity as to who is accountable within the PTC Program or NCTD organizations for quality management In-house staff accountability and capability for program quality management 15 icfi.com/transportation

17 GAP ANALYSIS RESULTS Human Resource Management Rating: PMP Current State includes the process that organize, manage, and lead the project team comprised of the people with assigned roles and responsibilities for completing the project. Analysis The program manager has authority to assign priorities, apply resources, and direct the work of individuals assigned to the project The PTC Program organization is largely geared to oversee the work of contractors and consultants is this precluding attention to internal work elements? Five Project Managers oversee 15 projects (contracts) Identified Gaps The organizational chart in the PMP does not match 1) the associated narrative description or 2) NCTD s organizational chart The PMP s listing of contractor and consultant resources is incomplete and does not match the contents of the PTC Vendor List Job skills, experience, and qualifications of personnel fulfilling positions within the PTC program are not documented This makes it difficult to assess whether the PTC Program has been appropriately and adequately staffed 16 icfi.com/transportation

18 GAP ANALYSIS RESULTS Procurement Management Rating: PMP Current State addresses specific procurement needs that are unique to managing the overall program and the needs of the constituent projects/components. Analysis An NCTD Contract and Procurement representative is matrixed to the program and provides contracting/procurement support Procurement management processes are integrated with, and leverage, NCTD s organizational procurement capabilities Identified Gaps Processes associated with formally closing out each contract after all deliverables and payments have been made are not defined The PTC Vendor List does not include all vendors and suppliers of each PTC Program system, subsystem, component, and associated product, and process in use system-wide. 17 icfi.com/transportation

19 GAP ANALYSIS RESULTS Stakeholder Management Rating: PMP Current State includes the activities associated with identifying, categorizing, and tracking all stakeholders who will interact with the program or be affected by the implementation of the program. Analysis PMP identifies three stakeholder groups: Communities Media Outlets Government Entities Preliminary analyses conducted by NCTD s Government Affairs and Communications Division identified 15 stakeholder groups in three categories: Internal General Public Targeted Identified Gaps A dedicated PMP section for stakeholder management A complete list of stakeholders to manage Coordination and collaboration between the PTC program and NCTD s Government Affairs and Communications Division this does not appear to take full advantage of the agency s capabilities 18 icfi.com/transportation

20 GAP ANALYSIS RESULTS Communications Management Rating: PMP Current State includes the activities for facilitating timely and appropriate generation, collection, distribution, storage, retrieval, and ultimate disposition of program information. Analysis The approach outlined in the PMP is largely focused towards external communications NCTD has a robust internal communications capability that has developed key communication-related elements regarding PTC Stakeholder management and communication is a critical aspect of the PTC Program and its success Identified Gaps The PMP fails to address how communications will be managed and disseminated to internal stakeholders regarding the status of the program Lack of internal communication, resulting in an insufficient level of understanding and engagement within the PTC Program among project team members 19 icfi.com/transportation

21 Recommendations for Improvement Prioritized based on expected benefit to the NCTD organization 20 icfi.com/transportation 20

22 RECOMMENDATIONS FOR IMPROVEMENT Recommended Improvements 1 Baseline the Program 2 Enhance program control 3 Re-align program organization 4 Implement a coordinated stakeholder strategy 5 Dedicate resources to prepare for postimplementation operations and maintenance 21 icfi.com/transportation

23 RECOMMENDATIONS FOR IMPROVEMENT Recommended Improvements Scope: Identify all work that s required to be accomplished by the PTC Program and develop an associated program-level WBS Cost: Reconcile current cost estimates and resource requirements with the newly developed WBS; develop risk-based budget contingencies Schedule: Develop a program-level integrated master schedule that encompasses all currently-identified tasks 22 icfi.com/transportation

24 RECOMMENDATIONS FOR IMPROVEMENT Recommended Improvements Evaluate and implement a program management information system sufficient for the size and complexity of the PTC Program Review and update the Risk Management plan (e.g. implement an integrated master risk register that includes quantitative analyses) Revise the PTC s Quality Management plan so it aligns with the future PTCSP and newly established Quality function within NCTD Identify and define configuration management requirements specific to the PTC Program Implement program-level variance analysis and reporting mechanisms Implement a process for maintaining and updating the PMP for the duration of the program 23 icfi.com/transportation

25 RECOMMENDATIONS FOR IMPROVEMENT Recommended Improvements Strengthen the organizational relationship between the PTC Program and NCTD s Project Management Oversight Office, Capital Projects Establish an independent Quality management capability within NCTD Formalize cross-organizational representation within PTC Program Establish formal job descriptions that include skills, experience, and qualification requirements for each role within the PTC Program organization Reissue an updated Program Charter to all NCTD staff that clearly highlights the purpose, objectives, governance structure, and roadmap for the PTC Program 24 icfi.com/transportation

26 RECOMMENDATIONS FOR IMPROVEMENT Recommended Improvements Review and expand upon the stakeholder analysis currently drafted by NCTD s Government Affairs and Communications Division Implement a coordinated internal and external communications strategy based on the updated stakeholder analysis Explore opportunities to link the PTC outreach strategy with Board of Directors and external project champions engagement 25 icfi.com/transportation

27 RECOMMENDATIONS FOR IMPROVEMENT Recommended Improvements Document on-going maintenance requirements for PTC-related components and ensure adequate maintenance plans are developed, implemented, and the appropriate personnel trained Identify additional organizational training needs, for example: Locomotive operators: work with Herzog to understand and document initial and recurrent training plans Maintenance of Way: determine which PTC-related components require familiarity with and understanding of by MOW personnel Plan, document, and integrate newly implemented PTC-related components into NCTD s configuration management capability Develop and document one-time, on-going, and incremental operating costs related to PTC and implement processes for reviewing and revising these costs (and assumptions) on a regular basis. 26 icfi.com/transportation

28 Next Steps 27 icfi.com/transportation 27

29 NEXT STEPS Plan of Action To include: Implementation plan for the recommendations identified in this task Current and proposed project management organization chart 28 icfi.com/transportation

30 29 icfi.com/transportation