EXECUTIVE SUMMARY...5

Transcription

1

2

3 Table of Contents EXECUTIVE SUMMARY...5 CONTEXT...5 AUDIT OBJECTIVE...5 AUDIT SCOPE...5 AUDIT CONCLUSION...6 KEY OBSERVATIONS AND RECOMMENDATIONS INTRODUCTION BACKGROUND OBJECTIVES SCOPE METHODOLOGY STATEMENT OF ASSURANCE OVERALL ASSESSMENT AUDIT CONCLUSION SCORECARD: CURRENT PROJECT STATUS OBSERVATIONS AND RECOMMENDATIONS PROJECT GOVERNANCE MEETING BUSINESS REQUIREMENTS PROJECT MANAGEMENT TECHNOLOGY AND INFRASTRUCTURE CONCLUSION...35 APPENDIX A DETAILED AUDIT METHODOLOGY...37 Table of Contents 3

4

5 Executive Summary Context Information Technology has been identified in Elections Canada s Strategic Plan as a key enabler to carry out the mandate of Elections Canada. The Information Technology sector is responsible for ensuring that the services it provides are aligned with the agency s business objectives. The current IT environment has served the agency well. However, Elections Canada must now make substantial investments in its IT environment to improve and enhance its capabilities. The agency has started an Information Technology Renewal (ITR) project to modernize its IT infrastructure and business applications. This project is one of the most significant information technology initiatives to be undertaken by Elections Canada and includes processes that were not previously required. It was considered important to include the IT Renewal project in the Risk-Based Audit Plan. The focus of this audit is on the Integrated Program Office, which is important for effective and efficient project management. It is important to note that IT Renewal was a system under development at the time of the audit. The criteria were assessed within that context, and this assessment is the result of comparing what was observed with what was expected at this point of the project. As the project moves forward, a new assessment of some or all of the criteria will be necessary. Audit Objective The objectives of this audit are to provide assurance to senior management that: the IT Renewal Integrated Program Office has implemented effective project management techniques, systems, processes and controls for managing the IT Renewal project in line with the size and importance of the project; the IT Renewal Integrated Program Office framework and program are complete and reporting is consistent with Elections Canada s overall governance, plans and best practices. Audit Scope The audit scope included: the IT Renewal Integrated Program Office and project management framework for the management of the IT Renewal project the governance structure used by Elections Canada and the IT Renewal Integrated Program Office to provide oversight to the project. The audit did not provide an overall assessment of the ITR initiative. Rather, it focused on the Integrated Program Office, project management and governance. Executive Summary 5

6 The audit was conducted from October to December Audit Conclusion The IT Renewal Integrated Program Office has implemented many elements of effective project management techniques, systems, processes and controls for managing the IT Renewal project, in line with the size and importance of the project. Much of this structure is new to Elections Canada and it will remain once the ITR is complete. However, the governance structure within which the ITR reports requires management attention and the level of involvement of the business sponsors should be improved. The IT Renewal Integrated Program Office framework and program are currently being implemented and many components were established and functioning at the time of the audit. The Integrated Program Office is to act as integrator across all ITR efforts but it has not yet put in place adequate reporting processes. Improvements are needed in management status reporting, financial control and reporting, risk management, project performance reporting and technology transition. Key Observations and Recommendations These are the key observations and recommendations of the audit. Governance Project governance is seen as a high risk. The governance processes are inconsistent and unclear. Management should clarify the roles, responsibilities and relationships of the three involved committees to strengthen the governance of the ITR initiative. Status reports appropriate for the various committees have yet to be defined. Without appropriate information, the governance structure may not be able to provide adequate oversight and reasonable assurance that the project will achieve its objectives. The project has not started to do any measurement of benefits achievement. A formal process to identify, measure and monitor benefits and outcomes of the project should be established. Business Requirements It is too early in the overall process to measure the degree to which business requirements have been identified. The focus of the audit was primarily to examine the degree to which the business sectors understood how ITR would benefit business users in the future. There has been little involvement from the business sectors since the business case was developed. ITR should fully engage the business sectors to ensure its deliveries ultimately meet the requirements of business users. ITR is bringing a new infrastructure and IT services to Elections Canada. Organizational changes needed to properly use and support it, need to be identified and established. 6 Audit of the IT Renewal Integrated Program Office

7 Project Management Project organization and structure A project management structure has been established, and it has evolved as the project progressed and management reacted to the complexity of the project. The organization has used many contractors, which add a level of complexity. Segregation of duties is in place and is appropriate for contracting and financial approvals. Efforts should be made to retain knowledge when contractors leave the agency. Elections Canada has experienced over a period of time difficulty in arranging the contracted resources required. Recent initiatives have improved the process. Development Process It is too early to assess most criteria under the Development Process. Documentation for implementation and fallback plan is not complete. These plans should be developed for the various projects in IT Renewal. Project Control Processes Project control processes should be set up to ensure that all project activity has the appropriate management oversight, and that it delivers the anticipated product with minimum risk. In some cases, it is too early to measure these standards. An approved integrated plan exists. The schedule is complex with many interdependencies. It is difficult to obtain a complete and clear corporate financial report of the current status of the project, and of the activities for which funding was approved. Technology and Infrastructure Architecture Past architectural choices have resulted in different technology silos. The IT Renewal will establish new standards for applications and IT services. Senior management should support the Architecture and Standards Board (ASB) and ensure that the ASB s standards and decisions apply to all Infrastructure Management/Information Technology (IM/IT) initiatives. Appropriate security is essential to deliver the business solutions. A plan for achieving security and establishing security requirements for applications that will use the new infrastructure should be developed and implemented. Technology Transition ITR project plans assume the use of a high-speed network linking to the offices of all Returning Officers. In reality the available network speeds will vary considerably across the country. The ITR project should be coordinated with the Telecommunications Infrastructure Evolution project, and a plan should be developed for telecommunications in areas where high speed bandwidth is not available. Executive Summary 7

8

9 1. Introduction 1.1 Background Elections Canada is an independent, non-partisan agency reporting directly to Parliament and headed by the Chief Electoral Officer of Canada. Its mandate is to be prepared at all times to conduct a federal general election, by-election or referendum, administer the provisions of the Canada Elections Act, monitor compliance and enforce electoral legislation. To be able to carry out its mandate, IT has been identified in Elections Canada s Strategic Plan as one of the four key Enablers. The Information Technology sector has the responsibility of ensuring that the services it provides are aligned with the agency s business objectives. The current IT environment has served the agency well. However, Elections Canada must now make substantial investments in its IT environment to improve and enhance its capabilities. In , Elections Canada started an Information Technology Renewal (ITR) project initiative to modernize its IT infrastructure and business applications. The chosen renewal option consists of an IT Enterprise Architecture with networked, high-speed connectivity, consolidated processes, centralized data, a primary data centre in new facilities and a backup data centre, and upgraded core field applications. The latter deal with elector registration, event results and payment of electoral workers. Remaining business process requirements are to be addressed subsequent to the initial ITR project. The ITR consists of three phases. At the time of the audit, Phase 1 was planned to be completed by October The total cost for the four-year ITR project is estimated at $39 million. A business case was accepted by the IT Renewal Steering Committee and approved by Elections Canada s Executive Committee (EXCOM) in February The IT Renewal project includes the establishment of an Integrated Program Office (IPO), which is the management focal point for the coordination and integration of the various IT projects. It was considered important to include the IT Renewal project in the Risk-Based Audit Plan. This audit focuses on the Integrated Program Office, which is important for effective and efficient project management. 1.2 Objectives The objectives of this audit are to provide assurance to senior management that: the IT Renewal Integrated Program Office has implemented effective project management techniques, systems, processes and controls for managing the IT Renewal project in line with the size and importance of the project; and, the IT Renewal Integrated Program Office framework and program are complete and reporting is consistent with Elections Canada s overall governance, plans and best practices. 1. Introduction 9

10 1.3 Scope The audit scope included: the IT Renewal Integrated Program Office and project management framework for the management of IT Renewal project the governance structure used by Elections Canada and the IT Renewal Integrated Program Office to provide oversight to the project. The audit did not provide an overall assessment of the ITR initiative. Rather it focused on the Integrated Program Office, project management and governance. The audit was conducted from October to December Methodology The audit methodology used is largely based on the Control Objectives for IT (COBIT) published by the Information Technology Governance Institute. COBIT is an IT governance framework and supporting tool set that allows organizations to bridge the gap between control requirements, technical issues and business risks. The model also draws from other sources of standard IT approaches including the IT Infrastructure Library (ITIL). ITIL, which has become the de facto standard for IT service delivery, is currently being researched and potentially implemented in the federal government. This assessment addresses the COBIT processes, allowing for the unique nature of each project. This risk methodology is modeled after the normal set of accountabilities involved in developing software and managing systems projects. This resulted in the following four domains: Governance Risk, Business Risk, Project Risk, and Infrastructure Risk. The detailed audit criteria used in this audit are provided in Appendix A. Our approach relied on a review of existing documentation and on gaining an understanding of the processes through interviews with key Election Canada staff and Integrated Program Office consultants. The audit team conducted 18 individual interviews and reviewed all relevant documentation. 10 Audit of the IT Renewal Integrated Program Office

11 2. Statement of Assurance In my opinion as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed with management. The opinion is applicable only to the entity examined. 2. Statement of Assurance 11

12

13 3. Overall Assessment 3.1 Audit Conclusion The IT Renewal (ITR) Integrated Program Office (IPO) has implemented many elements of effective project management techniques, systems, processes and controls for managing the IT Renewal project, in line with the size and importance of the project. This project is one of the most significant IT initiatives to be undertaken by Elections Canada and includes processes that were not previously required. The new structure will remain once ITR is complete. However, the governance structure within which the ITR reports requires management attention and the level of involvement of the business sponsors should be increased. The IT Renewal Integrated Program Office framework and program are currently being implemented and many components were established and functioning at the time of the audit. The Integrated Program Office is to act as integrator across all ITR efforts but has not yet put in place adequate reporting processes. Improvements are needed in management status reporting, financial control and reporting, risk management, project performance reporting and technology transition. It is important to note that IT Renewal was a system under development at the time of the audit. The criteria were assessed within that context, and this assessment is the result of comparing what was observed with what was expected at this point of the project. As the project moves forward, a new assessment of some or all of the criteria will be necessary. 3.2 Scorecard: Current Project Status The scorecard below reflects the current status of the IPO at the time of the audit. The rating and explanation indicate the status of the current implementation and the level of improvement required, given that ITR is still a system under development. Scorecard Criteria Refer to Appendix A Governance Senior Management Control Framework Management of Scope and Change Investment Management and Benefits Achievement Rating Explanation The mandates of three committees with influence over the ITR initiative are in conflict. Governance processes are inconsistent and unclear. Memberships of committees overlap. The IPO has yet to define a status report process appropriate for the various committees. Scope has not been an issue to date. Once Phases 1A and 1B are completed, management will have to confront more difficult questions of scope. The project has not started to measure benefits achievement. 3. Overall Assessment 13

14 Criteria Refer to Appendix A Business Requirements Business Requirements Management Business Solutions Analysis Management of Organization Development Project Management Project Organization and Structure Project Management Framework Standard Project Documentation Management Reporting IPO Properly Staffed and Supervised Contracting Arrangements for Hardware and Human Resources Development Process Adequate Software Development and Acquisition Standard Business Solution Design Approval Project Developed in Accordance with Design Specification Project Developed in Accordance with Development and Documentation Standards Project Assurance Methods Implementation and Fallback Plan Project Control Processes Approved Integrated Project Plan Rating Explanation Since the business case was created, there has been very little involvement by the business sectors. The business sectors do not have a good understanding of how ITR will benefit them, and how future business solution design will be affected. A communication strategy about benefits and outcome of ITR is necessary. A formal communications program is now being launched. ITR is bringing a new infrastructure and IT services to Elections Canada. Organizational changes to properly use and support it, need to be identified and established. Project management structure has been established, and it has evolved as the project progressed and management reacted to the complexity of the project. Project documentation exists and standards have been established for such deliverables. Management reports are not yet regularly submitted to the oversight committees - not enough management level information is available. Reports are being revamped to provide more management focus. The organizational structure is still evolving and the many consultants add a complexity to segregation of duties. IPO management has ensured that EC employees are in place to handle contracting and financial approvals. Efforts should be made to retain knowledge when contractors leave the agency. EC has experienced difficulty arranging the contracted resources needed over a period of time. Recent initiatives have improved the process. It is too early to measure this criterion. It is too early to measure this criterion. It is too early to measure this criterion. It is too early to measure this criterion. It is too early to measure this criterion This area is included in the work plan. Documentation is planned but not yet completed. Appropriate implementation and fallback plans still need to be developed for the various projects. An approved integrated plan exists. The schedule is complex with many interdependencies. 14 Audit of the IT Renewal Integrated Program Office

15 Criteria Refer to Appendix A Project Financial Control Project Change Control Process Quality Assurance Project Risk Management Project Performance Measurement, Monitoring and Reporting IPO Problem Management Procedures and Standards for Acquisition of IT-Related Solutions Technology and Infrastructure Infrastructure Management Technology Transition Rating Explanation It is difficult to obtain a complete and clear corporate financial report of the current status of the project at the activity level. A policy has been developed for Change Management (within the development process) and there is a Change Management form. This process is new and we have not looked at it in the operating context of the IPO There is a Quality Management Plan. This process is new and there is not enough evidence of application to rate the process. This process is new and the templates are not yet fully used. The IPO is working to gain acceptance. This process is new and outcomes measures are not yet defined There is an Issue Management Procedure. Issue logs are being kept and reviewed at the project level and discussed at the working committees. Procedures and processes associated with procurement process established. However, procurement process was over and above the ITR resource workplan, which created delays. It Renewal will establish new standards for applications and IT services. Senior management should support the Architecture and Standards Board and ensure that its standards and decisions apply to all IM/IT initiatives. A process should be established to ensure that all new applications are reviewed to verify compliance with the standards of the new architecture before being approved. A plan for achieving IT security should be developed and implemented. ITR should be coordinated with the Telecommunications Infrastructure Evolution project, and a plan should be developed for telecommunications in areas where high speed bandwidth is not available. 3. Overall Assessment 15

16

17 4. Observations and Recommendations The details of each observation, conclusion and recommendation resulting from our audit procedures are outlined below. 4.1 Project Governance The IPO is responsible for the governance of the various projects within ITR. However, since it operates within the larger governance of the project, both must be examined since they are interrelated. Senior Management Control Framework Committee Structure There are three committees involved in overseeing the ITR project: the IM/IT Committee, the ITR Steering Committee and Elections Canada s Executive Committee (EXCOM). The alignment and integration points among these three committees are unclear. The IM/IT Committee was set up in November 2007 at the request of the Chief Electoral Officer, as part of a new governance structure. Its mandate is to ensure that Elections Canada s longer-term direction and strategic objectives are supported by adequate IT infrastructure, systems, initiatives and information management framework. EXCOM approved the Terms of Reference though there are no minutes to support this. The committee is supposed to monitor project progress but it does not actually grant or deny funding. It recommends projects for approval to EXCOM. However, IT projects sometimes go to EXCOM for funds without first going through the IM/IT committee. The committee has met on an irregular basis, sometimes with large time gaps between meetings. The agenda of the IM/IT committee rarely addressed the ITR project. The ITR Steering Committee is meant to guide the ITR project, keep EXCOM informed and to monitor major changes to the project including budget. The Terms of Reference exist in draft form and there is no record of them having been approved. Members are unclear about this committee s role versus that of the IM/IT Committee. The ITR Steering Committee reports to EXCOM but because it is mainly composed of EXCOM members, it almost replicates EXCOM. EXCOM meets weekly for a roundtable meeting and on a monthly basis. EXCOM approves the funding for the ITR project. The Chief Information Officer (CIO) has presented items related to ITR on a regular basis. The items mentioned at EXCOM were mainly related to the large and high cost streams: the data centre move; the facilities management contract; the request for proposal for ITR support and the telecommunications project. EXCOM has made decisions on two subjects related to ITR, the business case and the data centre move options, at special meetings for which there are no minutes. 4. Observations and Recommendations 17

18 The mandates of the three key committees are not clear. Some of the committees duplicate each other in membership which may be the cause of the irregular meeting schedule and the lack of a defined order for consideration of issues. The actual process for decision making across the three committees is not clear. The practice has often been that ITR project issues go to the ITR Steering Committee and then directly to EXCOM, while some non-itr projects go through the IM/IT Committee after funding is approved by EXCOM and they are then included in the operating plan. Projects are sometimes approved, funded and monitored through different processes. An inconsistent and unclear governance process may result in ineffective and inefficient project monitoring and decision making. It is essential to clarify the mandates of the committees. The IPO has recommended governance changes to the CIO that will affect reporting relationships for ITR. However, steering committees are being set up for two other major projects, e-registration and Special Voting Rules System (SVRS), but if they follow the same model and remain independent from the IM/IT committee, they will exacerbate the problems mentioned. Recommendation 1. It is recommended that Elections Canada senior management strengthen the governance of the ITR initiative and clarify the roles and responsibilities of the three committees, and especially the relationships among them. Management Action Plan Responsible Position: CIO We accept this recommendation. Action has been taken to implement this recommendation. Given the size, complexity and risk associated with IT Renewal (ITR), it was recommended and approved on February 17, 2009 that an ITR Program Board comprising all EXCOM members be instituted to provide direction, oversight, and support to ITR. The Board will provide improved and expanded access to the business sectors providing detailed insight into the implementation activities of ITR. The Governance and decision making for ITR have been strengthened as a result. The ITR Program Board replaces the ITR Steering Committee. The CEO is the ITR Program Sponsor. The Draft Terms of Reference (TOR) for the Program Board clearly detail roles and responsibilities, membership, expectations, and authorities as well as identifying the relationship of Program Board with the IM/IT Committee and EXCOM. To ensure we have time to allow the Governance process to fully evolve we expect this recommendation to be completed by June Audit of the IT Renewal Integrated Program Office

19 Management Information It is important to provide sufficient management information on the project and its current status to properly support analysis and decision making. The IPO has not yet defined a status report format appropriate for the various committees. Some reports have been developed, but the level of information is too detailed to support the governance at senior management and committee level. Neither the ITR Steering Committee nor the IM/IT Committee have controlled or reviewed budgets, although a budget review function is part of the Terms of Reference of the ITR Steering Committee. Management status reporting needs improvement. Senior management needs to know how the risks of the project are being managed. Without appropriate information, the governance structure may not be able to provide adequate oversight to ensure the success of the project. For example, a scorecard or dashboard could clearly indicate where ITR is according to the plan, in terms of time, delivery and budget. Recommendation 2. The IPO should develop and present on a regular basis management status reports, concentrating on achievements, financial status and issues for the various committees. Management Action Plan Responsible Position: Director, IT Renewal. We agree with the recommendation. ITR Governance has been streamlined due to the establishment of the new ITR Program Board, meetings are held monthly. Project implementation plans will be presented at the Board level in order to communicate and discuss issues, risks, accomplishments, financial information, and next steps based on the master integrated schedule. The Project Team has committed to working closer with the Finance Directorate to consolidate and implement additional reporting at the overall Program level (see Recommendation# 9). Executive reporting will evolve to meet expectations of the new ITR Program Board but will include status against the master integrated schedule, accomplishments, risk, issues, and financial updates. A management dashboard report for the CIO has been refined and has been in use since January Finally, the Program office will ensure increased visibility at the IM/IT committee attending at least every 2nd meeting (meetings are held every 3 weeks). It is expected that these measures will be complete by the end of May, Observations and Recommendations 19

20 Change Management The ITR business case defined the initial scope for each phase of the project. Phase 1A represents the foundation or start-up phase in which the short term priorities for data centre services, ITR organization, overall architecture and portal capabilities are addressed. Phase 1B builds on the foundation of Phase 1A and introduces functionality that is aligned with specific business direction e.g. in support of Elections Canada s strategic goals to provide more e-access and e-registration services. Phase 2 introduces more e-services capabilities. There have been only two decisions on scope so far. One was made internally by the project team on the backup data centre issue, and another one was made in committee when the ITR Steering Committee agreed on the need to freeze the applications. Otherwise, scope has not been an issue. In the future, management will have to confront more difficult questions regarding scope. The scope of the project has been managed satisfactorily to date Benefits Achievement There is a gap in corporate expertise and experience in the area of business case management, especially regarding the outcome and benefits of an IT Renewal project for the organization. The business case was very long (116 pages) and an abridged 26 page version was also produced as a high level overview to facilitate decision-making. A significant amount of time was spent on developing the business case, partly because there were no business case models in Elections Canada and because of the time required to get models from Treasury Board Secretariat and from Elections Ontario. Despite that process, it is still not clear to many what will be the outcome and benefits of the IT Renewal project for Elections Canada. Business agility was listed as one of the benefits but that is difficult to define and measure. At present, benefits and outcomes are not being monitored but there are plans to hire an outcomes manager shortly. A lack of monitoring the defined benefits and outcomes may result in the project not achieving the desired or anticipated benefits for the agency. Recommendation 3. It is recommended that a formal process to identify, measure and monitor benefits and outcomes of the project, be established. Management Action Plan Responsible Position: Director, IT Renewal. We agree with the recommendation and have initiated a formal process to develop more comprehensive benefits and outcomes program, and Program performance measurement. Completion of this action plan will be Fall of Audit of the IT Renewal Integrated Program Office

21 4.2 Meeting Business Requirements It is too early in the overall process to measure the degree to which business requirements have been identified and captured. It is also too early to derive risks for future designs. Therefore, the audit primarily examined the degree to which the business sectors understood how ITR/IPO would benefit them in the future. Business Requirements It was accepted that business owners would not play a large role in ITR Phase 1, since this phase is an infrastructure project. Some business interviewees expressed the concern that information is not easily available about what is going on and how it will affect them. There is uncertainty regarding what the IPO does, and what the ultimate outputs of the ITR will be. Because business sectors have received little formal communication since the business case was created, there has been very little involvement by the business sectors in the ITR initiative. The business sectors in Elections Canada are not sufficiently involved in the ITR project. Lack of involvement from the business user increases the risk that the outcome of IT renewal will not be aligned with the needs of the users. The CIO and the business sponsor should now focus on building business involvement. Recommendation 4. It is recommended that ITR fully engage the business sectors to ensure ITR deliveries ultimately meet the requirements of business users. Management Action Plan Responsible Position: Director, IT Renewal Phase I of the Program involves implementing new underlying Information Technology infrastructure and systems, which is required to prepare for new IT systems functionality, and in consideration of the ITR Integrated Schedule, we do agree with the recommendation. Further, we do believe that the timing is right for IT Renewal to renew the process of closer involvement with the business sectors. In past months, the project has experienced access problems to key EC personnel as a result of constraints due to election readiness activities. We have also been unable to acquire a key project resource (the Organizational Change manager) which is critical to this activity. In both cases, we believe these issues have now been overcome. Resolution of the IT Renewal governance issue allows us to provide more focus because of the participation of all business stakeholders. We would plan to have at least one presentation to each of the Management Teams by October 2009 with additional presentations on an on-going basis. 4. Observations and Recommendations 21

22 Business Solutions Design The business case identified four IT Renewal options and scored them based on risks, benefits and costs. During this process the ITR/IPO conducted informal discussions with Elections Canada business areas in order to understand the priorities and expectations of clients. However, the business sectors do not have a good understanding of how ITR/IPO will benefit them. There is a perception that a lot of money has been spent on project management processes and paper deliverables and little spent on actual deliverables. This shows a misunderstanding of the role of the IPO because two of the deliverables of ITR are a system of documentation and a project management capability. They are intended to be used in all future major projects such as e-registration. A communications strategy is necessary in order to obtain buy-in from the whole organization. Without proper communication of the benefits of IT Renewal, there is a risk that management will not properly align their future business plans with the new architecture. At the time of the audit, it was noted that a formal communications program was to be initiated, with a communications expert to be hired soon. Recommendation 5. It is recommended that a formal communication program be developed for business sectors to inform them of the IT Renewal benefits, outputs and outcomes, and the current status of the renewal project. Management Action Plan Responsible Position : Director, IT Renewal We do agree with the recommendation. Further, we believe that the timing is now right for the Program Team to begin to engage stakeholders in discussions to ensure they begin planning for the use of ITR s deliverables. The Project Team has a Communications Plan which was developed several months ago but not executed due to resource constraints. As one example of a recent engagement, the Field Application Systems Technology project (FAST) have engaged business sectors through a working group to ensure planned changes, upgrades and fallback plans meet their expectations in consideration of electoral event responsibilities. The Web Enablement (Portal) project has recently defined their governance structure which will be used to engage business owners on an on-going basis. In addition, the 2 key Communications tools which have been recently developed ITR Key Messages and an ITR101 presentation will be important vehicles in presenting the project and its deliverables to stakeholders. We plan to undertake an activity to explain the components of ITR and engage business areas to help them plan for the use of ITR deliverables by October 2009, with additional presentations on an on-going basis. 22 Audit of the IT Renewal Integrated Program Office

23 Management of Organizational Development ITR is tasked with inculcating new processes as well as the infrastructure that will support Elections Canada business in the future. On the IT side, the organization has limited capacity with regards to permanent employees. The Information Technology Directorate (ITD) has a high ratio of contractors to employees and this has been the accepted way of doing IT business at Elections Canada ITR management has attempted to staff as many permanent positions as possible to retain knowledge in the organization. IPO resources were intended to mentor other staff on project management techniques. This is not working as well as intended at this point, partly because of problems in staffing IPO positions and partly because of the pressures of getting the project up and running. On the business side, the business representatives indicated that an overall vision for what ITR will enable has not been developed. This visioning process has been adversely affected by other business priorities and the fact that the ITR / business interface requires more work. The CIO indicated that this work is both necessary and anticipated, but also needs a common vocabulary so that both technologists and business specialists can understand each other. IT Renewal is the largest project that Elections Canada has attempted. There is a risk that the organization will not properly evolve to best use the new IT infrastructure and services. Organization change management is required for such a large project. Recommendation 6. It is recommended that the IPO identifies the required organizational changes needed to properly use and support the new IT architecture and services when they will be established. Management Action Plan Responsible Position : CIO We agree with the recommendation. There are significant challenges involved in moving away from the general contractor model, and in the short term we do not believe we will have any flexibility to do so, given the resource constraints on the organization. We will continue over time to add additional resources, but this will be a long term process and will probably only marginally lower our reliance on contracted resources. However, we did recognize this as a risk two (2) years ago and attempted to mitigate by planning for twenty-seven (27) new positions that would be required because of IT Renewal. Since then we have managed to acquire fifteen (15) of these positions and this has resulted in the core of the Technology Planning and Special Projects group acquiring some indeterminate resources. An Organizational Change Manager has now been retained to assist in defining opportunities to make organizational adjustments. As well, the recent establishment of the Architecture and 4. Observations and Recommendations 23

24 Standards Board (ASB) will be a forum to ensure consistent technical architecture and standards are developed and maintained. IT Management will continue to assess and update skill requirements including overall resource capacity and we are planning to participate in the Agency s upcoming A-Base review as well as any future TB HR submission to add required positions. 4.3 Project Management Project Organization and Structure The project management structure has been established. It has been changing dynamically as management reacts to the complexity of the project. The organization of the project was recently changed to transfer the responsibility for some functions from the IT Renewal initiative to other IT Directors. The CIO had identified the concerns of IT employees who saw changes being made without their involvement to the IT environment that they would eventually have to maintain. Standard project documentation for IM/IT is new to EC and work is ongoing to institutionalize it. Policies and standard forms and templates for all documentation were created and most have been reviewed and formally approved. For example, a manual of all templates and policies that have been approved is given to all new employees and contractors. In addition, a standard document storage system has been set up on the corporate shared drive. Project documentation exists and a standard format has been established for such deliverables. IPO staff understands that formal documentation is important and are enforcing requirements. The CIO has initiated a risk mitigation strategy that is realized by the use of the general contractor approach rather than the prime integrator approach. This approach gives the organization more control over how a project is carried out, but the start-up can be more difficult. The organization benefits from greater flexibility in planning, resourcing, scheduling and retaining technical and business knowledge in the organization. Contractors for the IPO have been specifically hired for their expertise. Elections Canada has had difficulty recruiting contractors through the competitive process which is very time consuming. In addition, accommodation for needed staff is a serious constraint on the project. The recently established Request for Proposal Supply Arrangement has helped the situation. The hiring of consultants, who are specialists in procurement to help evaluate bids, has sped up the process. A review of segregation of duties and assignment of financial signing authorities demonstrated that the IPO management is correctly applying the rules for contracting and financial approvals. Positions with these responsibilities are staffed by Elections Canada employees who have completed the required training courses to be delegated signing authorities and the delegations provide for segregation of duties. 24 Audit of the IT Renewal Integrated Program Office

25 Many of the contractors are in senior positions. One of the results is that there are contractors supervising contractors and groups of contractors have been transferred to different Elections Canada managers other than the managers who hired them. Using contractors increases the risk that the agency could lose key corporate knowledge with their departure. Recommendation 7. It is recommended that, before the contractors leave the project, efforts are made to capture lessons learned. Management Action Plan Responsible Position : Director, IT Renewal We accept the recommendation. The Project Team is now using standard templates for development of all deliverables and we have implemented a more frequent series of project walkthroughs which we expect will help ensure contractors pass on their knowledge on an on-going basis to team members. In addition, we will implement a formal process of debriefing, including the capturing of lessons learned, as part of our contract close out procedures. This new process will be in place by August Development Process The development process involves setting up software development and acquisition standards. These standards normally include how business solution designs are developed and approved, monitoring how the product is developed according to the accepted design, ensuring that documentation standards are being met, and ensuring the overall quality of the business solution. It is too early to measure these standards. The development process should be reviewed later once it is in operation. Standard documentation in the area of implementation and fallback plan is not completed. None of the projects has produced implementation or fallback plans for approval. Implementation and fallback plans are important to ensure the continuity of the operation. As an example of risk that might need a fallback plan, the centralized proposed infrastructure called for high-speed Internet to be the standard but it is not available in all areas of the country. Implementation and fallback plans are needed for each stage of the ITR project. This will become even more important as the projects get more complex and dependencies increase. 4. Observations and Recommendations 25

26 Recommendation 8. It is recommended that appropriate implementation and fallback plans are developed for IT Renewal. Management Action Plan Responsible Position: Director, IT Renewal. We accept this recommendation. The overall approach, which we have used, will be to maintain and operate existing systems, then transition over time to the new systems; as a fallback plan the old systems will remain and will eventually be decommissioned when the new systems are fully tested and stable. Customized and gated project management processes have been applied to each of the IT Renewal projects. This improves management and risk associated with projects as they move from phase to phase commensurate with the Master Integrated Schedule. These measures will reduce integration risk, ensure management attention and approval when projects transition between phases, will support decision making at that the board level, and enhance reporting. Each of the ITR streams, or projects, have applied thorough scope management. These plans will contain specific deliverables for implementation and fall back plans to ensure business continuity, a key goal and benefit of IT Renewal. While not all projects within IT Renewal require a fallback plan, those impacting operational delivery will have a formal plan developed by October 2009, which can be monitored at the Program and ITR Program Board level. Project Control Processes Project control processes should be set up to ensure that all project activity has the appropriate management oversight, and can deliver the anticipated product with minimum risk. These processes normally include managing resources, achievements, issues, problems, risks and changes. In some cases, it is too early to measure these standards. Approved Integrated Project Plan An approved Integrated Master Schedule for Phase 1A exists. The schedule is complex with many interdependencies. This is a far more complex project than Elections Canada has ever attempted before. Because of the unexpected election of October 2008 there has been approximately three months of slippage in some areas due to the lack of user availability. The team will walk through the new schedule and try to ensure that all dependencies are recognized and all elements are represented. 26 Audit of the IT Renewal Integrated Program Office

27 Financial Control IT Renewal is a complex multi-year project. There is a need to ensure that proper financial controls are in place to plan and monitor each part of the project, as well as the overall multi-year cost of the IT Renewal project. It was noted that it was difficult to obtain a complete and clear financial report of the current status of the project versus the amount approved for it. Managers, senior management, and committees are not receiving clear financial reports on IT Renewal. Expenditures are not coded in a consistent way that would enable various roll-up reports of the overall project. Without complete, accurate, and timely financial information, it is difficult for management to properly monitor the progress and make decisions supported by complete financial information. Funding is gated and approved by phases. It is also important to be able to monitor the actual costs of the activities that were approved. Recommendation 9. It is recommended that all costs relevant to IT Renewal be reflected in the corporate financial reports in a way that provides information on the current status of the overall project and approved phases, to assist in the planning, monitoring and decision making process. Management Action Plan Responsible Position : Director, IT Renewal. We agree with this recommendation and will be meeting with officials in the finance group to ensure our coding structure will allow appropriate roll-up and reporting at the overall Program level. It is expected these improvements will be implemented for the end of the first fiscal quarter, June 30th Project Change Control A policy has been developed for Change Management within the development process and there is a Change Management form. This process is new and it was too early in the audit to assess this process. Quality Assurance There is a Quality Management Plan. It describes the activities required to ensure that program and project products and services (including process documentation, systems documentation, software and hardware) are correct and to ensure that the program s defined project management, development, and service processes are applied. This document outlines the strategy, approach and plans for ensuring program quality. This process is new and there is not enough evidence of application to assess the process. 4. Observations and Recommendations 27

28 Project Risk Management According to the Risk Assessment Plan, a threat situation is assessed, assigned a severity code and then recorded in the Risk Log. Once recorded, the threat situation is managed as a risk to the project. This process is new and the IPO staff is working to gain its acceptance. The information flows that result in managers providing the same information to several people should be reviewed and streamlined. Key personnel should be aware of what other processes are affected by any changes they receive and they should proactively notify the appropriate people. As this project gets more complex, risk management will become more critical. Recommendation 10. It is recommended that the IPO ensures that the risk management process is fully implemented and operational. Management Action Plan Responsible Position: Director, IT Renewal. We agree with the recommendation. The ITR Program Office has in place a stringent process and procedure for managing risk across the program that is a Risk Management System. In September 2008, a risk consultant was hired who facilitated and produced the ITR Program Risk Level Assessment. The Program has a Risk Manager assigned to work with each project on risk, and manage the existing risk plan and the risk log. Formal risk management takes place during Project status meetings on a bi-weekly basis and during a formal monthly risk management meeting. Mitigation plans for high severity risks are tracked in the integrated master schedule. These are now presented at the ITR Program Board, and the ITR Executive Dashboard contains a risk component. The role of the Program Office also includes provision and education and support to ensure new projects managers understand and adopt the required Program risk management processes. We believe risk management processes are critical and need to be continually improved, however we believe we have already acted on this recommendation, as of March 31, Project Performance Measurement, Monitoring and Reporting There is an outcomes and performance measurement position in the plan which will implement a dashboard and other performance metrics and develop a plan for measuring benefits as outlined in the business case. This process is new and outcomes measures are not yet defined. IPO Problem Management There is an issue management procedure that outlines how to register an issue, how it should be logged, escalation and follow-up procedures and the decision making process. Issue logs are kept and reviewed at the project level and discussed at the working committees. 28 Audit of the IT Renewal Integrated Program Office

29 Procedures and Standards for Acquisition of IT-Related Solutions There is a document to establish the procedures and processes associated with Elections Canada s procurement process within the Information Technology Renewal project. The procurement process was over and above the ITR resource work plan, which created delays. The IPO has engaged procurement experts to assist in reviewing responses to request for proposals. This initiative has reduced the improved the process time. Contracting issues could cause cost overruns and delays. Recommendation 11. It is recommended that a procurement timeline be built in the project plan and be monitored at the program level. Management Action Plan Responsible Position: Director, IT Renewal We agree with the recommendation. At the time of the audit the Program Teams team were finalizing a critical Supply Arrangement which will be used to acquire resources over the duration of the IT Renewal Program. There are no other large procurement measures planned. Now that this supply arrangement is in place procurement procedures have been simplified to streamline execution of resource call-ups. An employee has been tasked with developing and providing oversight for an integrated procurement process at the Program level. We have negotiated service levels with the Agency s procurement division and our project managers have been instructed to factor appropriate timelines into their schedules. Overall monitoring of procurement performance at the Program level is currently being performed. Issues related to procurement service levels will be addressed with the Agency s procurement division and reporting will be integrated into the overall executive dashboard by June Technology and Infrastructure Infrastructure Management Architectural Environment One of the main objectives of the ITR project is to design and architect the appropriate IT environment. Past architectural choices have resulted in 12 to 14 different technology silos, since each business area has until now controlled its own architecture and the budget for it. The ITR architecture group has been tasked with developing an enterprise-wide architectural framework and governance. Agreement has been reached that an Architecture and Standards Board (ASB) will be created and that all IM/IT initiatives will be subject to its technical oversight. Although the membership of the ASB has been determined, it is too soon to evaluate its effectiveness. 4. Observations and Recommendations 29

30 Development of an architectural framework has started but it will require strong senior management support to succeed. The new IT architecture will require that the future application comply with its standards. Senior management should decide how to ensure that no procurement of technology can be done without assurance that the purchase meets Elections Canada technical standards. This may mean implementing a process in which a Chief Architect position or role would review planned purchases for compliance with standards before approving them. Recommendation 12. It is recommended that senior management support the Architecture and Standards Board s (ASB) work and get EXCOM s endorsement for its mandate, and ensure that the ASB s standards and decisions apply to all IM/IT initiatives. Management Action Plan Responsible Position : CIO We agree with the recommendation. The Terms of Reference (TOR) of the IM/IT Committee require that any new development activity seek ASB endorsement before IM/IT approval. The Terms of Reference have been approved by both EXCOM and the IM/IT Committee. The IT Directorate management team also have a collective responsibility to ensure that the mandate of the Architecture and Standards Board (ASB) is supported. We believe that we have taken necessary action to support this recommendation as of March 31, It is recommended that a process be established to ensure that all new developments are reviewed to verify compliance with the Standards of the new IT Architecture before being approved. Management Action Plan Responsible Position: CIO We agree with the recommendation. The Terms of Reference for the IM/IT Committee require that ASB approval be sought before new projects are authorized. The CIO and IT Management Team, having established the ASB, accept their responsibility for ensuring that internal activities also have ASB approval before they can commence. It is a priority for the ASB to identify and confirm the current set of IM/IT standards and ensure the processes are in place to adjust these on an on-going basis. 30 Audit of the IT Renewal Integrated Program Office

31 Since the ASB is a relatively new function, it is natural to expect that these actions will take time to mature, however we expect to have stable processes by late Fall Security Architecture Security has been raised as a concern in relation to the ITR project. There is no plan in place regarding the security architecture for the ITR project though it is understood that all ITR components will have to do a certification and accreditation. The ITR project must build in the security requirements for the field applications being centralized, and it must consider future requirements for E-Registration and other applications requiring a high level of security. There is a risk that the new IT infrastructure will not meet the security requirements of the Agency. Appropriate security is essential to deliver the business solutions. The privacy, IT, business and legal units need to consult together to develop their security requirements and make a decision. Recommendation 14. It is recommended that a plan for achieving security and establishing security requirements, for applications which will use the new infrastructure, be developed and implemented. Management Action Plan Responsible Position: CIO We agree. As we develop the infrastructure to facilitate web-enabled access for applications, both privacy and associated security concerns must be taken into account. The Government Security Policy (GSP) establishes the baseline security requirements for CG Departments and Agencies in order to safeguard employees and assets and assure the continued delivery of GC services. The GSP is supported by operational standards and technical documentation such as the Operational Security Standard: Management of Information Technology Security (MITS). The MITS standard establishes the baseline requirements for managing IT Security within the Government of Canada. The IT Directorate concluded its MITS project in December 2008, with the Governance and Framework elements endorsed by Elections Canada s Senior Management. Operationalization of the IT Security Program, developed as part of the MITS project, will continue throughout FY 2009/10 and FY 2010/11. To date, the Departmental IT Security Policy has been endorsed by the IM/IT Committee as of February 2009 and is pending approval from EXCOM in the coming months. An IT Security end user awareness program has been developed and will be launched during FY 2009/10. Network security zoning activities, as per CSE's ITSG-22, are being undertaken as part of the Elections Canada Data Centre relocation project. 4. Observations and Recommendations 31

32 Other components of the IT Renewal Program have completed a thorough network and computing security architecture as part of their implementation plans. Initiatives such as Field Applications Systems Technology and the Portal are beginning the process of conducting Privacy Impact Studies to determine the requirements which must be factored into any design activity. Business requirements for application(s) authentication solutions are still in discussion, from both strategic and tactical perspectives. Design of the authentication solution is predicated on the clarification of various legal and legislative frameworks, and is anticipated to begin in August The CIO in conjunction with the ITD Security Group will continue to provide the oversight to ensure that security plans are implemented consistent with appropriate standards, this will apply to IT Operations, all Programs and Projects Technology Transition Technology Transition Service Oriented Architecture (SOA) is a new model for Elections Canada. The plan for conversion to the new SOA model starts with critical systems first, but that may pose a large risk. While it is too early to assess the transition of technology, it will likely entail a high risk. As the project moves forward, this should remain an item to watch. Telecommunications Project ITR project plans assume the use of a high-speed network linking to offices of all Returning Officers where in reality the available network speeds vary considerably across the country. The bandwidth available to support the new thin client field applications will be determined as part of the Telecommunications Infrastructure Evolution (TIE) project. The Telecommunications Infrastructure Evolution (TIE) project s principal objective is to establish rapidly deployable high-speed connectivity at local offices during an electoral event. The project is now reviewing options for high-speed communications nationally. The data centres being implemented will depend on connectivity between them and to the offices of the Returning Officers, and those requirements will also have to be taken into account in the TIE project. Coordination with the TIE project is required to establish the minimum applications bandwidth requirements. Planning for transition and providing resources to support new technologies are areas that require attention. Delays may result if these are not in place to successfully convert to the new architecture. 32 Audit of the IT Renewal Integrated Program Office

33 Recommendation 15. It is recommended that ITR project management ensure coordination between the Telecommunications Infrastructure Evolution project and the applications and data centre projects capacity requirements, and create a plan for telecommunications in areas where high speed bandwidth may not be available. Management Action Plan Responsible Position: Director, IT Renewal We agree with the recommendation. As part of the Field Application Technology Systems project (FAST), network architecture is in the process of being developed. High speed broadband unavailability issues and associated risks are now being identified and mitigated. A mitigation plan will be available by summer 2009 so that it can be factored into the design phase of the FAST project. 4. Observations and Recommendations 33

34

35 5. Conclusion IT is a key enabler, essential to the delivery of Elections Canada s Strategic Plan. The ITR project is one of the most significant information technology initiatives to be carried out by Elections Canada. As a result, risk to Elections Canada is heightened by the fact that the project is a most complex endeavour, and that the final impact on Elections Canada s operations will be significant. Management must ensure key risks are addressed as ITR moves forward. Project management and controls are largely in place and working, with some exceptions. In some cases, since this is an audit of a system under development, it is too early to assess if the processes are working as intended. Key elements that will require attention as the project moves forward include: strengthening the governance structure, improving reports on project to senior management and the committee improving communication with business users increasing the involvement of business users improving overall financial reports on the project completing the implementation of the risk management process developing a plan for security requirements ensuring coordination with the Telecommunications Infrastructure Evolution Project. The IPO provides the structure to manage the various projects included in the IT Renewal. It is important to closely monitor progress as IT Renewal moves forward. 5. Conclusion 35

36

37 Appendix A Detailed Audit Methodology This section outlines the approach to the methodology used in this assignment. Our structure includes the following four domains: Project Governance Risk, Business Risk, Project Risk, and Technology and Infrastructure Risk. 1 Area of Project Risk Audit Criteria Project Governance Risk This class of risk pertains to the presence of a well-defined structure of roles, responsibilities and authorities within which the project operates, and within which all major decisions concerning the scope and objectives of the project, including changes to the same, are made. Senior Management Control Framework. Senior management should: define the relationship of the project to strategic plans; establish the assignment of responsibility, including project oversight; define the roles of key organizations and people; and, decide on the flow of management information. Management of Scope and Change. Senior and project management should establish processes to allow the project to adapt to changing internal and external conditions. Investment Management and Benefits Achievement. Senior and project management should define expected costs and benefits through a business case, and measure project benefits realized by the organization as they are achieved through the project. Business Risk This class of risk pertains to the clarity and stability of the business rules and processes from which the system's requirements will be derived, to the integrity and robustness of the design that will be prepared to address those requirements, and to the capacity of the organization to organize itself for and to manage the changes that the introduction of a new system implies. Business Requirements Management. Project and functional management should ensure the specification of business requirements adequately meet the functional requirements and achieve the stated benefits. Business Solution Analysis. Project and functional management should ensure there is a process in place to translate the business requirements into the business solution. Management of Organizational Development. Project and functional management should address the impact of the project on the major business processes of the sponsoring organization and the ability of the organization to deal with the overall change. 1 Note 1: Taken in the context of the IPO and not the IT organization as a whole Appendix A - Detailed Audit Methodology 37

38 Area of Project Risk Audit Criteria Project Risk This class of risk pertains to the internal organization and management of the project, and to its monitoring, reporting, control and communications functions. This class of risk also considers the tools, techniques, methods and procedures needed to do the actual work of the project: to understand the requirements that have to be addressed, and based on that understanding to design, develop, implement and make operational a relevant, reliable, usable system. Technology and Infrastructure Risk This class of risk pertains to the degree of inherent risk in the technology platforms chosen to support the system. Newer and less widely-proven platforms pose substantially higher risk than do mature and widelyused platforms. Not only is there a greater probability of a flaw in the platform, know-how to deal with flaws is rare. This class also pertains to the transition of the application into the infrastructure within which it will operate. Newly developed and implemented infrastructures pose more risk than do a structured mature ones. Project Organization and Structure. Project and technical management should define the roles and responsibilities of each major organizational component of the project structure, and provide for adequate staffing. Development Process. Project and technical management should adopt a formal development process with milestone deliverables. Project Control Processes. Project management should have a standard approach to project control. Infrastructure Management. Project and technical management should ensure that the technical solution conforms to the organizations technical standards and methods and technology environment. Project and technical management should measure the impact the project will have on this infrastructure. Technology Transition. Project and technical management should address the readiness of the organization to deal with the new technology, overall technology configuration management, and the organization s ability to offer support (short and long range). 38 Audit of the IT Renewal Integrated Program Office