Governance of Evaluation, Proof of Concept and Pilot Projects

Transcription

1 Governance of Evaluation, Proof of Concept and Pilot Projects (OCIO) Government of Newfoundland & Labrador Prepared By: (PMO)

2 Table of Contents Definition of an Evaluation Example Concept Initiation and Planning Analysis and Evaluation Close Out Duration Critical Successful Factors Control Documents RACI Evaluation Workflow Diagram Definition of a Proof of Concept Example Initiation Planning and Analysis Design to Close Out Duration Critical Success Factors Control Documents RACI Proof of Concept Workflow Diagram Definition of a Pilot Example Concept Initiation to Close Duration Critical Success Factors Control Documents Pilot Project Methodology Pilot Initiative Workflow Diagram... 13

3 1. This document outlines the governance for Evaluations, Proof of Concepts and Pilot Projects for the OCIO. The objective of governance is to ensure consistency, accountability and proper authorization to undertake any of the above activities. It ensures that the structure, oversight and management processes are in place for the delivery of the expected benefits from an Evaluation, Proof of Concept or Pilot Project. Accountability is one of the key concerns as it needs to address the needs of all stakeholders. 2. Definition of an Evaluation 1 The purpose of an evaluation is to perform an assessment that seeks to determine if a technology or a combination of technologies is the best possible fit for the needs of a given business problem. Based on a prepared list of criteria along with some practical experimentation, a technology evaluation makes it possible to determine if a technology would be helpful in solving the business problem identified. The actual evaluation is limited to a very small number of subject matter experts within a particular team. 2.1 Example An example of an evaluation would be the Entrust Two Factor Authentication product. This product was evaluated to see if it would be a good fit for the HRMS project which had a requirement for a Two Factor Authentication technology. Entrust provided the OCIO with a 90 day evaluation license of their product. The product was installed and evaluated by a couple of subject matter experts, after which the environment was de commissioned. 2.2 Concept A product evaluation can originate from various sources. It can be driven by the need of a project in flight, a perceived use in an upcoming project, or a desire from clients or other branches within the OCIO to evaluate a product for potential future adoption. In the future, a product evaluation could also be undertaken by the Enterprise Architecture Division in accordance with its Enterprise Architecture roadmap, currently being developed. All requests for evaluations must identify the requestor, the authorization for such a request, the sponsor (owner), and source of funding if there are any costs associated with the evaluation. If funding is required, funds must be secured before proceeding. These requests should also be accompanied by a business case. Requests from clients and other Branches 1 Defined by wisegeek ( Page 2

4 within the OCIO should be channeled through Client Services. This applies to evaluations that may be done by individual branches utilizing their own internal resources. Requests for evaluation will be submitted to the (PMO) where it will be assigned a project number (DTC) and have the appropriate ISC folders set up. Requests will be forwarded to the Executive Director of the Branch that will be doing the evaluation for approval to proceed. 2.3 Initiation and Planning The Initiation and Planning phase of the evaluation identifies the resources required. In planning for the evaluation, all resources, including those from other branches, should be identified. This resource identification will be useful for Solution Delivery as well as other OCIO branches in managing its resource planning and capacity, and to determine the impact on operational requirements and projects. These resources should be included in an evaluation plan which also identifies the start and end dates, the key stakeholders, the key milestones and any dependencies and constraints. In this phase the Sponsor is confirmed and the goals, objectives and scope are defined and agreed upon in a Project Charter. The Project Charter does not have to be completed in the level of detail as in a project, but with the relevant information to obtain the approval of the Project Sponsor to proceed with the evaluation. It is also important that out of scope items are clearly documented and included in the Project Charter. A Steering/Working Committee is established with the Chair being the Sponsor. Depending on the complexity of the evaluation or the impact it has on a project in flight, a Project Manager may be assigned. Otherwise, an internal resource will be assigned to provide oversight. The Project Charter should include the engagement and communication to the other branches of the OCIO and other stakeholders. Monthly reporting as a minimum will be submitted to the PMO and a Dashboard will be created and distributed to the OCIO management. 2.4 Analysis and Evaluation In this phase, the analysis and evaluation take place in accordance with the scope of work as outlined in the Project Charter. Any changes to the scope of work or time frames will follow the normal change control procedures. The project manager must then ensure that all documentation has been approved and stored in TRIM in the assigned folder. Page 3

5 2.5 Close Out In this phase, an Evaluation Report that has been approved by the Director of Enterprise Architecture and accepted by the Sponsor will be compiled and the final results communicated to all key stakeholders. 2.6 Duration The total duration of an evaluation shall be no more than 45 business days. The duration of the product installation, configuration and usage should be no more than 35 business days; and The written evaluation of the product, including all documentation, should take no more than 10 days. 2.7 Critical Successful Factors A Documented Request for an Evaluation; A Sponsor/Owner; Steering / Working Committee; Commitment of Key Stakeholders; Regular Status Reporting; and Approval and Acceptance of Final Report. 2.8 Control Documents Request for an Evaluation; Status Reports; and Steering / Working Committee Meeting Minutes. 2.9 RACI Evaluation RACI Chart Task # Task Description EA PMO Executive Director or SLT Client Services ISC/IM AS OPS Project Sponsor 1.1 Identify new Evaluation R I R, C R R, C R, C R, C A 1.2 Submit new Evaluation request with Business Case and Scope R, A I I R R R R C Document 1.3 Confirm funding for new Evaluation I R A I 1.4 Acknowledge Evaluation Requests I R, A I I 1.5 Prioritize Evaluation Requests C R, A C C C C I Page 4

6 Evaluation RACI Chart Task # Task Description EA PMO Executive Director or SLT Client Services ISC/IM AS OPS Project Sponsor 1.6 Assign Project Number and create I R, A R new TRIM folder 2.1 Create the Evaluation Plan R, A I C C C C 2.2 Create Project Charter R, A I I C 2.3 Establish Steering/Working I I R, A I C Committee (chaired by sponsor) 3.1 Conduct Analysis and Evaluation R, A I I I 3.2 Ensure all documentation and deliverables are approved and submitted to TRIM 4.1 Create Evaluation Report and Communicate Results R, A I I I I I I I R, A I I I RACI R = Responsible A = Accountable C = Consulted I = Informed Page 5

7 2.10 Evaluation Workflow Diagram Government of Newfoundland and Labrador Page 6

8 3. Definition of a Proof of Concept 2 Government of Newfoundland and Labrador A Proof of Concept is the delivery of a working system to prove that the technology works and functions as intended. It is usually small and may or may not include all functionality; instead, it refers to a partial solution that involves a relatively small number of users. Once complete, the PoC should be dismantled and considered complete after the results have been documented. 3.1 Example An example of a PoC would be the OCIO Wireless Network. This PoC resulted in the installation of a wireless infrastructure into three boardrooms within the Higgins Line building. There were a small number of executive members that were given access to this network for testing after which a lessons learned document was created. That document along with other documented recommendations on how to proceed with a full wireless network implementation were delivered at the end of the PoC. Concept A Proof of Concept can be the result of the need to test a product to ensure that it is performing as it is marketed in a controlled environment. In most cases PoCs originate from other branches of the OCIO or the SLT. All PoCs must have a Business Case and these should be submitted to the PMO by Client Services including requests from the OCIO Executives. If the POC is not a funded project, then the funding must be identified and funding secured prior to initiating a PoC Project. The PMO will assign a project number (DTC) and set up the appropriate ISC folders. The PoC shall follow the normal SDLC and the deliverables for each phase, which will be determined in the planning exercise. 3.2 Initiation In this phase, the funding source is confirmed by the PMO, a Project Initiation Document (PID) is created, the Solution Delivery Director, Delivery Manager, and Project Manager are assigned and the engagement of the other branches of the OCIO is completed. The Project Sponsor, Owner, Vendors and key stakeholders are identified. A preliminary scope document is created and high level costs and time frames are estimated. Resources are identified and, if professional services are required to supplement OCIO staff, then a Project Opportunity (PRO) or Staff Augmentation Request (SAR) will be issued subject to funding availability. 2 Defined by Claromentis ( Page 7

9 3.3 Planning and Analysis A Project Manager will be assigned from the Initiation phase and a detailed plan will be developed. At a minimum, a project plan, a resource plan, a stakeholder analysis and a Project Charter should be delivered from the planning exercise. The Project Charter must be signed by the Project Sponsor before any further work is done and would include the Communication Plan, the final agreed scope of work, the out of scope items, cost estimates and time frames. In the analysis, the Project Manager would have determined the tasks that need to be done in the planning of the PoC. These tasks would have been reviewed with the project sponsor and included in the Project Charter. The deliverables, as identified in the SDLC, may not apply to all PoCs, thus a preliminary list of deliverables specific to the PoC should be created during the planning process. These deliverables will then be finalized in the Design to Close phases of the PoC. It should also be noted that there are no mandatory deliverables for PoC projects, as it will depend on the type of PoC; however, once the list of deliverables is finalized the team must complete them. 3.4 Design to Close Out In the Design to Close phases of the PoC, the Project Manager, in consultation with the project team and sponsor, will finalize the deliverables that apply. The rigor that applies to a project may not apply to a PoC; however, the details will be determined in consultation with the stakeholders, which includes, at a minimum, the OCIO branches. 3.5 Duration The total duration of a PoC shall be no more than 100 business days. Duration of the product installation, configuration and usage should be no more than 90 business days; and The written evaluation of the product, including all documentation, should take no more than 10 business days. 3.6 Critical Success Factors Business Case; Approved Funding; Project Sponsor; Commitment of Sponsor/Owner; Scope Control; Steering / Working Committee; Management of Risks & Issues; and Page 8

10 Compliance to Processes. 3.7 Control Documents Project Charter; Stakeholder Analysis; Project Change Requests; Status Reports; Project Plan; and 3.8 RACI Proof of Concept RACI Chart Task # Task Description EA PMO Executive Director SLT Client Services ISC/IM Project Manager AS OPS Project Sponsor 1.1 Identify new PoC request 1.2 Develop Business Case and submit with new PoC request 1.3 Confirm funding for new PoC 1.4 Acknowledge PoC Requests 1.5 Assign Project Number and create new TRIM folder 2.1 Create Project Initiation Document (PID) 2.2 Create Preliminary Scope Document R, A I R, C R, C R R, C R, C R, C C R, A I I R, I R R R R I I R, A I I I I I I I R, A I R, A I R, I I I I R I I I I R, A 2.3 Determine if professional services are required I I I C I I R, A I I I 3.1 Conduct PoC Planning C C C R, A C C C 4.1 Finalize and complete all applicable SDLC deliverables C C R, A C 4.2 Document all findings and submit all deliverables to TRIM R, A 4.3 Finalize results of PoC and communicate to all stakeholders I I I I I I R, A I I I Page 9

11 3.9 Proof of Concept Workflow Diagram Government of Newfoundland and Labrador Page 10

12 4. Definition of a Pilot 3 A pilot is an initial prototype of a system into production, targeting a limited scope of the intended final solution. The purpose of a pilot project is to test, often in a production environment, whether the system is working as it was designed while limiting business exposure. 4.1 Example An example of pilot would be the Windows 7 Upgrade Pilot. This pilot included a small user base within the OICO to help the OCIO discover any unknown impacts to existing user hardware and desktop applications. The results of this pilot were used to determine challenges expected to be encountered during a corporate deployment and help develop solutions to these challenges. 4.2 Concept A pilot is a project and is subject to all the rigors of the Project Management Life Cycle as well as the OCIO s SDLC. At the end of a pilot, all the deliverables would have been met. The key difference is that in a pilot, the solution is rolled out into production to a limited set of users. All pilot projects will start with a Business Case submitted to the PMO from Client Services. These projects would have been on the OCIO s funded project list. If a request for a pilot is received and it is not funded, then the funding must be secured prior to project initiation. 4.3 Initiation to Close The activities from Initiation to Close will follow the OCIO SDLC. documents and processes can be found on the PMO website. All deliverables, control 4.4 Duration The total duration is generally no more than 180 business days depending on scope of work and business impacts. Duration of the product installation, configuration and usage should be no more than 170 business days depending on scope of work and business impacts; and The written recommendation as to how to proceed, including all documentation, should take no more than 10 business days to compile. 3 As defined Yahoo Answers - Neeraj Bihari Page 11

13 4.5 Critical Success Factors Business Case; Approved Funding; Project Sponsor; Commitment of Sponsor; Qualified Project Manager; Approved Project Charter; Project Plan; Steering Committee; and Compliance with the SDLC. 4.6 Control Documents A complete list of control documents that may be applicable to pilot projects can be viewed on the OCIO website at: Pilot Project Methodology Pilot projects must follow the normal SDLC methodology as outlined on the OCIO website. For a detailed breakdown of the SDLC methodology and related processes, please visit the following URL: Page 12

14 4.8 Pilot Initiative Workflow Diagram Government of Newfoundland and Labrador Page 13