How To Write Project Management Guidelines

Transcription

1 Project Management Guidelines July 2013

2 FOREWORD I have pleasure to present these BDT Project Management Guidelines 2013 that now supersedes the April 2008 version. These Guidelines are drawn up in accordance with Resolution 17 (Rev. Hyderabad, 2010) adopted by the World Telecommunication Development Conference addressing implementation of regional approved initiatives, at the national, regional, interregional and global levels. It also implements the Plenipotentiary Conference Resolution 157 (Rev. Guadalajara, 2010), which focuses on strengthening the project execution function of the ITU. These Resolutions also noted the need to maintain a level of expertise to permit the BDT to manage projects effectively and in a timely and efficient manner. These Guidelines aim to develop a common language and understanding of the main phases of the life cycle of a project, as well as the processes involved in and the key aspects of project design and implementation, while providing templates that will facilitate the work of project managers These Guidelines take into account the ongoing efforts of the BDT in the adoption of the results-based management approach, and will improve the implementation of projects by the BDT in close collaboration with its various partners. The drafting Guidelines also took into account the latest international standards for project management, such as "ISO 21500" which was announced in October The Guidelines and Annex serve as reference for all those involved in the implementation of projects in the BDT. This will no doubt improve our efficiency and effectiveness in implementing, monitoring and evaluating projects. Brahima Sanou, Director Telecommunication Development Bureau 2

3 INTRODUCTION The Plenipotentiary Conference in 2006 approved Resolution 157, which was subsequently revised by the Plenipotentiary Conference in Resolution 157 (Rev. Guadalajara, 2010) commits the ITU to strengthen its project execution functions. Resolves 1 and 2 of Resolution 157 (Rev. Guadalajara, 2010) provide, respectively, for the review of the experience of ITU-D in discharging its responsibility for implementing projects by identifying lessons learned and by developing a strategy for strengthening this function in the future, and for the review of best practices in the area of technical cooperation (which includes project management) within the United Nations systems and other organizations external to the United Nations, and to adapt such practices to the circumstances prevailing within the ITU. With respect to the aforementioned Resolves of Resolution 157 and in line with the trends in project management, from an activity-based approach to results-based methodology, the BDT carried out a thorough review on project management practices of the United Nations, other development agencies and private sector organizations. The findings of the study, together with the assessment of BDT s procedures in project management, demonstrated that Results-Based Management ( RBM ) approach should be reinforced in strengthening of the project execution function of ITU. In this respect, in 2008 the BDT developed Project Management Guidelines which take into account RBM best practices, methodologies and tools of other organizations as well as the specific kind of ITU-D project portfolio. In 2011, a restructuring took place which redefined the roles and responsibilities of the Project Support Division (PRJ). Within the Project Support and Knowledge Management Department (PKM), the PRJ strengthens the project execution function of the Telecommunication Development Bureau (BDT), is responsible for providing support for the development of regional initiatives and other projects, coordinates the drafting and finalization of project documents, directs the development of the supporting operational, reporting, and evaluation frameworks (including related procedures, best practices, systems, databases, tools, templates, reporting documents and performance indicators) for the consistent monitoring, implementation, evaluation and reporting of all BDT projects, monitors and evaluates project implementation, carries out and reports on impact assessments of BDT projects, coordinates the preparation of documentation for the closure of projects with partners, submits and ensures timely definitive closure of projects, participates, provides inputs and contributes to the processing of outputs of connect the world series of events, and develops guidelines for the effective project execution at all stages of the project cycle. In this context, the PRJ Division has updated the Project Management Guidelines published in 2008 to reflect the restructuring and best practices contained in the latest project management standards (ISO 21500:2012 launched on October 2012 and PMBoK 5 th Edition announced on December 2012). These Guidelines aim to provide clarification and guidance on a selected set of tools to be used by project managers based on the results-based management approach and project management lifecycle, the latter of which consists of three phases: Formulation, Implementation and Evaluation. 3

4 Table of Contents 1 PROJECT MANAGEMENT LIFE CYCLE AT GLANCE RBM AND PROJECT MANAGEMENT Results-Based Management Project management aligned on RBM approach FORMULATION PHASE Identification Fundraising Finalization IMPLEMENTATION PHASE Planning Work Breakdown Structure Project Schedule Procurement Plan Quality Plan Risk Plan Communication Plan Executing and Monitoring Build Deliverables Executing Reporting Closing Project operationally completed Project financially completed EVALUATION PHASE Post Implementation Review Impact Analysis and Sustainability Lessons Learned and Best Practices IMPROVEMENT PROCESS ANNEX : TEMPLATES Logical Framework Template Project Document Template Project Management Plan Template

5 7.4 Project Manager Log Template Meeting Minutes Template Risk Monitoring Form Change Request Form Progress Report Template Assessment / Feedback Report Template Closure Report Template Transfer of Title to Equipment Post Implementation Review Template Impact Analysis Report Template Lessons Learned Report Template

6 Figures and Tables Figure 1: Project Management Life Cycle Figure 2: RBM Approach for Project Management Figure 3: Formulation Phase Figure 4: Implementation Phase Figure 5: Evaluation Phase Figure 6: Learned Lessons and Best Practices Table 1: Project Management Standards Table 2: SMART Description 6

7 Project Management Guidelines PROJECT MANAGEMENT LIFE CYCLE AT GLANCE A Project Lifecycle is defined as a series of phases undertaken to provide a proper project outcome. Each phase includes a number of activities and supporting tools to be utilized in order to achieve the best results. The project management standards 1 ISO 21500:2012, PMBoK 5 th Edition (Project Management Institute) and Prince 2 suggest managing project under five groups of processes (Table 1). Taking into account the recommendations of standards and specifications of the BDT portfolio of projects, we have grouped these processes in two phases titled «Formulation Phase» and «Implementation Phase». Given that these standards do not include the evaluation and impact analysis processes, the project life cycle has been completed with an «Evaluation Phase». Thus, these Guidelines propose a Project Management Life Cycle grouped into three phases, each with three distinct stages, as depicted the diagram below: Figure 1: Project Management Life Cycle 3- Lessons Learned and Best Practices 1-Identification 2-Impact Analysis 2-Fundraising 1-Post Implementa tion Review 3-Finalization 3-Closing 1-Planning 2-Executing & Monitoring The first phase focuses on identification of key problems and needs of beneficiaries through a situation analysis, and clarification of roles and responsibilities of parties involved. This phase presents stages to be used to prepare and develop a project proposal. This phase also provides information on how to facilitate the fundraising process in coordination with the partnership and resource mobilization strategies for approaching donors. Once funds are secured, the responsible project manager should complete the Project Document (ProDoc). 1 Comparing PMBOK Guide 5 th Edition and ISO STS Sauter Training & Simulation SA. December

8 In the remainder of this document, the project manager is responsible of the project implementation from ProDoc until the submission of closure report, and the project coordinator is a member of the PRJ team responsible for supporting the project manager, as well as monitoring and evaluation of projects. When the ProDoc is approved and signed by the stakeholders, the project manager is responsible for the second phase (implementation); that is to plan, execute, control and close all project activities to achieve the objectives. The first stage is essential for the planning of all activities to achieve the expected results, and those related to risk communication and acceptances. Proper planning is the critical factor for the success of the project. The next stage is to achieve the planned activities and monitor their implementation by different stakeholders. The last stage of this phase is related to the closure of all project activities and the transfer of results to those concerned. The evaluation phase is performed by project coordinator. The first stage will be the post-implementation review that should be performed immediately after the implementation phase is finished; in this review, the achievement of the objectives and outcomes will be measured. The second stage is estimated to take place between 1 and 3 years after the project finishes, and will analyze the impact on the environment. The last stage is to take into account performance indicators already established and to consolidate the lessons learned in order to provide best practices for future projects. Table 1: Project Management Standards ISO-21500:2012 October 2012 PMBoK 5th Edition December 2012 Prince2 Initiating Initiating Starting up a Project Planning Planning Initiating a Project Implementing Executing Managing a Stage Boundary Controlling Monitoring and Controlling Controlling a Stage Closing Closing Closing a Project 8

9 2 RBM AND PROJECT MANAGEMENT 2.1 Results-Based Management In an organization, Results-Based Management (RBM) aims to ensure that activities and resources contribute to the achievement of desired results, in terms of outputs, outcomes and the expected impact. A key component of RBM is monitoring the performance that helps to measure objectively how well the results are being achieved, and report on measures taken to improve them. «It is said that if you do not know where you are going to, any road will take you there. This lack of direction is what Results-Based Management is supposed to avoid. It is about choosing a direction and destination first, deciding on the route and the milestones, checking progress with a map and making direction adjustments as required in order to achieve the desired objectives.» 2 The various stages in the implementation of Results-Based Management (RBM) and Results-Based Budget (RBB) in ITU include the following: RBB and RBM were first introduced in ITU in 2001 for the budget; Since then, RBM and its components have been progressively introduced and extended to other planning tools (Operational planning, Strategic planning, Financial planning); The adoption of Resolution 71 (Rev. Guadalajara, 2010), regarding the Strategic Plan of the Union for , with a 100% compliant RBM structure was a significant achievement to move towards Enterprise Performance Management (EPM) for ITU; During the different phases of the life cycle of a project, the RBM approach ensures that the components necessary for the achievement of expected results will be available. Monitoring, in this case, implies the definition of Key Performance Indicators (KPIs) that will help measure the achievement of the goals initially assigned to the project through it implementation. 2.2 Project management aligned on RBM approach For application of the RBM approach to project management, it is suggested to adopt the following terminology based on the definitions proposed by UN Development Group: «Results-based management (RBM) Results-based management is a management strategy by which all actors, contributing directly or indirectly to achieving a set of results, ensure that their processes, products and services contribute to the desired results (outputs, outcomes and higher level goals or impact) and use information and evidence on actual results to inform decision making on the design, resourcing and delivery of the project and activities as well as for accountability and reporting. Results Results are changes in a state or condition that derive from a cause-and-effect relationship. There are three types of such changes - outputs, outcomes and impact - that can be set in motion by a development intervention. The changes can be intended or unintended, positive and/or negative. 2 Results Based Programming, Management and Monitoring (RBM) approach as applied at UNESCO- Guiding Principles June

10 Impact Impact implies changes in people s lives. This might include changes in knowledge, skill, behavior, health or living conditions for children, adults, families or communities. Such changes are positive or negative long- term effects on identifiable population groups produced by a development intervention, directly or indirectly, intended or unintended. These effects can be economic, socio-cultural, institutional, environmental or technological. Positive impacts should have some relationship to the Millennium Development Goals (MDGs), internationally-agreed development goals, national development goals (as well as human rights as enshrined in constitutions), and national commitments to international conventions and treaties. Goal A specific end result desired or expected to occur as a consequence, at least in part, of an intervention or activity. It is the higher order objective that will assure national capacity building to which a development intervention is intended to contribute. Outcome Outcomes represent changes in the institutional and behavioral capacities for development conditions that occur between the completion of outputs and the achievement of goals. Outputs Outputs are changes in skills or abilities and capacities of individuals or institutions, or the availability of new products and services that result from the completion of activities within a development intervention within the control of the organization. They are achieved with the resources provided and within the time period specified. Activity Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources, are mobilized to produce specific outputs. Inputs The financial, human, material, technological and information resources which are used for development interventions. Performance indicator A performance indicator is a unit of measurement that specifies what is to be measured along a scale or dimension but does not indicate the direction or change. Performance indicators are a qualitative or quantitative means of measuring an output or outcome, with the intention of gauging the performance of a project or investment.» 3 3 UNDG-Results-Based Management Handbook-Harmonizing RBM concepts and approaches for improved development results at country level October

11 The pyramid below describes the adaptation of the RBM approach to Project Management and defines the various features of a project, from the resources to the expected impact. Figure 2:RBM approach for Project Management 4 IMPACT Measures the long-term and widespread development changes in society, economy and environment to which the project contributes Objective An overall desired achievement involving a process of change and aimed at meeting needs of end-users within a given period of time. Outputs/Results Final products or services delivered to beneficiaries Activities Activities taken to transform resources/inputs into outputs/results Resources/Inputs Personnel & resources necessary for producing outputs/results 4 ITU - Result-Based Management Result-Based Budget Training - Seminar - PwC - June

12 3 FORMULATION PHASE This phase integrates the general approach of identification of needs to finalize the project document through the stage of fundraising. Depending on the context and the kind of project, it will be necessary to scroll through all the stages, or to go directly to the finalization of the Project Document. Figure 3: Formulation Phase 1-Identification 2-Fundraising 3-Finalization 3.1 Identification During this first stage, the project proponent or the project manager should prepare a «Concept Note» 5 shared with stakeholders and donors. to be The identification stage intends to find answers to the question what does the project need to achieve? The first stage in project management is: To perform a situation analysis in a particular political and socio-economic context; To identify what are the problems on the ground and needs of beneficiaries; To analyze the capacities of main stakeholders to be involved in the process of the project implementation; To assess the sustainability of the project; To have an impact on ITU-D s programs, objectives, strategies and on ongoing Action Plan. In developing the project, it should identify the benefits that are designed to target a market or a population. The expected short-and long-term results must be clearly identified in order to assess the impact they will have on beneficiaries

13 This stage looks for a general assessment of problems that have been identified in an initial analysis. The key to identify the problems successfully is to understand the causes of the problems or constraints. Therefore is critical that project managers and project coordinators understand the root (cause) of the problem and its effect on the target population, before formulating any objective. In this stage it is essential to identify, consult and try to involve all stakeholders from the beginning of the process, so they will assume ownership over the project and its results. Roles and responsibilities should be clearly identified and assigned to the stakeholders. The stakeholders in a project are all parties who have an interest to be involved (partners) in or are affected by the project, or get benefits from the project (beneficiaries). They include among others governments, agencies, international organizations, non-governmental organizations, civil society associations and citizens. Key questions to be answered in the identification process include: What are the main challenges in a given political and socio-economic context? Who are the main domestic stakeholders and what have they experienced in this field? Describe the current strategies to deal with identifying problems and of beneficiary needs. In order to avoid duplication of efforts, specify which other international/regional organizations are working in this field. What are their lessons learned and strategies used to address similar issues? Which stakeholders would like to contribute to the project? Describe their roles and responsibilities during the implementation process. What is ITU s experience in addressing similar problems? How will the project be executed in comparison with existing or similar projects in ITU-D sector? To facilitate the analysis above, it is recommended to use the template of Logical Framework presented in Section 7.1 of Annex. 3.2 Fundraising This stage seeks to answer the question: who can invest in the implementation of the project and become a partner? The fundraising process is performed in line with the BDT s resource mobilization strategy. «Convincing donors is obviously a matter for resource mobilization (RM), but it is also a communication exercise. RM, communication and branding focal points must therefore work hand in hand to serve the capital formation needs of colleagues involved in programme and project management. The purpose of this Concept Note is to introduce a new communication and branding tool, as well as a vehicle to approach potential donors and expose our products, projects and initiatives in a harmonized way: the new concept note. A Concept Note (CN) is a synopsis or a (brief) preliminary description of a project or an initiative that aims at engaging potential donors deeper into a partnership/collaboration conversation. Concept Notes are used as an initial outline for potential partners consideration/review and as a guide for discussion, prior to the development and submission of a more complete proposal (as required according to donors mechanisms) or as part of formal submission processes. CN is therefore considered a critical tool during the initial stage of resource mobilization. The purpose of a concept note is to capture the interest of the donor and demonstrate that the project or initiative is worth further considering. It helps donors to determine if the described project or initiative fits with their goals and strategy and is likely to be accepted for fund allocation. 13

14 Partnership Building Division has introduced a new Concept Note Template to be used by all BDT colleagues looking for extra budgetary funding. This template has been designed to fit a variety of templates and guidelines published by donors. It is accompanied by Guidelines to facilitate the preparation of the notes. Objectives The specific objectives of a Concept Note are: To provide tools to support BDT Resource mobilization efforts by all BDT staff (as recommended in the resource mobilization strategy for BDT); To have harmonized documents to be presented to potential partners, clearly stating and justifying the need for support for the proposed project or initiative; To permit existing or potential partners to explore the possibility of partnering with ITU in a proposed project or initiative (including allocating resources); To use the concept notes to develop (more) effective and powerful external communication and promotion policy, as well as supporting material to approach partners.» Finalization As soon as sources of funds are secured and requirements and obligations of donors are clarified, a full-scale project proposal, in the form of a Project Document, can be prepared. A Project Document is a more comprehensive document that includes all available information gathered in the previous stages. The budget and work plan should be attached to the project document. The first activity is to determine the impact and objectives to be reached and to identify results. The second activity is to define indicators to measure performance of impact, objectives and results. In this stage, the logical framework tool should be developed to include data that would verify whether developed indicators are effective to measure the achievement of the best results. Finally, it is necessary to consolidate all information in the ProDoc, a template of which is presented under Section 7.2 of Annex. 1. Defining Project This analysis corresponds to the preparation of the content of the various sections of the ProDoc: a. Background and Context This section presents information and explanations related to the project, and in particular why it should be implemented. b. Objective Taking into account the context and needs of the beneficiaries, the objective must be clearly defined and shared with stakeholders. Expected results (as defined below) must contribute to the achievement of this objective. In order to develop well-constructed objective, the following issues should be taken into account: Objective should be articulated based on realistic and identified effects of problems; Objective is not equivalent to strategies, activities, processes and results; Provide a clear distinction between the formulation of objective and results

15 c. Expected Results In order to define results, a clear vision should be elaborated of what the project hopes to achieve. Results are desired outputs (short-term) and impacts (long-term), and include tangible or non-tangible benefits to beneficiaries or end-users. Results can be expressed in quantitative or qualitative standard, value or rate. In other words, results are the direct consequences or effects generated by activities and lead towards the achievement of stated objectives. The Impact measures the long-term and widespread development changes in the society, economy and environment to which the project contributes. d. Project Activities Activities should be selected on the basis of: A clear understanding of the problems; An analysis of the opportunities; An evaluation of the risks of the situation; An evaluation of available or expected resources and capabilities; The impact on ITU-D s mission and goals. e. Inputs The inputs include financial and in kind contributions (premises, equipment, staff...) of each partner. f. Roles and Responsibilities It is necessary to clearly define the roles and responsibilities of the parties in the main activities of the project, as this avoids misunderstandings and problems in implementation phase. g. Risk In this section, indicate relevant risks that can negatively affect the success of a project, and mention remedial actions foreseen in order to mitigate these risks. h. Project Management Provide an overall description of the project management. In this regard, elaborate management structure and terms of reference agreed upon by project partners. i. Monitoring and Evaluation Provide an overall description of the project monitoring and evaluation activities. In this regard, elaborate reporting requirements and frequency of such reports. j. Sustainability Elaborate on how the sustainability of the project is to be achieved. If applicable, describe sustainability measures, such as the self-financing mechanisms, that will be put in place upon the completion of the project. k. Budget and Work plan At the end of each Project Document, it will be accompanied by two annexes. The first one defines the budget with details about the contributions of each party, as well as lines for expenditure. The second annex provides the schedule of major activities and their sequence for the implementation of project. 15

16 2. Defining Key Performance indicators Once the Project impact, objective, results and activities have been clarified and agreed by stakeholders, the next stage is to develop indicators to measure the performance of each result, each objective and the overall impact. If results are clearly stated, it is easier to select indicators. An indicator is a mean of measuring actual results against planned or expected results in terms of quantity, quality and timeliness. The quantitative indicators are defined as numerical value, usually expressed in specific numbers or a percentage, but the qualitative indicators reflect perceptions, judgments, satisfaction or opinions. In order to select a good indicator, a participatory process of discussions amongst all stakeholders should apply. Most commonly used criteria to assess indicators are SMART. Table 2: SMART description Specific: Sufficiently precise to enable an objective assessment and measurement of the results achieved by project; Measurable: qualitatively or quantitatively, and formulated to facilitate comparisons between predictions and achievements; Attainable: defined at a level that can be achieved and that represents activities that allow to achieve a particular result; Realistic: measures should be determined in a realistic manner, given the respective project time and resources frame; Time-based: express benefits or changes that a project aims to achieve by the end of the implementation period. 3. Project Document The Project Document (ProDoc) is a key document that finalizes the formulation phase. It must contain all information about the context in which the project must be completed, its impact on the social and economic environment, the objectives and expected results, definition of roles and responsibilities of stakeholders, as well as the budget for its implementation. The details of the contents of this ProDoc are in set forth in Section 7.2 of Annex, which includes explanations and examples that will facilitate its drafting. 16

17 4 IMPLEMENTATION PHASE This phase seeks to answer the question of «what needs to be done in order for the project to be implemented successfully?». The first stage of implementation involves the planning of project activities to facilitate the allocation of resources to and the monitoring of such activities. Executing and Monitoring stages include a continuous process of controlling that planned activities take place and results are achieved as anticipated. The third stage is related to closure of the project activities The Project Coordinator could monitor the implementation of the project and ensure appropriate guidance where necessary, including providing feedback after reception of quarterly progress report. This also will facilitate the project evaluation phase, and prevent potential problems in implementation. Figure 4: Implementation Phase 3-Closing 2-Executing & Monitoring 1-Planning 4.1 Planning The key to a successful implementation depends on the quality of the planning that is done by the project manager before the start of activities. The main objective in the Planning phase involves the creation of a Project Management Plan, a mandatory document a template of which is proposed under Section 7.3 of Annex. The Project Management Plan provides a basis for evaluating the performance of the project during its implementation. It includes a complete list of the activities required to complete the project: Work Breakdown Structure, Schedule, and Procurement, Quality, Risks and Communication Plans. 17

18 4.1.1 Work Breakdown Structure The Work Breakdown Structure lists all the Outputs/Results and activities required to be achieved and undertaken, respectively, under the project Project Schedule As specified in Section 7.3 of Annex, the project manager should list necessary activities to achieve each deliverable defined as outputs/results. For each of the activities, the project manager will determine the workload and duration necessary to its realization, taking into account the interdependencies between them Procurement Plan The project manager should concentrate on definition of requirements (terms of reference for services, technical specifications for equipment). To estimate the delivery taking into account the needs, and to develop a preliminary schedule of purchases, the project manager must coordinate with the Procurement Division which is responsible for the relationship with suppliers. The project manager must plan the acquisition of products and services needed to achieve the project deliverables. Control and monitoring of the plan will allow the project manager to mitigate risks related to the non-mastery of suppliers and external service providers Quality Plan The quality control is to determine the quality criteria that must be achieved by the deliverables to be produced by the project in order to meet the needs of the beneficiary, and to plan the verification required. Quality assurance will check whether the processes to achieve the quality criteria of the deliverables are met. Thus, the project manager must plan the activities to be undertaken during the project to deal with the control quality and assurance quality Risk Plan This planning is to identify the key risks and lists the actions that will reduce their impact on the project. As indicated in Section 7.3 of Annex, the project manager must be: Identify the potential risks; Define for each risk a priority ratio (e.g. likelihood score * impact score); Select risks that will be followed; Identify activities that will mitigate the impact of risks on the project; Create an activities schedule related to risks mitigation Communication Plan Lack of communication is often at the root of the difficulties encountered during the project. This communication involves the steering committee, the project team, the beneficiary and other stakeholders. The project manager must plan specific activities to carry out effective communication, including the following: Define the list of stakeholders who should be informed about the progress of the project; 18

19 Identify the type of information to be transmitted to each group of stakeholders; Identify methods of communication to be used; Build a communication plan as proposed in Section 7.3 of Annex. 4.2 Executing and Monitoring Build Deliverables The first and more important activity in the Executing and Monitoring stage is the development of each of the project deliverables/results specified within the ProDoc. The deliverables are then reviewed to ensure that quality criteria are met. If all the quality criteria have been met, then the deliverables are signed off by the beneficiary. Title and ownership to the equipment and other assets which form part of the deliverables of a project should be effected in accordance with Section of these Guidelines and the Transfer of Title to Equipment template referred to in such section Executing The Executing process is to perform all the activities defined in the planning. The project manager must be able to assign activities to each person responsible for their implementation, while ensuring the monitoring and controlling of their execution. Executing the implementation of the project should be through regular meetings planned by the project manager. These meetings should be reported according to the meeting Minutes template provided in Section 7.5 of Annex. Monitoring is to make continuous assessments for judging the gaps between the schedule and the budget planned and the realizations. Monitoring aims to ensure effectively managed results through measurement and assessment of performance. The overall objective of monitoring is to enable project manager to take corrective actions. Executing is an internal management responsibility and can be performed by a project manager through these activities: a) Perform Time Management Time management will monitor the time spent for each activity, in order to prevent delays that will impact the delivery of the project. b) Perform Budget Monitoring The Project Manager must accede to the data or information regarding the project budget utilization (contributions received, commitments and expenditures). Budget monitoring is an essential aspect of good project management to ensure that expenditures and commitments remain within established allocations and the overall project budget. Close follow-up with appropriate financial services is the key to monitoring costs and forecasting any potential deviation from the original budget plans. Project partners and especially donors should be kept informed through financial reports, and substantive deviations from the original budget plans should be reflected in budgetary reviews each quarter. c) Perform Procurement Management After the definition of requirements for the project, the whole process of procurement of goods and services is the responsibility of the Procurement Division. 19

20 Therefore, the project must be monitored regularly with the Procurement Division assistance. d) Perform Quality Management Quality management is to ensure that the quality criteria previously agreed with the beneficiary are met. It will also achieve a quality control to ensure that the processes designed to achieve the desired quality criteria of the project deliverables are met. e) Perform Risk Management Risk management is to achieve the following activities: Take into account the risks identified during the assessment at the planning stage; Develop a detailed sheet for each of these risks using the Risk Monitoring Form template attached under Section 7.6 of Annex; Follow up activities related to the mitigation of each risk; Review the risk matrix for updates. f) Perform Communications Management This will entail the regular monitoring of communication activities, as they were defined at the Project Management Plan (Section 7.3 of Annex), and the carrying out of the following: Prepare the content of each communication action; Prepare the necessary materials; Communicate information to stakeholders according to the expected timing. g) Perform Change Control The process of change management is very important, as it can control the flow of change requests that usually come from the beneficiary. These requests may involve minor changes but they can also involve major changes that have a significant impact on time and on budget. Keeping a good track changes will prevent delays in project development and budget overruns. The process of change control is to perform the following actions: Receipt of request for change in the format provided in the Change Request Form set forth in Section 7.7 of Annex; Analysis of the opportunity to support the change request; Submit the request to the Steering Committee or Management Committee when the impact is significant (duration, budget); Modify activities and the action plan accordingly Reporting The Reporting is the formal presentation of controlling information using the Project Progress Report template that is proposed in Section 7.8 of Annex. The aims of reporting are: To inform stakeholders on the progress of the project, as well as of arrangements to ensure proper development of the project; 20

21 To provide a formal documentation of the activities of the period, in order to facilitate audits and evaluations; To serve as a reference to future projects in order to ensure that lessons learned (project successes, failures, best practices) through implementation are available for consideration when formulating and implementing similar projects; To promote transparency and accountability; To report to the donors on the project s progress. The Project Progress Report (a template of which is set forth in Section 7.8 of Annex) is required for all projects and is prepared by the Project Manager on a quarterly basis. The reports provide a framework for assessing status and a record of project implementation at any given time. Its primary aim is to ensure that supervisors of the project coordinator or project manager formally assess the status of project implementation. The progress reports should be prepared quarterly and be submitted in the second week of the following quarter. To facilitate the preparation of such report, it is recommended to use the Project Manager Log (a template of which is set forth in Section 7.4 of Annex), which integrates the main events and information related to the project, as well as specific columns to each report. The project manager can sort and retrieve information using the column for the report to be developed. Assessment/Feedback Report (a template of which is set forth in Section 7.9 of Annex) is prepared by the project coordinator based on the quarterly project progress reports produced by the project manager, and in some cases (e.g., difficulties encountered, timeout, insufficient budget ), decisions are proposed for approval by managers. In this assessment, the Project Coordinator must ensure that decisions have been implemented correctly and documented as useful. During the life of the project, it may be necessary to provide a mid-term evaluation to be conducted by the internal project team or by an external consultant depending on the importance of the project. 4.3 Closing The narrative and financial closures are determined by the dates indicated in the original agreement signed with the donor, or in subsequent approved project extensions. Narrative closure refers to completion of all operational activities and financial closure includes termination of all financial transactions. Closure should be performed immediately after completion of all activities of the project. It s important to distinguish between the two stages of project closure: the closure of activities (i.e., narrative closure) and financial closure Project operationally completed To ensure that the project was finished on the operational level, the Project Manager shall check the following activities: a) All activities listed in the project document are satisfactory completed; b) The envisaged results are produced; c) Closure Report and Post Implementation Review are prepared; 21

22 d) Contracts with personnel, suppliers and service providers are terminated; e) All rights, titles and ownership in and to all equipment and other assets that were acquired under the project for the beneficiary thereof are disposed and/or ownership have been transferred and assigned by ITU to, and have been accepted and assumed by, such beneficiary) (using the Transfer of Title to Equipment Template which is set forth in Section 7.11 of Annex). It is understood that the transfer will be made during the development of the project and the project manager will incorporate the evidence in the closing portfolio. The timing at which such transfer of title should occur will depend on the type of asset sought to be transferred (i.e., tangible assets vs. intangible assets). For more information concerning such timing, please refer to the General Instructions paragraph that is included in the Transfer of Title to Equipment Template; f) Products have been delivered to the satisfaction of the beneficiary; g) Decisions have been taken regarding the residual funds of the project, if any. The Closure Report (a template of which is set forth in Section 7.10 of Annex) is required upon completion of all project activities and contains information on the achievements of objectives, results and activities of the project. It should clearly explain any variances from the originally approved project in terms of duration, budget, results and delivered services. The closure report should be submitted within 30 to 60 days of the operational completion of the project, unless a different specific time frame is stipulated in the relevant agreements with project partners Project financially completed The financial closure is the responsibility of Finance. They must check all the points mentioned below before proceeding for the final closure. a) All requested funds have been received; b) Bills have been fully paid and liquidated; c) Expenditures have been recorded; d) The final budget revision has been carried out; e) The final financial report has been prepared and submitted to the donor; f) The final signed statement of account has been prepared and received by the donor; g) The final financial explanatory report has been prepared and submitted to the donor; h) The remaining funds, if any, have been returned to donors or re-allocated to other projects following donors approval. 22

23 5 EVALUATION PHASE Before the end of the project, the project coordinator should supervise two assessments: 1. Ex-ante evaluation of the project through the ProDoc, to be completed at the end of the formulation phase; 2. Mid-term evaluation to be carried out by the project coordinator or by an external consultant, depending on the size of the project. This evaluation is undertaken approximately half-way through project implementation. Mid-term evaluation analyses if the development of the project is carried out as planned, and identifies the difficulties to be overcome either in-depth or through desk evaluations. The evaluation phase is in addition to the ex-ante and a mid-term evaluation described above, and takes place after the completion of a project or main activities within the project. Several stages are involved in the preparation of the evaluation process, such as to plan evaluation with assistance of a Post Implementation Review Plan, to report on the outcomes of evaluation using the relevant report templates, and to describe lessons learned and best practices. Evaluation involves financial resources that should be incorporated in the budget planning before the project would commence. Figure 5: Evaluation Phase 3-Lessons Learned and Best Practices 2-Impact Analysis 1-Post Implementation Review 23

24 5.1 Post Implementation Review This stage will be performed after completion of the project according to a schedule previously established by management. Depending on the kind of project, the evaluation will be done on site or through desk review, and in some cases PRJ could make use of external consultants. Based on the key performance indicators defined on the ProDoc, this review assesses whether the objectives and results were achieved in an effective and efficient manner, and provides recommendations from project implementation. The following criteria are used in the process of designing the review plan: Relevance outlines the appropriateness of project objectives to the problems that it had planned to address. The review plan must integrate the quality assessment of the definition and formulation of the project. Efficiency provides the factual information that the project results have been achieved at a reasonable cost, i.e. how well the results have been achieved in terms of quality, quantity and time. To ensure that the most adequate process has been adopted, it would be useful to analyze if it was possible to implement alternative approaches. Effectiveness examines the contribution made to achieve the best results. This is designed to assess the benefits for the different categories of the population (women, children, elderly, disabled and vulnerable groups). A - Planning review The review plan outlines what is to be evaluated, by when, and with whom. It should also provide resources and budgets necessary to carry out these assessments. The assessor should pay particular attention to strategic and useful information for ITU and its partners in decision-making. In order to serve its purpose, the evaluation plan should take into account the following factors: 1. Choice of evaluation: management is responsible for the choice of results and projects to be evaluated. 2. Timing of evaluation: the plan should specify timing of each evaluation. 3. Stakeholders of Each Planned Evaluation: should identify the stakeholders who were involved in the project and who are affected by the results of the evaluation. 4. Financial Requirements: the evaluation plan should estimate the financial requirements for each evaluation. Dissemination of information should also be included in the budget plan. B - Reporting This review aims to assess the relevance, efficiency and effectiveness of the implementation of the projects and results achieved. As an outcome, the evaluation provides recommendations and a lesson learned for improving future policies and projects, and establishes a basis for accountability. The Post Implementation Review (a template of which is set forth in Section 7.12 of Annex) This review will allow an objective basis that reflects the results achieved and decisions taken, in order to draw lessons learned that may help to improve the implementation of projects. A good evaluation report communicates findings, lessons learned and recommendations clearly, accurately and appropriately, and while being an objective 24

25 presentation of the project, ensures that the concerns and comments of the involved parties are correctly reflected. Once the Post Implementation Review is prepared, it should be published and disseminated to all stakeholders. 5.2 Impact Analysis and Sustainability The contribution of a project on the development of a market or population must be measured several years after the end of it. Unlike the evaluation to be carried out a few months after the closure of the project to ensure that the objectives and expected results are achieved, the impact analysis is usually conducted one to three years after the project has been completed. The impact analysis assesses the level of success of the project and, in particular, its durability and the extent of the impact of such criteria which have been defined in the ProDoc. To perform this analysis, one must return to the indicators identified in the project document. The impact analysis allows evaluating the effects on the socio-economic environment, the general policy or sectorial objectives. A project can be said to be sustainable as long as it continues to deliver benefits to the project target groups. Therefore, for the monitoring of sustainability, quality factors (such as ownership by beneficiaries, appropriate technology, institutional and management capacity, socio-cultural issues, environmental factors and damages) should be considered. The Sustainability Analysis assesses the project's ability to continue its activities without external funding, as well as the beneficiaries' ability to manage their project in an independent manner. The impact evaluations focus on the entire range of a project s results, including unforeseen and longer-term impacts as well as sustainability and impacts on affected people outside the immediate target groups. The realization of this analysis should include the following considerations: 1 - From the previously established indicators, define the scope of the analysis to obtain useful information for the ITU and its partners; 2 - Determine the schedule of actions to be taken and who should carry them out; 3 - Estimate the budget needed to achieve the mission; 4 - Prepare media collection of information; 5 - Collect data with project stakeholders; 6 - Develop an Impact Analysis Report using the template that is presented under Section 7.13 of Annex. 25