Monitoring and Evaluation of ICT Sector Reforms in a Pacific Island Country: A case study based on recent/actual work done

Transcription

1 Monitoring and Evaluation of ICT Sector Reforms in a Pacific Island Country: A case study based on recent/actual work done

2 The Assignment was funded by The World Bank through a grant to a country referred in this presentation as a Pacific Island Country (PIC) The author worked as a Consultant for Design as well as Implementation of the Monitoring & Evaluation (M&E) system in two separate assignments respectively, each lasting for 45 working days each

3 An effective ICT sector in this case is an important component for national development as among others, it greatly assists in sectors such as: Tourism; Import/Export of goods and services; Provision of Agricultural extension services, such as information on planting seasons, weather information, availability of improved variety of seeds and other planting materials etc.; Timely health care information; and Improving access as well as the quality of education, particularly in remote areas.

4 However, the effectiveness can only be measured by an Monitoring and Evaluation System that works, thereby ensuring ICT Sector is performing as desired. Clearly, M & E indirectly assists in national development.

5 Telecommunication Authority (Regulator) was established for carrying out major reforms in the ICT sector. The objectives were to: Manage the transition from a monopolistic to a competitive and multi operator market; Acquire and develop skills, tools and resources for the telecom industry; Facilitate and manage the growth of this industry; and Measure and optimise the impact of telecommunication reform on the broader community.

6 Regulator considers ICT as an important source of growth, skills development and employment generation in the country: Directly through new investors; and Indirectly through promoting other sectors of the Economy

7 Regulator aims to: Improve access to telecom infrastructure particularly in rural and other island areas for improved economic development; Establish the necessary environment for separate and diverse private sector development; and Among other initiatives, set up a state of art ICT sector Monitoring and Evaluation System. The goals of the M&E system, based on the above, are in the following slides.

8 In broad terms Goal of the M&E System was to provide critical information to Ministry of Communications (MoC) and the Government on effective deployment of ICT for improved economic growth of the nation

9 In particular, goals included: Monitor and assess the performance of the telecommunication industry operators in their delivery of services to households and business; Through monitoring and evaluating effectiveness of schemes such as Universal Services, ensure effective deployment of ICT in rural and remote areas;

10 Inform whether the objectives of the Regulator are being met; Enable examination of impact; Enable redesign of the activities, if required; Develop the capacity of the Regulator to sustain progress; and Enable drawing lessons learnt for future direction.

11 Key Performance Indicators suggested under World Bank TOR for the design of the M&E system included: Access: Service density by Market Segment, Urban vs. Rural, Network Coverage, Service quality (speed); Usage of scarce resources: spectrum, numbering;

12 Bottleneck infrastructure; Access to and usage of internet; and Broad Data on Industry Revenues.

13 However, on further research the World Bank indicators were considered too generic and broad and thought not to fully meet the requirements of this country. There is also a requirement for the Regulator to report on KPIs prescribed by the International Telecom Union (ITU), basically to bench mark all countries. The M&E System had to be therefore tailor made to meet the information requirements of ITU, meet World Bank s expectations and fulfil the M&E requirements of the country.

14 A1: Fixed Telephone lines per 100 inhabitants A2: Mobile cellular telephone subscription per 100 inhabitants A3: Fixed Internet subscribers per 100 inhabitants A4: Fixed broadband Internet subscribers per 100 inhabitants A5: Mobile broadband subscription per 100 inhabitants A6: International Internet bandwidth per inhabitant (bits/second/inhabitant) A7: Percentage of the population covered by a mobile cellular telephone network

15 A8: Fixed broadband Internet access tariffs per month in US$, and as % of per capita income A9: Mobile cellular telephone prepaid tariffs per month in US$ and as a % of monthly per capita income A10: Percentage of localities with public internet access (PIACs) ICT1: Proportion of total business sector workforce involved in the ICT sector (Operators only excludes manufacturing, export/import houses etc.) ICT2: ICT Sector share of gross value added (Operators only and excludes manufacturing)

16 Examples of KPIs added by the Regulator included to be able to fully meet its objectives, were: Quality of Service; Technology deployed; Investments made by foreign owned entities in ICT sector; Investment made by domestically owned entities in ICT sector; Consumer complaints; Etc.

17 A high level framework for M&E System is shown in the slide that follows

18 DATA FROM OPERATORS DATA FROM MOC OTHERS INPUT REGULATOR S WEBSITE OUTPUT Info for PUBLIC VIEWING DATA FOR the ) REGULATOR DATA for MoC

19 M&E system was designed to be built using state of art techniques. A part of it was under a Sector Monitoring Tab on the Regulator s website. Sector Monitoring Tab has drop down windows for: Input proformas Country wide report

20 It also has links to websites of other Government bodies such as Bureau of Statistics, ITU etc. The designated entities are able to download input proformas, fill it up and upload using the Submit button. This is done every quarter

21 The Regulator maintains an M&E database on its ITC System. This contains key monitoring information and quarterly updated automatically. This Database is kept highly confidential, both within and outside of the Regulator.

22 Quarterly and Annual reports for the Regulator s Board and MoC are produced automatically from the Database on the website using pre-designed/ prescribed templates Previous quarter s progress is evaluated together with the current quarter against an agreed baseline Any adhoc information requirement is met from information stored on the website using SQL Data collection and reporting framework is on the next slide

23 ITU & OTHER BODIES AGGREGATED DATA MINISTRY OF COMMUNICATION ICT CONSOLIDATED DATA FOR FPIC PROVIDES DATA FOR INDICATORS HH1 HH12, HHR1, B1 B12, ICT1 & ICT2, ED1 ED8, EDR1 BUREAU OF STATISTICS FCC PROVIDES DATA ON ICT3 & ICT4 AND DOMESTIC INVESTMENT MINISTRY OF INDUSTRY & TRADE PROVIDES DATA ON CONSUMER ISSUES AND TARIFF FOR SECTOR PROVIDES DATA FOR INDICATORS A1 A10, ICT1, ICT2 (FROM OPERATORS ) REGULATOR DATA TO MEET TAoPIC BOARD REQUIREMENTS FOR POLICY & DECISION AGGREGATED DATA GOES ON TAoPIC WEBSITE INVESTMENT FIJI CONSUMER COUNCIL CONSUMER ISSUES PROVIDES DATA ON NEW FOREIGN INVESTMENT GATHERS DATA FROM LICENSED OPERATORS BASED ON TAF INDICATORS INCLUDING CONSUMER ISSUES DATA FROM USS PROJECT SURVEY OTHER LICENSED OPERATORS FIXED LINE OPERATORS MOBILE SERVICES OPERATORS ISPs INTERNATIONAL CARRIERS

24 ITU s handbook for Collection of Administrative Data on ICT recommends annual data collection cycle. However, public listed operators generally report to their shareholders on quarterly as well as annual basis. In this case the M&E system was proposed to collect the data quarterly and annually.

25 At the Heart of the M&E System is the Database of information. The database has 7 sections as follows: Section A Aggregated Information from Operators Section B Information from Ministry of Communications Section C Foreign Investment in ICT Sector

26 Section D Consumer Complaints (from Consumer Council and Operators) Section E Information on Universal Service Scheme project of TAoPIC Section F Information from Annual Spectrum Survey Section G Country wide information based on information in Sections A F above. This Section will be available for viewing to the public on TAF s website.

27 This in summary is a snapshot of the M&E System In conclusion I would like to present the Lessons Learnt during the course of this assignment.

28 Programs that reach out to rural areas might contribute more than programs based just in urban areas; Financial sustainability is a challenge in island countries and over regulations may not help; and Involve stakeholders in programme design and monitoring from the very start;

29 Pay particular attention to: Sustainability; Infrastructure requirement; local availability or willingness to acquire new skills; Training facilities; and Technical challenge;

30 I am told by my friends and colleagues worldwide that not only Pacific Island Countries, but many other countries could benefit from a similar system In conclusion I would like to thank PTC for giving me an opportunity to make this presentation today and the audience for showing an interest in my presentation I hope you all have found it interesting and useful

31 Mr Davender Jain, Executive Director, Eze Solutions Pty Ltd, Sydney, Australia Phone: , Mob: