Measuring the impact of ICT for development

Transcription

1 Measuring the impact of ICT for development CSTD Inter-sessional Panel, Theme 1 16 December 2010, Geneva Sheridan Roberts, consultant to UNCTAD sheridanroberts@infosocietystats.com

2 This presentation will cover Why measuring the impact of ICT for development is important and why it is difficult The complexity of ICT impacts The role of the Partnership on Measuring ICT for Development Measurement approaches Empirical evidence for the impacts of ICT Issues for consideration

3 Why measuring the impact of ICT for development is important Arguably, ICT has the potential to directly and indirectly change the lives of much of the world s population how individuals live, learn, work and interact processes of business and government the quality of the natural and built environment poverty alleviation via direct and indirect effects. The Tunis Commitment from the second phase of WSIS expressed a strong belief in the benefits that ICT can bring to humanity and linked them to fulfilment of the MDGs.

4 Why measuring the impact of ICT for development is important From the 10 WSIS targets, important impact areas can be inferred ICT access for poor and rural communities ICT in education ICT networks in health availability of e-government services electronic access to information and knowledge. Comparable and reliable ICT statistics are required for designing, implementing, monitoring and evaluating ICT policies. Measures of ICT impact are possibly the most important but also the most challenging.

5 Why measuring the impact of ICT is difficult Any impact is hard to measure even with an obvious relationship between variables, causality can rarely be proven. The impact of ICT is even more difficult to measure because there is a variety of ICTs and they have different impacts in a variety of contexts indirect impacts of ICT as a general purpose technology. What do we mean by impact? There are qualitative and quantitative differences between macro-level and micro-level impacts, direct and indirect impacts, short and long term impacts etc.

6 Why measuring the impact of ICT is difficult It is not possible to categorise most impacts as simply social, economic or environmental there is a complex web of relationships. There are important but varying roles of context factors such as level of ICT infrastructure, human capital, government regulation, e-government. Lack of data and data collection infrastructure. In an international context, there are extra challenges in achieving comparable measures across countries.

7 Existing ICT infrastructure Government ICT policy & regulation, e-government Level of education, skills, income Economy economic performance poverty alleviation Society employment privacy & security health individuals & communities innovation & research education citizen participation Environment

8 The Partnership on Measuring ICT for Development The Partnership was formed after the Geneva phase of WSIS to further international evaluation and benchmarking. Its members are 11 international organisations (8 UN including UNCTAD; plus Eurostat, OECD and the World Bank). Partnership members are involved in various activities directed towards achieving internationally comparable and reliable ICT statistics, including developing and maintaining a core list of ICT indicators compiling and disseminating ICT data providing technical assistance to developing countries through manuals and workshops.

9 Role of the Partnership in measuring the impact of ICT The Tunis Agenda specifically noted the Partnership and its role in the measurement of ICT impact. To date, the Partnership has not developed either ICT impact indicators nor measurement methodologies that reflects the difficulties involved in such tasks. However, the Partnership has established a Task Group on Impacts, led by the OECD. Its objective is to give an overview of the economic and social impacts of ICTs, how these impacts can be measured and what the data requirements are and to provide countries with a set of statistical and analytical tools to carry out their own assessment of ICT impact. Its terms of reference recognise both economic and noneconomic impacts and a variety of methodologies and data sources.

10 Measurement Analytical techniques Can be performed at the macro-economic, sectoral and micro-economic (firm) level econometric modelling using regression, growth accounting and input-output analysis. The usual objective of an ICT impact analysis is to examine the relationship between ICT and productivity, economic growth or employment ICT includes the ICT sector and ICT demand. Strengths include objectivity, use of existing data. Main weaknesses are model assumptions and dependence on the reliability and availability of input data.

11 Measurement Statistical surveys Can be a direct source of data on ICT impacts; inputs to other studies e.g. analytical. Surveys cover households, individuals, businesses, other entities e.g. government organisations. Strengths strong international statistical infrastructure reliability of output is reasonably subject to control data can be cross-classified by various characteristics perceptions questions provide data on causal links. Weaknesses Expensive to do properly perceptions questions lack objectivity.

12 Measurement Other main approaches Case studies source of much of the data on ICT impacts a wide range of approaches and data sources flexibility of scope but findings not generalisable. Panel studies longitudinal data, may be survey-based can provide good baseline data and account for time lags expensive and may suffer from unit attrition over time. Controlled experiments can show causality but difficult for complex topics. Use of administrative data some data e.g. trade readily available for many countries not usually designed for statistical purposes.

13 Empirical evidence for the impacts of ICT economic performance Increased economic performance provides potential for poverty alleviation. Macro-economic and sectoral impacts increase in size and productivity of ICT sector and related industries; appears to be the main source of benefit for developing countries ICT investment across the economy contributes to capital deepening and leads to increase in economic growth and labour productivity; there may be a critical mass limitation for many developing countries multi-factor productivity growth across the economy, via role of ICT in helping firms innovate and increase overall efficiency, there is also likely to be a critical mass limitation for many developing countries.

14 Austria France Contribution of ICT investment to GDP growth, percentage points % (1) Germany Italy Greece Ireland Finland Portugal Netherlands Spain New Zealand Japan Canada Belgium United Kingdom Denmark Sweden United States Australia Source: OECD

15 Empirical evidence economic performance Firm level studies most studies are for developed countries; they have generally found a positive relationship between use of ICT and labour productivity importance of complementary factors such as skills and organisational change impacts may be different for developing countries, with greater impact possible from less sophisticated ICTs, such as computers. Case study evidence especially on the benefits of mobile phones to small and micro businesses in developing countries.

16 Firm level study in Thailand A 2007 study by UNCTAD and the Thai National Statistical Office analysed the impact of ICT on labour productivity in urban manufacturing firms with 10 or more employees in Thailand. An econometric analysis, controlling for non-ict factors, showed that firms with a combined use of computers, the Internet and the Web had on average 21% higher sales per employee than firms without any of these ICTs. The ICT with the greatest single association with increased sales per employee was computers (14%).

17 Empirical evidence economic performance Various studies on the impact of broadband have found a relationship between broadband penetration and economic growth but broadband is not as available in developing countries therefore critical mass may be insufficient. There has been relatively little focus on negative economic impacts privacy and security impacts systems failures, data loss, disclosure of data loss of productivity through use of ICT in work time.

18 Empirical evidence employment ICT s role in the creation of employment and selfemployment opportunities for poverty alleviation direct impacts through growth of the ICT sector and ICTusing industries, especially telecommunications services for developing countries, ICT manufacturing and IT-enabled services for a small number of developing countries indirect impacts through multiplier effects. Case study evidence in developing countries indicates positive impact of ICT skills on employment prospects. Possible negative impacts through loss of employment as tasks are automated.

19 Empirical evidence education There is considerable policy interest in the benefits that ICT can bring to education however, impact studies show mixed results measurement is complicated. Potential benefits from ICT include tools for teaching and learning processes provision of skills needed in an information society improved attitudes/motivation to learning development of teachers technology skills increased access of the community to adult education and literacy.

20 Empirical evidence education Highly controlled and specific studies of the impact between ICT use and educational outcomes show positive (causal) impacts in certain conditions. PISA studies of performance of 15-year olds, coordinated by OECD and conducted triennially show positive impacts of ICT use in some conditions home use of computers but not necessarily use at school quality of ICT use (confidence, length of experience) more important than quantity other factors important capital (social, economic, cultural and technological).

21 Students use of ICT and average science scores, PISA Index of ICT Internet/entertainment use Index of ICT program/software use Bottom quarter Second quarter Third quarter Top quarter Source: OECD

22 Empirical evidence health Could expect positive impacts from use of electronic health records (EHR) telemedicine m-health (mobile health) ICT as enabling complex and networked medical equipment Internet as a source of information. Evidence is generally from case studies and indicates positive impacts benefits to individuals in efficacy of health care, convenience cost-saving benefits to health systems interoperability (between EHR and other clinical and nonclinical systems) appears to be a prime driver of benefits WHO has plans to establish indicators for monitoring e-health and assessing the impact on health systems.

23 Empirical evidence citizen participation, individuals, communities ICT can facilitate democratic processes and increase participation by citizens by various means including facilitation by e-government. It is clear that ICT can change many aspects of individuals lives there are measurable intermediate impacts (e.g. Internet commerce, time spent using ICT) benefits may be greater for the poor and other disadvantaged. There are obvious negative as well as positive impacts e.g. on children, Internet fraud. Perceived impacts (positive and negative) on individuals as measured by household surveys. Case study evidence of impacts on communities e.g. information and communication, social capital.

24 Empirical evidence environment There are positive and negative links between ICT and the environment potential to improve efficiency of energy-using processes and equipment facilitation of dematerialization (e.g. teleworking) essential role in climate change monitoring and modelling, dissemination of information, and administration of carbon pollution reduction schemes BUT negative impacts of ICT from energy usage and greenhouse gas (GHG) emissions from ICT use, manufacturing and transport of ICT products pollution from disposal of e-waste. Some impacts from scientific measurement and modelling with inputs from a range of sources.

25 Contribution of ICT to GHGs Estimates from 2007 and 2008 indicate that the ICT sector and ICT products are currently responsible for about 2% of global GHG emissions. High growth in ICT penetration and increases in processing power indicate that this contribution will grow further.

26 Issues for consideration prioritisation Given the range of ICT impacts and the fairly low availability of evidence on impacts, should the measurement of impact in certain areas be given higher priority than others in the years leading up to 2015? This question may be considered in conjunction with considerations of feasible and affordable data collection work.

27 Issues for consideration what can be done to extend indicators of ICT impact? Given the importance of relevant and internationally comparable data for impact studies, what can governments, development partners and international organisations especially those that are members of the Partnership do to extend ICT impact indicators? Examples may include setting statistical standards, accelerating the building of capacity for the production of relevant statistics and allocating sufficient funds to undertake surveys.

28 Issues for consideration policy imperatives What types of impact studies are the most useful, for example: extend macro-economic analysis to developing countries using methodologies applied by the OECD extend the measurement of firm level impacts to more developing countries consider the use of perceptions questions on surveys of business and household use of ICT extend the PISA programme to more developing countries to shed light on the impact of ICT on learning outcomes of 15-year olds attempt to standardise and extend case studies and controlled experiments.

29 Issues for consideration policy awareness raising What can be done to raise awareness among different stakeholders about the need for the measurement of impact of ICT? How can the CSTD contribute?