Measuring Output from R&D Activities in Innovation Surveys

Transcription

1 Measuring Output from R&D Activities in Innovation Surveys Luuk Klomp Statistics Netherlands, Department of R&D and innovation statistics April 2001 Paper prepared for the ISI 53 Conference Seoul, Korea, 2001 Abstract This article starts with an overview of the development in R&D statistics in the last four decades. R&D expenditures and R&D personnel were the basis for measuring research activities in the 1950 s. The imperfections of R&D indicators became more and more clear as time progressed. A major deficiency of R&D is that it covers the input side of the innovation process, thereby neglecting the output side. Innovation surveys are supposed to close this gap in measurement. The second part of the paper discusses the contribution of innovation surveys, while the final part is about possible future developments to fulfil the data requirements in a progressing knowledge-based economy. Special attention will be paid to the recent initiative in the European Union to develop a list of indicators which should cover the concept of the knowledge-based economy. Keywords: R&D measurement, innovation output, knowledge-based economy. Résumé Cet article commence par donner une vue globale sur le développement des statistiques de la R&D durant les quatre dernières décennies. A la fin des années cinquante du dernier siècle, les dépenses de R&D et le personnel de R&D étaient pris comme base pour la mesure des activités de recherche. Or, avec le temps les imperfections des indicateurs de la R&D devenaient de plus en plus criantes. L une des déficiences majeures de la R&D est sa couverture du côté input du processus de l innovation en négligeant le côté résultats «outputs». Les enquêtes de l innovation sont supposées réduire cet écart de mesue. La deuxième partie de l article présente et discute la contribution des enquêtes d innovation, alors que la dernière partie concerne les possibles futurs développements à même de répondre aux besoins de données dans une économie fondée sur la connaissance et qui est en constante évolution. Une attention particulière est donnée à la récente initiative prise par l Union Européenne pour le développement d une liste d indicateurs devant couvrir le concept d économie fondée sur la connaissance. Mots clés: Mesure de la R&D, output de l innovation, économie fondée sur la connaissance.

2 Measuring Output from R&D Activities in Innovation Surveys 1 The perception of knowledge as an important production factor and a prerequisite for technological innovation is generally accepted among scientists and policy-makers alike. The importance of innovating firms for micro and macro economic growth was the main subject of a conference on innovation and enterprise creation held in France in November That conference, based on the results of the second wave of innovation surveys in the countries of the European Community, was well timed with the growing interest from politicians for the knowledge-based economy. In the next few years, high quality indicators on the knowledge-based economy will be required to enable European political leaders to judge whether the EU region is developing into the most competitive and dynamical knowledge-based economy around the world within a decade. One might expect the huge interest within the European Union in the knowledge-based economy, and thus in the innovation process, to lead to the development of new indicators, better harmonisation of indicators and an improved insight into the effect innovation on economic performance. This article starts with a concise historical overview of R&D measurement, goes on to discuss the contribution of innovation surveys alongside R&D questionnaires, and concludes with the measurement of the knowledge-based economy. Historical review: from R&D questionnaires to innovation surveys The Dutch Nobel prize laureate Jan Tinbergen made a plea for measuring investment figures just after the second World War. Among other things investment statistics are supposed to serve as a forecast of growth in the business cycle. In addition to what we nowadays call tangible investments, investment in research and development (R&D) was the first intangible asset to be measured by statistical offices. R&D surveys were conducted as early as the 1950 s and Statistics Netherlands, for instance, has been publishing R&D figures since the late 1950 s. The methodology for the measurement of R&D expenditures and R&D personnel has been elaborated in the so-called Frascati Manual (OECD, 1993). This manual, developed by the OECD in cooperation with Eurostat and Unesco, gives elaborate definitions of all aspects of R&D statistics. Although measurement of R&D statistics is still being adapted proposals for revisions in the Frascati Manual will be discussed in Rome in May 2001 figures on R&D are relatively well harmonised within the OECD and for several non-oecd countries. Differences in data collection and statistical methodologies between countries are well known by the OECD secretariat enabling them to publish figures that are comparable across countries. 2 Despite the valuable information R&D figures provide on the innovation process, these figures do not paint the whole picture. R&D indicators have several deficiencies (see for instance Kleinknecht (1996) and Kleinknecht and Bain (1993)). First, the so-called linear model of innovation processes was the leading paradigm in the 1960 s and 1970 s. It assumes logical and one-way steps from the input (e.g. R&D), via the throughput (e.g. co-operation), to the output stage (e.g. innovations) of the innovation process. The linear model became more and more discredited over time. One issue the model overlooked is that firms may operate at different levels of efficiency when introducing new products and processes, i.e., higher levels of R&D input do not imply automatically higher levels of innovative output. Moreover, the relevance of feedback loops between the three stages of the innovation process, which are omitted in the linear model, became more and more apparent. Kline and Rosenberg (1986) introduced a theoretical model which incorporated feedback loops. Klomp and Van Leeuwen (2001) adapted this model and reported important feedback loops between the stages of the innovation process in their empirical application. 1 The author wishes to thank Jos van Deventer, Frank Foyn, Ibrahim Laafia, George van Leeuwen, Gerhard Meinen, and Sjaak Pronk for helpful comments, Lieneke Hoeksma for correcting my English, and Ibrahim Laafia for translating the abstract into the French language. The views expressed in this article are those of the author and do not necessarily reflect the policies of Statistics Netherlands. 2 The OECD publications about R&D, for instance the Main Science and Technology Indicators, contain a lot of footnotes explaining country specific data characteristics. Notice of this information is necessary to be able to compare R&D figures between countries appropriately.

3 Secondly, intramural R&D is confined to the input stage of the innovation process, but it does not cover all inputs. R&D is just one of seven categories of innovation expenditures applied in the Community Innovation Survey (CIS) co-ordinated by Eurostat, the statistical office of the European Union. Although intramural R&D is one of the largest categories, it accounts for less than 50 percent of the total amount of innovation expenditures. Third, traditional R&D questionnaires tend to capture formal R&D. Therefore, R&D in small firms seems to be undercounted (see, for instance, Kleinknecht (1996)), an assumption that has been confirmed by the integration of the R&D and innovation surveys at Statistics Netherlands. Small firms answer questions on R&D involvement affirmatively in innovation surveys, while some of them do report no R&D activities in traditional R&D questionnaires. Undercounting of R&D typically has only a limited impact on total R&D expenditures in the business enterprise sector, as R&D figures are largely determined by the big multinational firms. Indeed the huge influence of a very limited number of firms on R&D figures is the fourth and final deficiency of R&D indicators we shall discuss here. Figures for R&D expenditure do not reveal whether the innovation process receives attention across firms and industries. The importance of the comparison of R&D intensities between countries to assess the state of the knowledge-based economy is therefore rather limited. 3 Moreover, the R&D budget for the defence system distorts the comparison between countries. Innovation surveys: rich data bases for the analysis of all stages of the innovation process The innovation survey in the EU (CIS) contains information on several aspects of the knowledge-based economy, e.g. R&D expenditures, the co-operation with partners seeking innovation, the realisation of innovative products and processes, and factors hampering the innovation process. One of its strengths is its coverage of all three stages of the innovation process: input, throughput and output. The most striking output indicators in the innovation survey are the realisation of new products and processes and the contribution by new products to turnover. These products are distinguished by whether they are just new to the firm or whether they are also new to the market. Eurostat has published figures from the CIS2 data base, covering the period , which may serve as a basis for comparisons between EU member countries (Eurostat, 1999 and 2000). When the ranking of the R&D intensities for EU countries are compared with the innovation intensities innovation expenditure as a percentage of turnover a strong positive relation is observed (see Klomp and Van Leeuwen (1999)). Hence, countries with a high innovation intensity also have a high R&D intensity, and vice versa. Comparison of innovation output, measured as the percentage of innovating firms, with innovation input (R&D or innovation intensity) presents a different picture. The relatively weak position of the output indicator of two Scandinavian countries Sweden and Finland is remarkable in relation to their high R&D intensity. A possible explanation for the differences between the rankings for the input and output indicator is that a high R&D or innovation intensity is determined by a few large multinational firms, while such firms hardly influence the proportion of innovative firms in a country. The output indicator sheds light on the knowledge diffusion in industries. A general conclusion is that assessing the state of the innovation process in a country is better balanced if a collection of indicators reflecting the different stages of the innovating process is used. International comparisons of innovation surveys benefit from the Oslo manual, which deals with the methodology for (technological) innovation (OECD and Eurostat, 1996). This manual serves as a basis for the CIS in the European Union, and more importantly the methodology applied in the Oslo manual for innovation is linked to that in the Frascati manual for R&D, enabling integration of R&D and innovation surveys. 4 Although harmonised figures may be drawn from the CIS, the questionnaire requires further improvement. Eurostat has set up two task forces in preparation of CIS3 to be launched in Among other topics, two issues for improvement which have been amply discussed are: The need to broaden the concept of innovation, i.e., should organisational change be included? It has been decided to exclude pure organisational change from the concept of innovation, but in line with the Oslo manual, one question will be included on non-technological innovation. At Statistics Netherlands 3 R&D intensity for a country is defined as R&D expenditures as a percentage of gross domestic product (GDP). 4 Statistics Netherlands integrated its R&D and innovation surveys in The connection between the Frascati and Oslo manual will be discussed at the meeting on the revision of the Frascati manual in Rome in May 2001.

4 we have already started to ask whether technological or non-technological innovations are most important for improving the competitiveness of the firm. Although about 50 percent of respondents report that they are unable to split the impact of the two types of innovation, the majority of the other 50 percent of innovating firms judge non-technological innovations as most important. Examples of nontechnological innovations are organisational innovations, managerial innovations and changed corporate strategic orientations. The need for more output indicators. Policy-makers in the EU underline the importance of output indicators enabling them to assess the effects of (European) innovation policies. The impact for product innovation is measured by the share from sales of new products in total turnover. Such an indicator cannot be established for process innovation. Two supplementary alternative measures for output indicators are: patents and literature-based counting of innovations. As information on patents is collected outside national statistical institutes (NSI s), for instance by the European Patent Office (EPO) in Munich, most NSI s collect no or only limited information on patents to reduce the response burden. However, the confrontation of the two data sources containing patenting indicators might offer new insights into the importance of patents in the innovation process. Hence, CIS3 contains a limited number of questions on patents and other protection methods. Guellec (2001) explores this topic in more detail. Literature-based counting of innovations is a way to collect information on innovative output outside the CIS framework. This approach is based on new innovations reported in technical and trade journals, a socalled object approach. The CIS adopts the subject approach, i.e. the firm is the unit of observation. Although it would be possible to apply the object approach within the CIS, asking for all innovations in a firm, this approach is not recommended in the Oslo manual. Application would result in excessive reporting burden for firms (Oslo manual, par. 379). 5 The final CIS3 questionnaire has been determined and will be launched in the EU in OECD and Eurostat encourage other countries to adopt the CIS concept. Recently, a meeting in South Africa was organised to exchange ideas between countries thinking about launching a CIS-like survey and countries being involved in this kind of surveys already. Outside the EU, Australia and Canada acquired experience conducting innovation surveys. Measuring the Knowledge-based economy The Knowledge-based economy (KBE) is defined by the OECD as an economy directly based on the production, distribution and use of knowledge and information. (OECD, 1996). The interest in the knowledge-based economy and in accompanying statistics received a strong stimulus in Europe in March 2000, when the political leaders of the EU countries formulated the new strategic goal for the EU for the next decade to become the most competitive and dynamic knowledge-based economy in the world.. Obviously, Europe is not the only continent eager to measure the KBE more accurately. The Australian Bureau of Statistics (ABS, 2001) reports that the APEC Economic Committee extended the OECD definition of the KBE to state that in a KBE the production, use and dissemination of knowledge is the main driver of growth, wealth creation and employment across all industries.. This implies that a KBE is not restricted to a few high-tech industries; old economy industries are knowledge intensive too. The European Commission has introduced an Innovation Scoreboard to monitor the development of the KBE statistically. The scoreboard contains 16 indicators divided into four main categories: human resources; knowledge creation; transmission and affiliation of knowledge; and innovation, finance, output and markets. Another recent European statistical initiative is the benchmarking of national research policies. Four interrelated themes in this exercise are: human resources in R&D (including attractiveness of science and technology professions); public and private investment in R&D; scientific and technological productivity; and impact of R&D on economic competitiveness and employment. The dimensions in the two European initiatives coincide to a large extent with those nominated at ABS (2001). New in the ABS approach is the inclusion of the concept of social capital defined as networks together with shared norms, values and understandings that facilitate co-operation within or among groups.. 5 The pros and cons of patents indicators and from data collected by the object approach are well known in the literature. Drawbacks of patents are that many innovations are not protected by patents, the propensity differs between industries and size classes, and many patents never make it to commercial products (see Griliches, 1990, for an overview). I refer to annex I of the Oslo manual, and to Kleinknecht and Bain (1993) for more details on the object approach.

5 In recent discussions on the KBE more attention has been paid to the transmission and use of knowledge (Gault, 2001). Innovation surveys deal with this matter through indicators on co-operation in innovative projects and in the use of (external) information sources. Moreover, at Statistics Netherlands more information on partnerships, in particular public-private relationships, is collected for co-operating firms. Linking the CIS database to other sources provides the opportunity to analyse the relationship between innovation and economic performance. It seems that linking databases is the empirical way to test our theoretical concepts of the processes at work in the KBE. As the CIS is already large in its present form, there is little room within that framework to extend the questionnaire with additional topics. I conclude with a topic for future research. As far as I know, there is no general conceptual framework for the KBE/new economy at present. There is a widely felt and still growing need for such a conceptual framework, as this would enable the data collection to be tuned between different surveys. I refer to the R&D and innovation surveys and the ICT use surveys as an example. Recently, the OECD started an initiative to try to collect data on knowledge management in the private sector. This is a promising effort as it may serve as a guide to fill the conceptual framework of the KBE statistically. At the EU level resources are devoted to refining the measurement of the KBE. Efforts are being undertaken in the European Commission, and support is also manifest in the 5 th and apparently in the 6 th Research and Technological Development (RTD) framework programme. Up to now, progress is hindered by confidentiality issues resulting in no or limited access to micro data by (academic) researchers. Eurostat does its utmost to solve the problem of restricted access to the data by constructing a framework that meets all legal conditions of EU member countries. Let s hope that the joint and intensive efforts will help to expand our knowledge of the processes playing a part in the KBE. References ABS (Australian Bureau of Statistics), 2001, Measuring the Knowledge-based Economy and Society, draft paper, for information: sheridan.roberts@abs.gov.au. Eurostat, 1999, Community Innovation Survey 1997/1998, Statistics in Focus, Theme 9-2 / 1999, Luxembourg. Eurostat, 2000, Community Innovation Survey 1994/1996, CD-ROM with final results of the survey on innovation, Luxembourg. Gault, Fred, 2001, Statistical Indicators of Technologies and Practices: Definitions and Issues of Measurement and Aggregation, Proceedings of the 53 rd Session of the International Statistical Institute, August 22-29, Seoul, Korea. Guellec, Dominique, 2001, Patents as Indicators of Technology Output: A Review, Proceedings of the 53 rd Session of the International Statistical Institute, August 22-29, Seoul, Korea. Griliches, Zvi, 1990, Patent Statistics as Economic Indicators, Journal of Economic Literature, vol. 28, pp Kleinknecht, Alfred (ed.), 1996, Determinants of Innovation: The Message from New Indicators, MacMillan Press Ltd., London. Kleinknecht, Alfred and Donald Bain (eds.), 1993, New Concepts in Innovation Output Measurement, St. Martin s Press, Ipswich. Kline, S.J. and N. Rosenberg, 1986, An overview of innovation, in Landau, R., and N. Rosenberg, (eds), The Positive Sum Strategy. Harnessing Technology for Economic Growth, National Academy Press, Washington DC. Klomp, L and G. van Leeuwen, 1999, The importance of innovation for firm performance, Discussion paper LNM-series, 9902, CBS, Voorburg, The Netherlands. Klomp, L. and G. van Leeuwen, 2001, Linking innovation and firm performance: a new approach, proceedings of the conference Innovation and Enterprise Creation: Statistics and Indicators, Session C: Innovation and Firm Performance, OECD and Eurostat, 1996, Oslo Manual, second edition, Paris. OECD, 1993, Frascati Manual, third edition, Paris. OECD, 1996, The Knowledge-based Economy, Paris. OECD, 2000, Main Science and Technology Indicators 2000/2, Paris.