Series on Monitoring Research Networks No. 01

Transcription

1 Series on Monitoring Research Networks No. 01

2

3

4 Acknowledgements The authors wish to thank Sophie Alvarez, Claudia Oriolo and Joslin Isaacson for invaluable research support, and also Graham Thiele, Gordon Prain, Graham Durant-Law, Douglas Horton and Rick Davies for their useful comments. This project was part of RTB s research portfolio on partnerships (Theme 7). It was jointly financed by RTB and ILAC with grants from the Ministry of Foreign Affairs of the Netherlands and the International Fund for Agricultural Development (IFAD).

5 Contents Acronyms and abbreviations v Glossary vi Executive summary viii 1. Introduction 1 2. Literature review 2 3. Methods Limitations of the study 4 4. Types of research collaborations reported by RTB researchers Differences between RTB-induced and non-rtb-induced collaborations Research models The structure of RTB s research networks Analysis of the whole data set Analysis of the network of 92 survey respondents Centre-based analysis Conclusions 36 References 39 Annex 1. Description of the questionnaire 41 Annex 2. Definitions of the terms used in the questionnaire 43 Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas iii

6 List of figures Figure 1. Regional distribution of researchers who completed the questionnaire 4 Figure 2. Regional focus of all collaborations 13 Figure 3. Distribution of in- and out-degrees (whole network of 624 collaborators) 18 Figure 4. A tree-like network 19 Figure 5. Map of all 15 components in the whole network 20 Figure 6. Facilitation of communication by discipline of the researchers in the main component of RTB s network 21 Figure 7. Women-centred sub-network and components 23 Figure 8. RTB induced sub-network and components 24 Figure 9. Global research sub-network and components 26 Figure 10. African research sub-network and components 26 Figure 11. Latin American research sub-network 27 Figure 12. Asian research sub-network 28 Figure 13. Collaborations between researchers and non-research actors 28 Figure 14. Distribution of in- and out-degrees (sub-network of 92 respondents) 30 Figure 15. Sub-network of 92 respondents main component, reciprocity of links, and isolates 30 Figure 16. Intermediary power of nodes (betweenness) by disciplines 31 Figure 17. Intermediation (betweenness) of CGIAR centres in the full RTB network 33 Figure 18. Interactions among centres and RTB-induced links (92 respondents) 34 Figure 19. Two-mode network: research areas shared by CGIAR centres 35 List of tables Table 1. Types of organizations mentioned, by type of collaboration and relative importance 6 Table 2. Collaborations by type of organization and research area (as a percentage of collaborations by research area) 8 Table 3. Collaborations by CGIAR centre 9 Table 4. Research areas of CGIAR centres mentioned by RTB researchers 10 Table 5. Location of the collaborations 11 Table 6. Research areas of all RTB collaborations 11 Table 7. Purpose of capacity-building collaborations 12 Table 8. Purpose of advocacy collaborations 12 Table 9. Location of collaborative activities (by gender of collaborator) 13 Table 10. Geographic focus of RTB-induced and other collaborations 13 Table 11. RTB s interactions by region and type of organization 14 Table 12. RTB s research and non-research collaborations by region 14 Table 13. RTB s interactions with different types of non-research organizations by region 15 Table 14. Characterization of collaborative research activities 15 Table 15. Number of respondents that reported a specific number of collaborators and were mentioned as collaborators by others 18 Table 16. Distribution of the components by size 19 Table 17. Proportion of female collaborators per component in the women-centred sub-network 23 Table 18. Links between and within RTB s four member centres 25 Table 19. Number of researchers with a specific number of links 30 Table 20. Intermediation of CGIAR centres (betweenness) 32 Table 21. Density matrix intensity of links within and between centres 33 Table 22. External Internal (E I) index by centre 34 Table 23. Regional distribution of links by centre 35 iv Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas

7 Acronyms and abbreviations AfricaRice: Africa Rice Center Bioversity: Bioversity International CBO: community-based organization CGIAR: A global research partnership for a food-secure future (formerly Consultative Group on International Agricultural Research) CIAT: Centro Internacional de Agricultura Tropical (International Center for Tropical Agriculture) CIMMYT: Centro Internacional de Mejoramiento de Maíz y Trigo (International Maize and Wheat Improvement Center) CIP: Centro Internacional de la Papa (International Potato Center) CIRAD: Centre de coopération Internationale en Recherche Agronomique pour le Développement (Agricultural Research for Development), France FAO: Food and Agriculture Organization of the United Nations GFAR: Global Forum on Agricultural Research GIS: geographic information system GIZ: Deutsche Gesellschaft für Internationale Zusammenarbeit GmbH ICRAF: International Centre for Research in Agroforestry (also known as the World Agroforestry Centre) ICRISAT: International Crops Research Institute for the Semi-Arid Tropics ICT: information and communication technology IFAD: International Fund for Agricultural Development IFPRI: International Food Policy Research Institute IITA: International Institute of Tropical Agriculture ILAC: Institutional Learning and Change Initiative ILRI: International Livestock Research Institute IRRI: International Rice Research Institute IWMI: International Water Management Institute MDS: multidimensional scaling NGO: non-governmental organization RTB: Roots, Tubers and Bananas SNA: social network analysis SPIA: Standing Panel on Impact Assessment Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas v

8 Glossary This glossary is intended to help non-specialists understand this report; technical terms have been avoided as much as possible in these definitions. Betweenness centrality: For node i, this is the number of shortest paths between other nodes that run through i. It can also be expressed as a percentage of the maximum possible betweenness that a node could have. Centrality or global centrality: A node is globally central if it lies at short distances from many other nodes (see definition of distance ). Centralization: This measures how central a node is in a network by looking at the differences between the centrality scores of the most central node and those of all other nodes. The most common centrality scores are degree and betweenness. Clustering coefficient: For node i, this is calculated by dividing the number of links between the nodes close to i by the number of links that could possibly exist between them. The clustering coefficient for the whole network is the average of the clustering coefficients for all the nodes in the network. Component: This refers to a set of nodes that are connected through one or more paths, but that have no connections outside this group. Cut-point: A node whose removal would increase the number of components by dividing the network or any of its components into two or more disjointed subsets is a cut-point. Degree: This is the total number of nodes to which a particular node is connected. Density: This is the number of links in a network divided by the maximum possible number of links. Diameter: This is the largest distance (or geodesic distance) in the largest subset of connected nodes (i.e., the largest component; see definition of component ). Distance (or geodesic distance): This is the length of the shortest path between two nodes. Ego network: The ego network for node i contains i and the other nodes that i is connected to (i s neighbours). It is possible to define as many ego networks as there are nodes in the network. The size of the ego networks is set by the researcher, usually extending two or three steps from the central node. Eigenvalue centrality: For node i, this is the sum of its connections to other nodes, weighted by their degree centrality. External Internal (E I) index: This index compares the numbers of links within groups and between groups. It is calculated by taking the number of links of group members to outsiders, subtracting the number of links to other group members, and dividing by the total number of links. Freeman s graph centralization index: This index is the proportion of existing links to the links that would be present in a star graph of the same dimension (see definition of star graph). In-degree: This is the total number of links that end in a particular node. k-core: This is a connected set of nodes in which each node is adjacent to at least k other nodes; all the nodes within the k-core have a degree greater than or equal to k. Link: A link is a connection between two nodes. In this study it is also known as a collaboration. Main component: This is the largest component in a network. Navigability: This refers to the possibility that any particular node can reach any other node in the network. The small world property means that short paths exist between any two nodes and the scale-free property implies that nodes can find those paths using only local information. Neighbourhood: For node i, this includes all the nodes that are connected directly or indirectly to i and that are within a predefined distance, usually, two steps. vi Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas

9 Node: A node is a point (actor) in a network; in this study it refers to each survey respondent and any person named by a respondent as a partner or collaborator. Out-degree: This is the total number of links that originate in a particular node. Path: A path is a continuous sequence of alternating nodes and links that starts at a particular node and ends at a different node. Power law: A distribution where the frequency with which an event occurs varies as a power of some attribute of that event (e.g., its size) is said to follow a power law distribution. The number of links (degrees) in many networks follows power law distributions. These are networks in which a few nodes have many links while most nodes have very few links. The distribution of links in the World Wide Web is an example of a power law because Google or Amazon have millions of links while the vast majority of nodes have fewer than 20 links. Scale-free network: This is a network where the distribution of degrees follows a power law. A major feature of scale-free networks is that they do not have a representative parameter (e.g., the mean does not explain the behaviour of the distribution). Small world: This property applies to a network with low average distances and a high clustering coefficient. All nodes in a network with a small world property are connected to other nodes by short distances and are organized in relatively local tight groups (i.e., they have high clustering coefficients). The best-known example of this property is the theory of six degrees of separation. Star graph: A network where all nodes are only connected to one central node is a star graph; there are no links other than those to the central node. A star graph has the highest possible centralization score. Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas vii

10 Executive summary CGIAR is implementing a change process that aims to develop new approaches for research and innovation, focus research activities and increase the collaboration among centres and with other partners. An essential component of the change process is the creation of 16 research programmes, known as CGIAR Research Programs; Roots, Tubers and Bananas (RTB), which started operations in January 2012, is one of them. Given the complexity of CGIAR and the breadth of the reforms, successful implementation of the change process will require substantial adaptation of goals and strategies as the process evolves. This report describes a cost-effective, easy-to-use methodology based on social network analysis (SNA) to monitor changes in a CGIAR Research Program with the goal of providing useful information primarily to RTB s management and stakeholders, but also to other CGIAR Research Programs and CGIAR as a whole. This report describes a pilot project that developed a methodology that seeks to contribute to answering important questions about RTB s impact pathway by identifying the actors RTB is collaborating with. The methodology used a user-friendly, Internet-based survey to collect information about active collaborations between RTB-financed researchers and a wide range of partners. Collecting the information and cleaning the database required almost three months of intensive work by the research team and the active support of RTB s director. All researchers mentioned in RTB s project documents were asked to list the collaborations they engaged in while conducting their research activities along with some attributes of each collaboration, including type and topic of research conducted, gender of the collaborator, geographical focus and location of activities. The researchers were asked to report only collaborations that had been active in the 12 months prior to the survey. Although it was not possible to fully map RTB s impact pathway, it was possible to sketch them. Analysis of the survey data showed that about 80 percent of RTB s collaborations involve other researchers. A reduction in the proportion of partnerships with non-research partners was observed in the RTB-induced collaborations. This is to be expected in the initial phase of a complex programme that seeks to build up links between CGIAR partners and to make the existing research portfolio more coherent. With the explicit priority CGIAR places on partnerships with actors in the agricultural innovation system, new partnerships outside CGIAR are expected to emerge in the near future. The lack of diversity in RTB s research network could affect its productivity. Studies of the interactions between research and diffusion networks have shown that often these networks have different structures and that they share different types of information. It has also been shown that researchers who interact with different types of actors are more productive academically and more creative than researchers who only work with other researchers. To address these issues, RTB should explore new approaches to conducting research as part of innovation processes, including sharing different types of information among different types of actors. It should be noted that the process should not involve only sharing scientific information with non-scientists (e.g., extension) but also passing information from non-scientists to researchers so that the latter can better understand the needs of the potential users and the opportunities created by their research. While the centres and the CGIAR Research Programs are the operative and administrative units, increasing CGIAR s impact will require focusing strongly on collaborations across the CGIAR Research Programs and on collaborations with other actors in the agricultural innovation system. From the perspective of evaluation, understanding coalitions of CGIAR Research Programs and the changing interactions among research and non-research actors should be an important component of the evaluation of the CGIAR Research Programs. The interactions with non-research partners follow geographical patterns, since collaborations with extension organizations and private firms dominate in Africa, Asia and Latin American, while interactions with a global focus involve mainly policy-makers and private firms. RTB has few direct contacts with non-researchers but there may be more indirect links. Mapping the networks of all RTB s partners could shed light on this issue, but the pilot project showed that this would be an extremely difficult and costly endeavour. A more manageable alternative is to map other CGIAR Research Programs, especially those that collaborate closely with RTB, to assess the nature and evolution of their joint impact pathway. viii Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas

11 The CGIAR Research Programs were created as umbrella organizations for pre-existing projects with the expectation that over time they would reshape CGIAR s research portfolio according to predetermined priorities. As RTB was only recently created, RTB s research activities do not yet follow a coherent strategy and this is reflected in the network s sparse connectivity especially the dearth of reciprocal interactions. This architecture hampers the sharing of information across RTB and with partners essential for integrating the research and innovation processes. However, RTB has already induced important changes in its research portfolio, fostering greater interaction among CGIAR centres, and refocusing partnerships according to the partners capabilities and RTB s research priorities. The survey was only partially successful in capturing the different types of research conducted by RTB researchers. From the answers it was clear that many researchers were not familiar with different research approaches (e.g., the difference between on-farm research and action research ). This lack of understanding constrains their ability to explore alternative research models and, therefore, diminishes their ability to achieve CGIAR s development goals. Most researchers engage in multidisciplinary networks, but it is not clear to what extent their research is influenced by other disciplines. Further research should explore how other disciplines and non-research actors influence RTB researchers. In addition to the information on the structure of RTB, this pilot project has provided useful lessons about the methodology s possibilities and limitations. On the one hand, the information enabled mapping of the research networks and provided a baseline that will facilitate future monitoring of RTB s evolution. The baseline contains information on the type and number of research organizations that participate in RTB s research activities, gender issues, research approaches and locations, the range of disciplines and geographic focus. Repeating this exercise periodically will enable RTB to identify changes in its research portfolio and to link those changes with learning along its impact pathway. On the other hand, the project did not collect information on financial issues or the size of the research projects (in terms of the number of partners and disciplines involved and the geographic area covered) or the expected length of the projects. Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas ix

12

13 1. Introduction The CGIAR change process aims to develop new approaches for research and innovation, focus research activities and increase the collaboration among centres and with external partners (CGIAR, 2011). An essential component of the change process is the creation of 16 research programmes, known as CGIAR Research Programs. Roots, Tubers and Bananas (RTB), which started operations in January 2012, is one of the CGIAR Research Programs. RTB brings together four CGIAR centres (Bioversity, CIAT, CIP and IITA) and many other partners. It includes research on six sets of crops: bananas (and plantains), cassava, potatoes, sweet potatoes, yams and other roots and tubers. Given the complexity of CGIAR and the breadth of the reforms, successful implementation will require substantial adaptation of goals and strategies as the process evolves (Christensen, Anthony and Roth, 2004). The execution of such an adaptive strategy requires timely information on the evolution of the change process (Patton, 2010; Davila, Epstein and Shelton, 2006). In the case of the CGIAR Research Programs, the implementation of an adaptive strategy has two main components: defining impact pathways and monitoring whether the actions of the CGIAR Research Programs conform to those pathways. For the impact pathways to provide effective guidance for action, they should first identify the partners the CGIAR Research Programs expect to work with, which may differ from the partners they actually work with. Such discrepancies have been identified in many organizations and are usually explained in terms of the espoused theory and the theoryin-use (Argyris and Schön, 1974). 1 For example, a CGIAR Research Program may define its impact pathway (the espoused theory) as developing research outputs (e.g., basic seed) that are passed on to private firms (e.g., seed companies) who bring them to the farmers. However, the analysis of the actual interactions (the theory-in-use) may indicate that the CGIAR Research Program researchers interact mainly with other researchers and not with the firms. Recognizing differences between the espoused theories and the theories-in-use can help the CGIAR Research Programs to discover emerging problems and opportunities. Comparing the espoused theory with the theory-in-use requires describing the impact pathway with a certain 1 The espoused theory is the theory of change that an organization explicitly develops, which guides its decisions; the theory-in-use is the actual strategy that emerges from the many individual decisions made by the members of an organization. The gap between the espoused theory and the theory-in-use differs from the discrepancies that arise between actual and planned activities, in the sense that the former is not usually conscious and organizations are often unaware of its existence. degree of detail and making value judgements about observed discrepancies. For example, how many nonresearch actors should a CGIAR Research Program interact with? And how large should the difference between the expected and actual pathway be to trigger a managerial intervention? Appropriate answers to these questions cannot be provided in the abstract; they require an understanding of the different types of research conducted by the CGIAR Research Programs and of the role of research in pro-poor agricultural innovation processes. Previous research has found that research and innovation networks change as the research process matures (Kratzer, Gemuenden and Lettl, 2008; Gay and Dousset, 2005). However, no practical guidelines were found on how to use social network analysis (SNA) and the impact pathways for monitoring and evaluation of research for development activities. This report describes a pilot project that developed a methodology that seeks to contribute to answering important questions about RTB s impact pathway by identifying the actors RTB is collaborating with; future research will need to address the comparison of actual collaborations with those posited by the impact pathway. This project was commissioned by the management of RTB with the explicit goal of supporting organizational change not as an accountability tool. Monitoring changes in research and diffusion processes as they are implemented complements the management system being developed by the CGIAR Consortium Office. The project had three objectives: To help RTB understand how its research products are developed, how other actors in the agricultural innovation system influence (and are influenced by) the research portfolio and how the research outputs are diffused; To provide a baseline for monitoring the evolution of the RTB research networks, by identifying the main partners, disciplinary areas of research, geographical focus and how the research is being conducted; To develop a methodology that can be used to monitor RTB s learning along its impact pathway. Section 2 reviews the approaches taken by previous studies of research networks and Section 3 describes the research methods of the current study. Section 4 uses basic tables to analyse important features of the collaborations of RTB researchers. Section 5 applies social network analysis (SNA) methods to the structure of the researchers networks. Section 6 presents the conclusions of the study. Introduction 1

14 2. Literature review Three reasons are behind the recent upsurge of analysis of research networks. First, research is increasingly implemented by multidisciplinary, multi-institutional teams that form networks with both local and global connections, and researchers and policy-makers are increasingly interested in understanding the structure and dynamics of these networks (Cassi et al., 2008; Powell and Grodal, 2005; Bennett, Gadlin and Levine-Finley, 2010; Wagner, 2008). Second, programmes to foster interdisciplinary, inter-institutional collaborations between researchers and other actors in innovation systems have been implemented in several countries and policy-makers are asking about the impact of these programmes on research activities. Third, it has been found that scientific collaborations are an important influence on researchers productivity (Klenk, Hickey and MacLellan, 2010; Wagner, 2008), and collaborations with non-researchers help researchers to better contribute to innovation processes and to become more productive and creative (Rivera-Huerta et al., 2011). When researchers learn how to participate in innovation processes, they change the way they conduct research (Ekboir et al., 2009). Several methodologies have been used to analyse research networks. For this project, only publications that reported on SNA approaches were reviewed. Published analyses of the structure and dynamics of research networks have focused on networks classified in three different ways: networks identified from joint publications (co-authorship relationships), networks identified from project documents (project partnerships) and networks based on data from direct surveys of researchers Studies based on co-authorship relationships explore databases of scientific articles to identify patterns of interaction. Two authors are considered to have a direct link if they have co-authored at least one paper. When a large number of co-authorships are identified, a map of interactions emerges. If several years of data are available, it is possible to analyse the evolution of these networks. Due to the public availability of data, the structure and evolution of co-authorship networks have been well studied; recent papers include Klenk, Hickey and MacLellan (2010), Palla, Barabási and Vicsek (2007), Newman (2004), and Lyrette (2002). These data only document collaborations that resulted in a scientific publication while they ignore connections based on capacity-building and advocacy activities. Also, they do not include informal collaborations and exchanges of information, which have been recognized as important components of scientific work even though they may not result in scientific publications (Varga and Parag, 2009; Wagner, 2008; Kratzer, Gemuenden and Lettl, 2008). Another problem with this approach is that publication patterns vary among scientific disciplines (Okubo, 1997); some researchers may be included as co-authors for social reasons (La Follette, 1992) or because they provide data or equipment (Stokes and Hartley, 1989). Finally, scientific works often take several months to be published, by which time the collaboration may no longer be active. Studies of project partnerships analyse project documents, especially joint proposals and publications, to identify research collaborations. The main difference with co-authorship relationships is that project partnerships are more likely to include non-scientific collaborations. In recent years, many countries have implemented projects promoting research networks. Among these, the European Framework Programmes and the Canadian Network of Centres of Excellence were particularly important because of their continuity and level of investment. Some of these programmes have been analysed by combining SNA with qualitative study methods (Klenk and Hickey, 2012; Protogerou, Caloghirou and Siokas, 2010; Cassi et al., 2008). One drawback is that these data only capture interactions that are relevant for documentation of the project, which may not have resulted in effective collaborations. These data also overlook informal interactions. Approaches based on co-authorship relationships and project partnerships cannot be used in research areas with low propensity to publish (such as development of agricultural equipment or action research projects) or where informal interactions are common. Additionally, they may lead to the inclusion of collaborations that exist only on paper. Following Bozeman and Corley (2004), we instead adopted a strategy based on self-reported information, which captures active formal and informal collaborations as well as non-research interactions. This approach is described in the next section. 2 Monitoring the composition and evolution of the research networks of the CGIAR Research Program on Roots, Tubers and Bananas

15 3. Methods The mapping of RTB s networks was conducted in three stages: data collection, data cleaning and data analysis. The data for this project were collected by asking RTB researchers to complete a questionnaire about all the collaborations they have engaged in as part of their research activities during the 12 months prior to the survey. A research collaboration was defined as a relationship where researchers effectively cooperate with other actors in the innovation system (including researchers, extension agents, research managers, input suppliers, output buyers and policy-makers) and that includes both formal and informal links. This definition is broader than that of a partnership, which is usually defined as a formal interaction (Horton, Prain and Thiele, 2009). Potential study respondents were identified from a variety of sources. RTB provided four lists of professionals working with RTB-funded projects, including researchers, communication specialists and support professionals. The lists were collated and checked for inconsistencies, missing information was completed, and non-researchers and those no longer working for RTB were excluded. Some researchers forwarded the names of colleagues who were not included in the original lists, and they too were added to the list of potential respondents. In all, 126 researchers were invited to complete the questionnaire, including 33 women and 93 men. Only seven professionals in the list were not affiliated with CGIAR. The final database included each researcher s full name, gender, job title, institutional affiliation, the country where she or he was based and contact information. This updated list of researchers was the first output generated by the project. An Internet-based, user-friendly questionnaire was designed so that completion would take half an hour at most (full details are provided in Annex 1). In the questionnaire, researchers were asked to identify their collaborators and describe the nature of each of these relationships. The only information provided to the researchers was the definition of a collaboration (see Annex 2), leaving to them to decide which relationships to report and how to categorize them. In other words, their responses indicated how they perceived the interactions, which may be different from what those interactions actually involved. An important limitation of self-reported information is that researchers may forget to report important collaborations because they value their interactions differently. One alternative is to present respondents with a list of potential contacts and let them choose which ones to report as collaborators (Marschall, 2012). This alternative was not feasible in this study because a complete list of all RTB partners was not available. Different approaches to agricultural research have been developed in the last half-century, including traditional research, on-farm research, participatory methods and action research. However, the names of the different approaches were not used in this survey because it was expected that many researchers would not be familiar with them. To describe the nature of each reported collaboration, the researchers were instead asked to choose among six statements (see Annex 2). To avoid the problem of memory lapses with regard to past collaborations and to capture active relationships instead, researchers were asked to report only collaborations that had been active in the 12 months prior to the survey. The questionnaire was pretested before being provided to respondents. However, a problem with the definition of the type of research conducted by the respondents and another with the lack of variability found for two variables (frequency and importance of the collaboration) only became evident after all the information had been collected. There were two stages of data collection. In the first stage, the questionnaire was sent to the 126 researchers on the compiled list, 90 of whom responded, including 61 who provided complete and valid answers. The survey was opened for a second time after three problems were identified. First, the effective response rate of the first round (49 percent) was considered insufficient for a valid analysis of the RTB network. Second, 22 respondents reported exactly 10 collaborations. This was the limit mentioned on the first page of the questionnaire, but respondents could report more collaborations by requesting a new link. The decision to include only 10 pages on which to report collaborations was based on the assumption that many researchers would be discouraged if they were asked to report more contacts. It was also expected that few researchers would report Methods 3