RISIS data platforms New opportunities for science and innovation studies Peter van den Besselaar VU University Amsterdam

Size: px

Start display at page:

Download "RISIS data platforms New opportunities for science and innovation studies Peter van den Besselaar VU University Amsterdam"

Vivian Franklin
8 years ago
Views:

1 RISIS data platforms New opportunities for science and innovation studies Peter van den Besselaar VU University Amsterdam

2 Second generation science and innovation studies Existing: Restricted by external data: Official statistics Bibliometric data bases Some surveys / case studies Not integrated New: Heterogeneous (digital / big / linked open) data New techniques for coupling, integrating, analyzing and visualizing and quality assessment EC Brussels 6/4/14 2

integrated New: Heterogeneous (digital / big / linked open) data New techniques for

3 What would that bring? New and more complex questions about the functioning of the science & innovation system New approaches to old questions Dynamics and effects of collaboration not restricted to coauthoring International mobility and its effects New questions can be addressed Where emerge the good ideas, and how not restricted to highly cited papers Multi-dimensional impact assessment more media included / Alt metrics EC Brussels 6/4/14 3

questions Dynamics and effects of collaboration not restricted to coauthoring International mobility and its

4 Simple example: impact of a research infrastructure (ViBRANT) Use of the infrastructure by category of users s of visits per month: no manual analysis Self learning classifier Log files From IP address to type of user Academic; education; industry; government; and more Identifiable users for in depth further analysis (and type of use Duration; reading; downloading; uploading) EC Brussels 6/4/14 4

000s of visits per month: no manual analysis Self learning classifier Log files From IP address to

5 Vibrant: effect on research Coupling member data with heterogeneous publication and project data Can be done manually for 1 site, but not for 300! Brings together formerly unlinked researchers And cross-disciplinary research? collaboration? EC Brussels 6/4/14 5

6 Platforms in RISIS SMS (VUA) Disambiguation tools Analytical tools Integration tools Data (Research blogging; DBLP; etc) Cortext (UPE) Handling, analyzing and visualizing complex textual data EC Brussels 6/4/14 6

7 What could be integrated Research performing organizations and their performance Universities; PRO; Big companies; Fast growing knowledge intensive; Leiden Ranking Funding programs Evaluations Researchers Careers Scopus; WoS; Cordis; Patstat etc Linked open data; web resources Tools: SMS; Cortext; tools out there EC Brussels 6/4/14 7

Funding programs Evaluations Researchers Careers Scopus; WoS; Cordis; Patstat etc

8 Issues Data and tools integration platform Data cleaning tools Data integration tools Analytical tools Data quality assessment tools Skills Privacy Data ownership EC Brussels 6/4/14 8

9 Architecture Sketch DB1 RISIS Platform Website RISIS Analytical Tools (e.g. Cortext) DB2 Integration Engine API DB2 User Registration/Aut hentication/aut horization APPS APPS APPS EC Brussels 6/4/14 9

10 Integrating = entity resolution Difficult What level? Methods & tools General issue: trade off between Precise (but small) data Error rich (but big) data From quality versus large scale To quality assessment of large scale data EC Brussels 6/4/14 10

small) data Error rich (but big) data From quality versus

11 Example: Measurement error in coupling Project database & publication database Only part of the publication database: NL addresses / social science journals Coupling through family name and first initial Goes quickly; is easy; some manual control afterwards needed EC Brussels 6/4/14 11

science journals Coupling through family name and first initial Goes

12 But: factored error Synonyms: Same author, different names Complex names (M Obibio de Castro -> MO de Castro) Different initials: A D Donkers -> B Donkers Underestimation / overestimation Homonyms: Different authors, same name Only one initial -> more risk Overestimation Restricting the set Only one country; only social science: Less homonym problems Less overestimation Test: compare automatically created set with a correct (manually) created dataset EC Brussels 6/4/14 12

Only one initial -> more risk Overestimation Restricting the set Only one country; only social science: Less homonym

13 Nature of factored error EC Brussels 6/4/14 13

14 Random error: no real effect EC Brussels 6/4/14 14

15 Measurement error and bias challenge: how to assess error? EC Brussels 6/4/14 15

16 Complex example: careers and mobility What is size and geographical scope of international mobility? Change over time? Effect on quality of knowledge production and on careers? Obvious source: CV data Performance data on the individual and institutional level However: CV databases are not (easily) accessible, if existing at all. CVs on the Web as solution? EC Brussels 6/4/14 16

Obvious source: CV data Performance data on the individual and institutional level However: CV

17 Very heterogeneous. Retrieving CVs from the web Web CVs Name-Entity-Recognition-Disambiguater Four entities: Person; Organization; Location; etc Geonames Filter on organization name -> all organizations mentioned in CV plus the country Wikidata -> name variants (Wikidata = objects in Wikipedia) Position of geo-location in CV = time sequence Result = error rich international experience measure Manually testing of nature of error for a sample But if random error, large scale of the data enables reliable analysis Research questions Coupling with WoS; GS -> mobility and performance Coupling with Leiden ranking -> up/down mobility Typical mobility networks? EC Brussels 6/4/14 17

organizations mentioned in CV plus the country Wikidata -> name variants (Wikidata = objects in Wikipedia) Position of geo-location in CV = time sequence Result = error

18 Conclusions Big data and data integration have many promises Problems to solve: Entity resolution; new analytical tools; Legal and privacy issues Quality assessment tools Within RISIS: Develop platform and tools further; Access to platform and tools; Training and support EC Brussels 6/4/14 18

Quality assessment tools Within RISIS: Develop platform and tools

19 Thanks for your attention Questions? Comments?

Scientometrics as Big Data Science:

Scientometrics as Big Data Science: On integration of data sources and the problem of different types of classification systems Henk F. Moed (Elsevier, The Netherlands) With: Marc Luwel (Hercules Foundation,