A software module for import of theses and dissertations to CRISs

Transcription

1 A software module for import of theses and dissertations to CRISs Lidija Ivanović a, Dušan Surla b a Faculty of Education, University of Novi Sad, Serbia b Faculty of Sciences, University of Novi Sad, Serbia Summary This paper presents a module for import of theses and dissertations to CRISs. The main feature of the module is support for various formats of metadata of theses and dissertations. The model is extensible with plugins that provide support for import of other scientific-research results (papers published in journals, papers published in conferences proceedings, monographs, etc.) in various metadata formats (Dublin Core, ETD-MS, CERIF, etc.). The module imports data through a user interactive process by which consolidation of data is achieved. 1 Introduction Scientific-research results can be found in various systems accessible via Internet such as: institutional repositories, digital libraries, publishers platforms, research information systems, etc. Europe aims to create a unique database for all relevant scientific-research results. During the two recent decades, CERIF standard for research information systems has been being developed. Development of systems based on CERIF standard (called CRIS systems) is rapidly increased in recent years. It is evident that migration of data about scientific-research results from various systems to CRISs is necessary. Because of diversity of sources from which data should be migrated, a module for data import to CRISs should be able to import data in various metadata formats from various sources. Furthermore, the module should provide support for consolidation of imported and existing data. The paper presents a software module for import of theses and dissertations implemented for the CRIS of the University of Novi Sad (CRIS UNS). The module software architecture is extensible with plugins for import of theses and dissertations from various sources, as well as for import of other types of scientific-research results. Moreover, import of data through interactive userinterface enables creation of database of unique authority records about authors, advisors, boards members and institutions where theses and dissertations have been defended. The proposed software module can be easily adopted for data migration between other software systems. The module was verified by migration of data from Digital library of bachelor theses, master theses and doctoral dissertations of the University of Novi Sad (DIGLIB UNS) to CRIS UNS. This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 1

2 2 Related work Nowadays, important part of scientific knowledge is available via published scientific-research results in digital form along with associated metadata accessible through various applications such as digital libraries, research information systems, institutional repositories, etc. In recent years, the relationship between those systems and possibility of integration of those systems is discussed (Joint 2008; Krause 2002). The paper (Dijk et al. 2006) describes the NARCIS portal that is access point to all scientific-research data created by researchers from the Netherlands. That system joins scientific-research information system and Digital Academic Repositories (DARENET) in the Netherlands. Furthermore, the paper (Olivier 2009) presents collaboration between scientific-research information system and digital library of the University of Pretoria. Moreover, there is similar collaboration between scientific-research information system and institutional repository of the University of Glasgow (Nixon 2010). DRIVER is a global network of institutional repositories (Peters & Lossau 2011). Purpose of integration of systems containing scientific-research data is to avoid or reduce duplicating input of the same data in various systems, as well as to increase quality of services based on those data. Existing attempts of integration of systems containing scientific-research data are based on data models that can store metadata about scientific-research results prescribed by various standards. One such model is CERIF compatible data model based on MARC 21 format (Ivanović et al. 2011a). Metadata about theses and dissertations in Dublin Core, ETD-MS, MARC 21 and CERIF format can be mapped to this model (Ivanović et al. 2012a). 3 Module architecture The module imports complete network of objects related to target objects that should be imported into a CRIS. For example, import of theses and dissertations beside target objects (Theses/Dissertations) also imports the following related objects: Institutions (where thesis is defended), Persons (authors, advisors, boards members). The module is implemented using Java platform, JSF technology and open source libraries written in Java. The module architecture is shown in Figure 1 by UML components diagram. Dublin_Core_Reader Institution_Importer MARC_21_Reader Import_Manager Person_Importer CERIF_Reader Records_List Import_Record DIGLIB_UNS_Reader Thesis_Dissertation_Importer Figure 1. The module component diagram This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 2

3 The module consists of: Reader components (Dublin_Core_Reader, MARC_21_Reader, CERIF_Reader, DIGLIB_UNS_Reader), Importer components (Institution_Importer, Person_Importer, Thesis_Dissertation_Importer), The Import_Manager component Reader components are intended to transfer input data from various formats (Dublin Core, MARC 21, CERIF, DIGLIB UNS, etc.) to list of objects. Scalability of the Reader components: The module for data import is extensible with a new Reader component that provides support for import of a new metadata format. Also, the existed Reader components can be extended to provide support for transformation of other scientific-research results such as: papers published in journals (Papers the target objects, Journals and Persons the related objects), papers published in the conferences proceedings (Papers the target objects, Conferences, Proceedings, Persons the related objects), i.e., the Reader components can be extended to provide support for transformation of input metadata format to extended list of objects (not limited to Institutions, Persons, Theses/Dissertations). Importer components are intended to establish relations between the module and a CRIS. The appropriate Importer component performs the following functions for an object that should be imported (Institution, Person, Thesis/Dissertation): Checks whether the object already exists in the CRIS. Shows all objects stored in the CRIS s database which metadata meet the criteria of similarity with the metadata of the object. Objects could be ordered or even more an object can be automatically selected according to decisions previously made by user. It will enable the system to learn from the user decisions during massive migrations in order to set up real-time import services for data updates. The criteria of similarity of two metadata is defined as follows: o o o The Levenshtein distance (edit distance) between two words must be less than or equal to the integer quotient which is obtained by dividing the number of characters of longer word with five. If shorter of the metadata (with fewer words) has more than 5 words, then the previous criteria are valid for 80% of the words. Cyrillic and Latin scripts are equal. Store the object s metadata by the following algorithm: o o If there is no dilemma whether the same object already exists in the CRIS s database, the Importer component stores the object as a new record in the CRIS s database. Whenever there is dilemma whether the same object already exists in the CRIS s database, the module shows a potentially the same object, and the user who started the import has to decide what to do with the object s metadata: Leave metadata that already exist in the CRIS s database, Overwrite existing metadata with the object metadata, Perform merging of metadata controlled by the user, This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 3

4 Store the object s metadata as a new record. The Import_Manager component coordinates the whole process of data import: Invokes appropriate Reader component to create a list of objects from input metadata format, For each object from the list the Import_Manager component invokes appropriate Import component that imports the object. If the module for import data is extended with support for transformation of new type of scientific results (e.g. papers published in journals) to the extended list of objects, some Importer component (e.g. Journal_Importer) must be added to check existence and to import the new type of objects (e.g. Journal). 4 Case study: migration of data from DIGLIB UNS to CRIS UNS The proposed module for data import is verified by migration of data from Digital Library of bachelor theses, master theses and doctoral dissertations of the University of Novi Sad (DIGLIB UNS) to CRIS of the University of Novi Sad (CRIS UNS). Data about bachelor theses, master theses and doctoral dissertations, data about institutions on which theses and dissertations have been defended, as well as data about authors, advisors and boards members have been migrated. As we already stated, DIGLIB UNS and CRIS UNS has been developed by the same institution. Therefore, component DIGLIB_UNS_Reader has had direct access to relation database of the DIGLIB UNS system, i.e., format of imported data has been defined by schema of relation database of DIGLIB UNS. So, the imported data haven t been in XML format; data have been read from database management system (DBMS) that stores DIGLIB UNS data. This is a case study of using module for import described in this paper, but it is evident that data for import can be obtained by various protocols such as Z39.50, SRU, OAI-PMH, and data can be in various formats defined by XML schemas or ontologies for describing a semantic web service. 4.1 DIGLIB UNS DIGLIB UNS ( is digital library of bachelor theses, master theses and PhD dissertations developed for the needs of University of Novi Sad. The system is in use since Theses and dissertations in this system can be described using set of metadata by two languages: Serbian and English. The set of metadata is prescribed by the University of Novi Sad and includes: author, advisor, board s chair, board s member, title, subject / keywords, abstract, note, language, physical description [chapters / pages / references / tables / pictures / graphs / appendixes], publisher, publication year, record type, thesis type (bachelor thesis, master thesis, PhD dissertation), scientific field, scientific discipline, accepted by competent scientific institution on [date], institution (name, city and address, country), defended on [date], holding data. Besides those metadata, electronic theses are also stored (pdf, doc, etc.). Development of this system is described in the papers (Zarić et al. 2004; Zarić & Surla 2004; Milosavljević & Konjović 2004). The system provides searching theses and dissertations via Internet using the followings metadata: keywords, title, abstract, extended abstract, author, scientific field and scientific discipline. This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 4

5 4.2 CRIS UNS CRIS UNS ( is CERIF-compatible scientific-research information system developed for needs of the University of Novi Sad. The system has been under development since 2008 within BISIS project ( The system is built on CERIFcompatible data model based on MARC 21 format described in the paper (Ivanović et al. 2011a). This compatible model stores CERIF prescribed data about publications, patents, products, persons (researchers), projects, events and part of data set about organization units using appropriate MARC 21 format (bibliography and authority). For example, an instance of the CERIF entity ResultPublication and entities connected with it (ResultPublicationBiblNote, ResultPublicationTitle, ResultPublicationSubtitle, ResultPublicationNameAbbrev, ResultPublicationKeywords, Result- PublicationAbstract) are mapped to an instance of MARC 21 bibliographic record. The first phase of system implementation is main subject of the paper (Ivanović et al. 2010; Milosavljević et al. 2011). Automatic extraction of metadata from scientific publications for CRIS systems is described in the paper (Kovačević et al. 2011). Published results from the CRIS UNS database are publically available via Internet. Results metadata are entered by researchers, and researchers do not have to know CERIF and MARC 21 format. The CRIS UNS system besides international requirements prescribed by CERIF standard meets requirements related to results evaluation prescribed by the Ministry of Education and Science of the Republic of Serbia. Therefore, the system data model is extended with a few entities as it is described in the paper (Ivanović et al. 2011b). Data model and system architecture enables easy integration of CRIS UNS with librarian information systems, and enables interoperability between CRIS UNS and other CERIF-compatible scientific-research information systems of European country. The second phase of CRIS UNS development includes integration of institutional repositories of theses and dissertations of the University of Novi Sad with the CRIS UNS system, i.e. extension of the CRIS UNS system with subsystem that enables input of metadata about theses as well as electronic theses (pdf, doc, etc.). Theses and dissertations can be described by languages prescribed by MARC 21 ( using the following set of metadata: author, advisor, board s chair, board s member, title, alternative title, subtitle, keywords, abstract, extended abstract, note, language, ISBN, physical description, UDC, publisher, publication date, record type, content format, URI, access rights, thesis type (bachelor thesis, master thesis, PhD dissertation), name of author degree after defence, level of education, scientific field, scientific discipline, accepted by competent scientific institution on [date], institution, defended on [date], holding data. Metadata about theses and dissertations in Dublin Core, ETD-MS, MARC 21 and CERIF format can be mapped to the CRIS UNS model (Ivanović et al. 2012a). The second phase of CRIS UNS development is described in the paper (Ivanović et al. 2012b). 4.3 Data mappings Set of metadata for describing theses and dissertations within the DIGLIB UNS system is subset of metadata for describing theses and dissertations within the CRIS UNS system. The Table 1 shows in which way are DIGLIB UNS metadata about theses mapped to CRIS UNS metadata about theses. The first column holds names of DIGLIB UNS metadata, the second column holds names of appropriate CRIS UNS metadata, and the third column describes method of metadata storing in the data model of CRIS UNS system. As we already stated, data about published results This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 5

6 within the CRIS UNS system are stored using the MARC 21 bibliographic records, instead of the CERIF entity cfresultpublication. Therefore, the CERIF entity cfresultpublication_class in the CRIS UNS data model is renamed to MARC21Record_Class. DIGLIB UNS CRIS UNS Method of storing author author M21 df: 100 (1#) a advisor advisor M21 df: 700 (1#) a board s chair board s chair M21 df: 700 (1#) a board s member board s member M21 df: 700 (1#) a title title M21 df: 245 (00) a - alternative title M21 df: 246 (0#) a - subtitle M21 df: 245 (00) b subject / keywords keywords M21 df: 653 (##) a abstract abstract M21 df: 520 (3#) a - extended abstract M21 df: 520 (##) a note note M21 df: 500 (##) a language language M21 cf: 008 (35-37) - ISBN M21 df: 020 (##) a physical description [chapters / pages / references / tables / pictures / graphs / physical description M21 df: 300 (##) a appendixes] - UDK M21 df: 080 (##) a publisher publisher M21 df: 260 (##) b publication year publication date M21 df: 260 (##) c record type record type M21 ld: 6 - content format M21 df: 856 (##) q - URI M21 df: 856 (##) u - access rights M21 df: 540 (##) a thesis type - thesis type name of author degree after defence entity MARC21Record_Class M21 df: 502 (##) a - level of education M21 df: 502 (##) b scientific field scientific field entity MARC21Record_Class scientific discipline scientific discipline entity MARC21Record_Class accepted by competent scientific institution on institution (name, city and address, country) accepted by competent scientific institution on institution M21 df: 502 (##) g M21 df: 502 (##) c defended on defended on M21 df: 502 (##) g holding data holding data M21 df: 852 (##) a Table 1. Data mappings If a metadata is mapped to a particular data field of MARC21 record, method of storing (the third column of Table 1) is presented in the following syntax: This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 6

7 M21 df: <data field code> (<indicator1><indicator2>) <subfield code> The character # indicates that indicator is not defined. If a metadata is mapped to a particular character position of MARC21 record leader, method of storing is shown in the following syntax: M21 ld: <character position> If a metadata is mapped to a particular control field of MARC21 record, method of storing is shown in the following syntax: M21 cf: <control field code> (<start>-<end character position>) Numeration of character positions starts with 0 at leader and control fields. 4.4 The migration overview Data migration from the DIGLIB UNS system to the CRIS UNS system can be started by selecting a menu item allowed just for admin account. After that, the DIGLIB_UNS_Reader component (Figure 1) using Java Database Connectivity (JDBC) opens a connection to the database management system (DBMS) that stores the DIGLIB UNS data. The component using this connection retrieves a list of records that have to be imported: Institutions, Persons, Theses/Dissertations. After the list has been retrieved the Import_Manager component (Figure 1) coordinates import of Institutions, then import of Persons and at the end import of Theses/Dissertations, i.e. the component invokes appropriate Importer components. Whenever there is dilemma whether the imported object already exists in the CRIS s database, the module for import provides a list of similar objects using the dialog shown in Figure 2. Figure 2. Similar records The user who started the import has to decide what should be done with the imported object s metadata. The module provides the followings options: This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 7

8 Leave metadata that already exist in the CRIS s database: the button Select institution, Overwrite existing metadata with the imported object metadata: the button column Change, in the column in the Perform merging of metadata controlled by the user: the button in the column Merge, Store the object s metadata as a new record: the button Save stores imported object s metadata without changes; the button Change opens form for changing metadata and after that stores metadata. If the user selects to merge the data, the form shown in Figure 3 is opened. The user on this form can see imported object s metadata (within the input fields) as well as metadata of existing object in the CRIS UNS system database (messages next to the images ) that should be merged with the imported object. Figure 3. Records merging 5 Conclusions It is evident that data about scientific-research results are scattered throughout various systems. Interoperability of those systems is very important, i.e., data exchange between those systems is a requirement of the highest priority. The main goal of the CRISs is collecting of all relevant data related to scientific-research results of university, state or region. To achieve this goal faster and easier, migrations of data from various sources in CRISs are necessary. Thus creation of a flexible module for data import in various formats from various systems is very important. This paper presents such a flexible module for data import into CRISs. The module is extensible with plugins that provide support for new metadata formats and new types of scientific-research results. Usage of the presented software module for import data about theses and dissertations in CRIS UNS system enables consolidation of authority data (institutions, persons who are the authors of This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 8

9 theses and dissertations, advisors and board s members). Consolidated authority data increased quality of service for generation of various reports about scientific-research activities for faculties and university s management, as well as for generation of reports for Ministry of Education and Science of the Republic of Serbia and Provincial secretariat for Science and Technological Development of Vojvodina. Acknowledgements The work is partially supported by Ministry of Education and Science of the Republic of Serbia, through project no. OI174023: "Intelligent techniques and their integration into wide-spectrum decision support". References Dijk, E.; Baars, Chr.; Hogenaar, A.; Meel, M. van (2006): NARCIS: The Gateway to Dutch Scientific Information. Elpub 2006 Conference, Bansko, June 14-16, 2006, pp Ivanović, D.; Milosavljević, G.; Milosavljević, B.; Surla, D. (2010): A CERIF-compatible research management system based on the MARC 21 format. Program: Electronic libarary and information systems, Vol. 44, No. 3, pp Ivanović, D.; Surla, D.; Konjović, Z. (2011a): CERIF compatible data model based on MARC 21 format. The Electronic Library, Vol. 29, No. 1, pp , DOI: / Ivanović, D.; Surla, D.; Racković, M. (2011b): A CERIF data model extension for evaluation and quantitative expression of scientific research results. Scientometrics, Vol. 86, No. 1, pp , DOI /s Ivanović, L.; Ivanović, D.; Surla, D. (2012a): A data model of theses and dissertations compatible with CERIF, Dublin Core and EDT-MS. Online Information Review, Vol. 36, No. 4 (in print) Ivanović, L.; Ivanović, D.; Surla, D. (2012b): Integration of a Research Management System and an OAI-PMH Compatible ETDs Repository at the University of Novi Sad, Republic of Serbia. Library Resources and Technical Services, Vol. 56, No. 2 (in print) Joint, N. (2008): Current research information systems, open access repositories and libraries. Library Review, Vol. 57, No 8, pp Kovacevic, A.; Ivanovic, D.; Milosavljevic, B.; Konjovic, Z.; Surla, D. (2011). Automatic extraction of metadata from scientific publications for CRIS systems. Program: electronic library and information systems, Vol. 45, No. 4, pp , DOI: / Krause, J. (2002): Current Research Information as Part of Digital Libraries and the Heterogeneity Problem Integrated searches in the context of databases with different content analyses. Proceedings of the 6th International Conference on Current Research Information Systems, University of Kassel, August 29-31, 2002, pp Milosavljević, B.; Konjović, Z. (2004). Pretraživanje multimedijalnih informacija u sistemu digitalnih biblioteka magistarskih i doktorskih radova. Infoteka, Vol. 5, No. 1-2, pp This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 9

10 Milosavljević, G; Ivanović, D.; Surla, D.; Milosavljević, B. (2011). Automated Construction of the User Interface for a CERIF-Compliant Research Management System. The Electronic Library, Vol. 29, No. 5, pp , DOI: / Nixon, W. (2010): Enrich: Improving integration between an Institutional Repository and a CRIS at the University of Glasgow. Proceedings of the 10th International Conference on Current Research Information Systems, Aalborg, Denmark, June 2-5, 2010, pp Olivier, E., (2009): Open Scholarship & research reporting in tandem: creating more value. The African Digital Scholarship & Curation Conference, May 12-14, 2009, available at: (provereno 4. septembra 2011.) Peters, D.; Lossau, N. (2011): DRIVER: building a sustainable infrastructure for global repositories. The Electronic Library, Vol. 29, No. 2, pp Surla, D.; Konjović, Z.; Milosavljević, B.; Zarić, M.; Sladić, G.; Protić, Z.; Komazec, S.; Okanović, D. (2004): Prikaz realizacije mrežne digitalne biblioteke doktorskih, magistarskih i diplomskih radova. Infoteka, Vol. 5, No. 1-2, pp Zarić, M.; Surla, D. (2004): Implementacija protokola za prikupljanje metapodataka u mreži digitalnih biblioteka teza i disertacija. Infoteka, Vol. 5, No. 1-2, pp Contact Information Lidija Ivanović University of Novi Sad, Faculty of Education Podgorička Sombor Serbia lidija.ivanovic@pef.uns.ac.rs Dušan Surla University of Novi Sad, Faculty of Sciences Trg Dositeja Obradovića Novi Sad Serbia surla@uns.ac.rs This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. 10