Information access through information technology

Transcription

1 Information access through information technology 1 Created to support an invited lecture at the International Conference MDGICT 2009 in Tamil Nadu, India, December 2009 by Paul.Nieuwenhuysen@vub.ac.be

2 These slides should be available from the WWW site (note: BIBLIO and not biblio)/ 2

3 contents = summary = structure = overview 1. General: Access to information: the evolution 3 of this presentation 2. More specific: So much information, so little time

4 4 Information access through information technology Access to information: the evolution

5 5 Information is important for development Research / Education / Journalism Access to information = important

6 Libraries Online databases on DIALOG host computer Personal computers CDS/ISIS free cataloguing software by UNESCO Library automation with integrated library management systems CD-ROM with cheap 600 MB memory Internet Electronic mail

7 Libraries Online databases on DIALOG host computer Personal computers CDS/ISIS free cataloguing software by UNESCO Library automation with integrated library management systems CD-ROM with cheap 600 MB memory Internet Electronic mail WWW based on Internet (without search engine) Microsoft Windows 95 PC operating system WWW search engines to uncover the WWW DVD with more than 4000 MB memory CDS/ISIS for Windows with user-friendly interfaces Google WWW search becomes most popular information discovery tool

8 Authentication and authorization to access proprietary digital content Electronic full-text journals Local link generators Digital libraries, including repositories

9 Authentication and authorization to access proprietary digital content Electronic full-text journals Local link generators Digital libraries, including repositories Library automation with free software such as CDS/ISIS = ABCD Computing + memory for storage in the Cloud Social WWW, including Facebook, Flickr and YouTube Electronic books Google Scholar allows discovery of academic information free of charge Libraries implement Electronic Resource Management Systems

10 Authentication and authorization to access proprietary digital content Electronic full-text journals Local link generators Digital libraries, including repositories Library automation with free software such as CDS/ISIS = ABCD Computing + memory for storage in the Cloud Social WWW, including Facebook, Flickr and YouTube Electronic books Google Scholar allows discovery of academic information free of charge Libraries implement Electronic Resource Management Systems Better integration, aggregation, federation??????????

11 11 Information access: difficulties and bottlenecks Cost of content»books»journals»bibliographical databases Cost of computer hardware»at server side»at client side Cost of computer software

12 12 Information access: difficulties and bottlenecks Need to train personnel Fast evolution

13 13 Information access: difficulties and bottlenecks in developing countries Poor infrastructure»power supply»internet access

14 14 Information access through information technology So much information, so little time

15 15 Introduction: scattering of sources Users want to exploit information sources fast and effectively. This is hindered by the fact that digital, electronic information sources that may contain relevant information are created and scattered, distributed on numerous computers all over the Internet and the WWW.

16 16 Introduction: scattering of sources In other words: integration / aggregation is still far from perfect.

17 17 Introduction: scattering of sources difficulties Using many information retrieval systems costs time: 1. They must be used one after the other which requires many decisions and actions

18 18 Introduction: scattering of sources difficulties Using many information retrieval systems costs time: 2. They offer different user interfaces in the retrieval phase, which is confusing

19 19 Introduction: scattering of sources difficulties Using many information retrieval systems costs time: 3. They offer found information items in various data formats

20 20 Information access through information technology Problem statements

21 21 Problem statements 1. Which methods have been developed and applied to cope with this reality?

22 22 Problem statements 2. Which concrete applications are available and how can an end-user exploit systems created in this domain?

23 23 Problem statements 3. How can information intermediaries evaluate and apply these methods to bring information more efficiently to end-users?

24 24 Information access through information technology Various methods for information retrieval from scattered sources

25 25 Method 1: Merging = aggregating into a searchable database User User User User Search engine Aggregated database Database or web site or Database or web site or Database or web site or

26 26 Method 2: Federated searching through scattered databases User User User User Federated search engine Search engine Search engine Search engine Database Database Database

27 27 Both methods offer benefits to the users + Saves the users time that would be needed to execute queries towards various servers or to browse through various systems.

28 28 Both methods offer benefits to the users + The users have to learn only 1 user interface for searching and only 1 search syntax, instead of a user interface and a search syntax for each database.

29 29 Both methods offer benefits to the users + The system offers a uniform / consistent display of results in the output phase.

30 30 Method 1: Merging = aggregating into a searchable database User User User User Search engine Aggregated database Database or web site or Database or web site or Database or web site or

31 31 Method 2: Federated searching through scattered databases User User User User Federated search engine Search engine Search engine Search engine Database Database Database

32 32 Federated searching through scattered databases: why? The perfect trip: A cheap and nice flight A cheap and nice hotel A visit to to a nice museum Something nice to to read (free via your library)

33 Example 33 Federated searching: application: finding a suitable flight Example: for the USA

34 Example 34 Federated searching: application: finding a hotel room in some city

35 Example 35 Federated searching: searching in a museum

36 Example 36 Federated searching: searching in a library

37 37 So many digital libraries through information technology The various methods applied for end-users

38 38 Method 1: Merging = aggregating into a searchable database User User User User Search engine Aggregated database Database or web site or Database or web site or Database or web site or D or

39 39 Internet global subject directories: introduction They are virtual libraries with open shelves, for browsing. They are manually generated, man-made by many people. They can be browsed following a tree structure or a more complicated variation.

40 Example 40 Internet global subject directories: Yahoo! A hypertext global subject directory can be found at Entries are NOT rated. Accessible free of charge.

41 Example 41 Internet global subject directories: BUBL LINK A hypertext global subject directory to more than WWW sites for the higher education community can be found at [accessed 2008] Accessible free of charge. The categories are based on the well-known general Dewey classification system.

42 Example 42 Internet global subject directories: dmoz Open Directory Project A hypertext global subject directory can be found at Accessible free of charge. It is allowed to use the contents also in other systems; this is indeed done in Webbrain.

43 Example 43 Internet global subject directories: Librarians' Internet Index A hypertext global subject directory can be found at [accessed 2008] Accessible free of charge. Librarians select the sites and build the overview. The name Internet Index may create some confusion, because this term means in many cases an index as part of a full-text searchable database, that is an Internet search engine. This is NOT the case here.

44 Example 44 Internet global subject directories: Intute [accessed 2008] From 2006 Accessible free of charge. Offers a collection of hypertext subject directories that focus on academic information sources Also tutorials are offered about how to find information in specific subject domains.

45 45 Internet indexes: automated search tools The basic, fundamental architecture of the WWW does NOT include a system to discover relevant information resources. Thus search systems / engines have been implemented besides the WWW, mainly by commercial companies.

46 46 Internet indexes: automated search tools The situation is already since 1990 s as follows:»www = decentralised without central control = good»search through the WWW = centralised in a few systems, each one managed by a commercial US company = NOT (?) good

47 Example 47 Internet indexes: Google Available since 2001 with most of its features. The most popular search system since 2003.

48 Example 48 Internet indexes: Google Scholar Google Scholar allows us to search for more scholarly information sources, including journal articles. A beta (test) version has been available since November The system is accessible starting from the home page of Google as one of the additional services, or more directly from

49 Example 49 Internet indexes: Bing Available in 2009 in beta = test version. Replaces Microsoft Live

50 Example 50 Internet indexes: Scirus The search interface: Since Offers not only access to files in html format, but also to files in PDF.

51 Example 51 Internet indexes: Ask Available from: Offers a feature that is not offered by most other search systems: categorization = classification = refinement = clustering of search results, to help the user coping with the problem of ambiguity of meaning of the search query that was made

52 Example 52 Internet indexes: Yahoo! An Internet search system is offered through This is offered BESIDES the well-established, classical Yahoo! subject directory.

53 53 Current awareness services focusing on WWW pages: Google Alerts Available at and then see the page with additional services or more directly from Since Can discover relevant changed or new WWW pages for you in the future. Is based on the popular Internet index Google. Works with search queries given by you that are stored on their server computer.

54 54 Current awareness services focusing on WWW pages: Google Alerts

55 Example 55 Databases accessible over the Internet: example: scientificcommons Since 2007 Similar to OAISTER: Allows you to search the full texts in scientific open access repositories all over the world.

56 Example 56 Databases accessible over the Internet: example: Medline Medline/PubMed offers bibliographic descriptions of publications on medicine, free of charge.

57 57 Internet with WWW and printed books Since a few years, Internet with the WWW have become the primary information source for many people. However:»A lot of information is still distributed only in the form of printed books»the content of old printed books can still be interesting.»the content of most printed books is (still) not available on the Internet.

58 58 Public access book databases provided by bookshops To find currently available books, the bibliographic databases assembled by big bookshops are interesting. Several offer a good coverage. Many are accessible free of charge. The added price information can be useful for the acquisition and accounting department of a library or if an individual user wants to buy a book. Some provide a current awareness service, also free of charge. Take into account delivery costs: postage + import tax

59 Examples 59 Book databases accessible free of charge: examples in U.S.A. Amazon.com (US): This company offers also different, more local versions that offer books in other languages, such as note: amazon, NOT amazone Subject description is poor. Take into account delivery costs: postage + import tax

60 Examples 60 Book databases accessible free of charge: examples in U.S.A. Barnes and Noble (US): or

61 61 Search systems for books that are made available by dealers User Book dealer catalog database descriptions of books & real books for sale

62 62 Search systems for books that are made available by dealers User descriptions of books & real books for sale Book dealer catalog databases

63 63 Search systems for books that are made available by dealers User Book dealer catalog databases descriptions of books & real books for sale

64 64 Free public access search systems for books: multi-dealer databases» Multi-dealer database = database obtained after merging of several existing catalogue / inventory databases, which are managed and updated by individual dealers/shops/sellers.» Such a system can include from a few to more than shops/dealers.

65 65 Search systems for books that are made available by dealers User Book dealer catalog databases descriptions of books & real books for sale

66 66 Search systems for books that are made available by dealers User Multi-dealer database = merged book dealer databases Book dealer catalog databases descriptions of books & real books for sale

67 67 Search systems for books that are made available by dealers User Multi-dealer databases = merged book dealer databases Book dealer catalog databases descriptions of books & real books for sale

68 68 Search systems for books that are made available by dealers User Multi-dealer databases = merged book dealer databases Book dealer catalog databases descriptions of books & real books for sale

69 69 Free public access multi-dealer book databases: examples Amazon Marketplace: [accessed 2009] In synergy with the online bookshop Amazon on 1 WWW site: Used books are displayed alongside Amazon s new books. the world s biggest online book bazaar Subject description is poor. Take into account delivery costs: postage + tax

70 70 Free public access multi-dealer book databases: examples [accessed 2009] (NOT ANTIQBOOK unites more than 400 independent booksellers from all over the world. You can use our search pages for a free search of over 3.8 million books, and order them directly from your bookseller. Strong areas in our database are books from European booksellers, many of them specialist antiquarian booksellers. While ANTIQBOOK takes care that you can order safely from our booksellers we do not take part in their sales. We just bring you in touch with some of the finest booksellers in the world. You can order your books straight from the source, at their original prices and no hidden costs or markup fees.

71 71 Full-text databases of books: introduction Some organisations have scanned the contents of thousands of books, to make them full-text searchable through the Internet.

72 72 Full-text databases of books: Google Books Since 2005

73 Example 73 Online Public Access Catalogues: union catalogues of libraries Some systems offer access to the merged catalogues of several libraries, so-called union catalogues. Example: Copac is accessible free of charge.

74 74 Online access databases about journal articles: overview Thousands of fee-based online access databases offer bibliographies or full-texts of journal articles in particular subject domains and published by many publishers. Many publishers offer searchable bibliographies, but only of their own publications. (for instance Elsevier, Emerald, Sage) Only few large databases offer access to bibliographies of articles published in journals from many publishers, free of charge.

75 Example 75 Online access databases about journal articles: Ingenta Available from: Ingenta allows you to search a bibliographic database of millions of journal articles, including titles, authors, in many cases abstracts. The organisation claims to be The most comprehensive collection of academic and professional publications

76 Example 76 Online access databases about journal articles: Infotrieve ArticleFinder Available from: Infotrieve allows you to search free of charge in a bibliographic database of the articles of more than journal titles and conference proceedings, NOT full-text. Payment is required to receive the full text of a document.

77 Example 77 Online access databases about journal articles: Scirus The search interface: This is a specialised Internet index that allows you to search for selected scientific information (only) on the WWW. This includes the peer-reviewed articles in the journals that are published in ScienceDirect by Elsevier. Offered free of charge by Elsevier. An article can be downloaded in full-text format only when a fee has been paid to the publisher.

78 Example 78 Online access databases about journal articles: Google Scholar Google Scholar allows us to search for more scholarly information sources, including journal articles. A beta (= test) version has been available since November The system is accessible starting from the home page of Google as one of the additional services besides the normal, classical WWW search.

79 79 Finding images on the Internet: introduction + Several public access search systems are available free of charge to search for images / pictures (either artwork, either photos, or both) on the Internet. + When searching for images, the search results from such a system offer not only links to the image files on the Internet, but also directly small versions of the images (so-called thumbnails ).

80 Example 80 Finding images on the Internet: screen shot of a Google image search

81 Example 81 Finding images on the Internet: examples of search engines or through [accessed in 2009] The largest database in this category (at least in ). For each result, not only a thumbnail is offered, but also directly the origin with the readable URL; this makes it easier to guess the relevance of the document.

82 Example 82 Finding images on the Internet: examples of search engines [accessed in 2007] or or and then go to searching Images

83 Example 83 Finding images on the Internet: examples of search engines [accessed in 2007] Ask Offers no indication of the number of images retrieved, which is a disadvantage when many pictures are found, but only a few can be seen at the time.

84 Example 84 Finding images on the Internet: examples of search engines Available in 2009 in beta = test version. Replacing Microsoft Live and Yahoo Search?

85 85 Method 2: Federated searching through scattered databases User User User User Federated search engine Search engine Search engine Search engine Database Database Database

86 Example 86 Databases accessible over the Internet: example A global science gateway connecting you to national and international scientific databases and portals. Accelerates scientific discovery and progress by providing one-stop searching of global science sources.

87 Example 87 Databases accessible over the Internet: example Federated searching through various scientific databases

88 Examples 88 Meta-search systems on a server computer = = =

89 Example 89 Meta-search systems: server-based: example: Clusty Adds value by analysing the retrieved results / hits / links / WWW documents, in order to cluster / group / categorize / classify / map these under headings / classes / categories, to make further selections by the user / searcher easier and faster. Can accomplish this on the fly, that is WITHOUT preprocessing the documents before the search.

90 Example 90 Meta-search systems: server-based: example: Clusty screenshot in 2009

91 91 Free public access book meta-search systems: types We can make the following distinction between various types of meta-systems for searching: 1. Database resulting from merging several existing smaller databases = aggregator database In this case of books: multi-dealer database = listing service 2. Federated search system = cross-database search system

92 92 Search systems for books that are made available by dealers User Federated book search systems Multi-dealer databases = merged book dealer databases descriptions of books & real books for sale Book dealer catalog databases

93 93 Search systems for books that are made available by dealers User Federated book search systems Multi-dealer databases = merged book dealer databases Book dealer catalogue databases descriptions of books & real books for sale

94 94 Free public access federated search systems for books: examples [accessed 2009] Covers many book dealer databases and multi-dealer databases, including unique databases that are not covered by competing search systems. Can calculate the cost to ship/send a book to you, taking into account your country and currency. Searches only new books; to find used books, a companion system should be used. This is inconvenient if the user is interested in both types of books.

95 95 Free public access federated search systems for books: examples [accessed 2007, 2008, 2009] BookFinder Covers many book dealer databases and multi-dealer databases, including unique databases that are not covered by competing search systems. It is efficient that new and used books are searched in 1 action; the results are presented in 2 columns: new used.

96 Example 96 Online Public Access Catalogues: simultaneous searching: examples Simultaneous access to catalogues of libraries related to water, organised by IAMSLIC, using Z39.50

97 97 Federated searching offered by a university library Main goal of such a system is offering easy and fast access to various information sources and NOT sophisticated and complicated searching. The user interface is simple, in agreement with the aim of such a system.

98 98 Information access through information technology Comparison of methods for efficient information retrieval

99 99 Comparison of methods for efficient information retrieval 1. Merging databases 2. Federated searching Presearch analysis of all data + -

100 100 Comparison of methods for efficient information retrieval 1. Merging databases 2. Federated searching Presearch analysis of all data + - Size of the coverage - +

101 101 Comparison of methods for efficient information retrieval Presearch analysis of all data Size of the coverage Independent of Internet / WWW 1. Merging databases Federated searching

102 102 Comparison of methods for efficient information retrieval Presearch analysis of all data Size of the coverage Independent of Internet / WWW Up-to-date information 1. Merging databases Federated searching

103 103 Comparison of methods for efficient information retrieval Presearch analysis of all data Size of the coverage Independent of Internet / WWW Up-to-date information Speed of retrieval and display 1. Merging databases Federated searching

104 104 Comparison of methods for efficient information retrieval + The evolution of information and communication technology makes systems more powerful, easier to implement and use, and cheaper: + Merging information sources is pushed forward mainly by the decreasing costs of hard disks and of computer memory in general. + Federated searching is pushed mainly by the evolution of the Internet.

105 105 Both methods bring difficulties / challenges / problems - In many cases there are differences among the merged sources in the formatting/structuring of their database records in fields. This hinders - searching limited to a field - displaying selected fields only (such as title) - sorting of the displayed records on the contents of a particular selected field (such as author or date)

106 106 Both methods bring difficulties / challenges / problems - In many cases there are differences among sources in the metadata schemes that are applied in the databases to improve retrieval, such as»classifications»taxonomies»thesaurus systems»ontologies - This hinders the exploitation of the added value of such metadata.

107 107 Information access through information technology Conclusions

108 108 Introduction: scattering of sources difficulties

109 109 Introduction: scattering of sources difficulties

110 110 Methods for efficient information retrieval: conclusions The examples given show at least that progress in this field is impressive.

111 Questions? Suggestions? Remarks? 111

112 112 You are free to copy, distribute, display this work under the following conditions:»attribution: You must mention the author.»noncommercial: You may not use this work for commercial purposes.»no Derivative Works: You may not change, modify, alter, transform, or build upon this work. For any reuse or distribution, you must make clear to others the license terms of this work.