Possible Relationships between Summit and Search Engines: A Report from the Internet Search Engines Working Group

Transcription

1 Internet Search Engines Working Group Report 1 Possible Relationships between Summit and Search Engines: A Report from the Internet Search Engines Working Group Michael Spalti, Willamette University, Chair Michael Boock, Oregon State University Sara Brownmiller, University of Oregon Bryan Miyagishima, Western Oregon University Heidi Senior, University of Portland Rob Truman, Lewis & Clark-Northwestern School of Law Marilyn Von Seggern, Washington State University Laura Zeigen, Oregon Health & Science University

2 Internet Search Engines Working Group Report 2 Executive Summary The Internet Search Engines Working Group investigated ways to integrate Summit with Internet search engines. Our research focused on three general approaches to integration. These were to: Rely on OCLC Open Worldcat Incorporate Summit holdings into search engines, with particular emphasis on Google Scholar Educate libraries about local steps that they might take to facilitate some degree of integration with search engines. We offer four recommendations based on what we have learned. The Alliance might: Pursue with OCLC the concept of adding a Summit link to the Open Worldcat interface. Continue a promising conversation with Google about creating a Summit link from within Google Scholar. Investigate development of an OAI repository of harvestable Summit data for potential use by Internet search engines or by local library projects. Provide recommendations and education to local libraries regarding linking to and optimizing links from Open Worldcat, OpenURL resolvers, and new search engine services. Our Working Group s conversation with Google Scholar blossomed late. Once channels were opened, Google was quite responsive to our questions and concerns. Our working group recommends continued discussion with Google and with OCLC. Both organizations seem responsive to suggestions on how they might work with us to integrate Summit holdings. I. Linking to Summit from Open WorldCat The Internet Search Engines Working Group (ISEWG) has determined that Open WorldCat is the best existing option for making library holdings visible in Yahoo! Search, Google, and other internet search engines and bookselling sites 1. If a web user searches for a book in a participating search engine that matches information contained in Open WorldCat, the retrieved search results will include a Library Search link to Open WorldCat (Figure 1, circled links). 1 A list of search engines and other partner sites is available at

3 Internet Search Engines Working Group Report 3 Approximately 3.4 million records, representing books owned by more than 100 libraries, were exposed to Internet search engines during the Open Worldcat pilot project. In a 2004 newsletter 2, OCLC announced plans to make all OCLC records available. We are told that this work has been completed. To date, however, Google and Yahoo! have not indexed the entire OCLC database, and there is no indication of when, or if, this will happen. Google and Yahoo! continue to index only 3 million of the most widely held titles and a set of 450,000 records that represent unique items in the OCLC database. Google Library Search and Open Worldcat The Library Search feature is Google Scholar s way of connecting to the OCLC Open Worldcat service. Figure 1 Search for king lear in Google Scholar Selecting Library Search pulls up an entry for the book title in Open WorldCat. Users then enter a zip code or state and are presented with OCLC member libraries in their region that hold the item. Individual libraries with an OCLC holdings symbol and holdings in WorldCat automatically have links to their holdings appear in Open WorldCat (see figure 2, in shaded rectangle). On-campus users will also see a My 2

4 Internet Search Engines Working Group Report 4 Library section in the upper right of the screen, with a link to their local library (See figure 2, circled), if their library subscribes to WorldCat and the FirstSearch administrative module has been configured properly. Figure 2: Open WorldCat page Configuration of FirstSearch administrative module & Summit In the administrative module of FirstSearch, a library can use the Linking --> Web Library Catalogs function to customize the search links and corresponding text that appear throughout their FirstSearch databases and in Open WorldCat. A library can therefore customize the text of the link in the "My Library" box and specify the catalog that is used for the link (See Figure 3). This link would then appear in the My Library box in Open WorldCat. Although OCLC will use the library's own OCLC symbol as the key, the library can actually configure any library catalog for the link. Thus, a Summit member library could link their patrons to the Summit Union Catalog rather than their own local catalog. For example, the University of Oregon Libraries is registered with Open WorldCat to be identified in a Library Search request. Currently, when a user finds a book in Google, selects the Library Search link, and enters the zip code for the University of Oregon, they

5 Internet Search Engines Working Group Report 5 are offered links to the UO Libraries and a link to locate the book at the library. This second link has been configured to perform a standard number search using the ISBN. Figure 3: Administrative module in FirstSearch A Summit library can configure the ISBN search link for Summit, rather than its own local catalog (again, see Figure 2). For the UO, the local link would be In addition, the library can customize the text of the link, for example, "Find in Summit" rather than "Search the catalog at UO Libraries". This strategy would assume that Summit patrons are familiar with Summit and its requesting functionality. The decision to customize the link text and the link itself must be made very carefully, for it also determines the text and link behavior in a library's FirstSearch databases. If a library uses the FirstSearch interface for Medline, for example, a patron is offered the ability to "Search the catalog at the UO Libraries". This is the same link that would appear in Google Scholar or Yahoo. A library would need to decide if it wants students and faculty using a FirstSearch database to be offered a link for Summit rather than the local catalog.

6 Internet Search Engines Working Group Report 6 Other options for linking to Summit At present a library cannot choose to link to both their local catalog and a consortial catalog such as Summit in the My Library section. The only way to have Summit holdings and local holdings available at the same time in Open WorldCat would be for Summit holdings to be collected under a single holdings symbol using OCLC s group catalog. This would be costly and require a great deal more discussion. In response to an query, Bob Schulz, OCLC WorldCat product manager, expressed an interest in investigating implementation of a separate link for an entire consortium associated with the individual institution so that a patron from a particular Summit library IP range would see both a link to their library and a link to the Summit catalog. II. Google Scholar and Summit Our working group contacted Google Scholar several times to explain the Summit union catalog and Summit borrowing. We were initially unsure about Google s interest in working with a multi-state library consortium. After weeks of exchange, Michael Spalti had a phone conversation with Anurag Acharya, the lead engineer for Google Scholar, and Christian DiCarlo, our primary contact on the Google Scholar Team. (The conversation took place on March 10, 2006.) Both Anurag and Christian expressed clear interest in working with the Alliance to integrate Summit holdings into Google Scholar. They also want to discuss ways to enhance Alliance library involvement in the Google Library Links program. Based on Google s desire to work with the Alliance, we recommend pursuing further discussions with the Google Scholar Team. Anurag and Christian have made themselves available for this conversation. Possible benefits include: Obtaining use statistics from Google that provide picture of how Google Scholar is being used. Discussing ways to improve Google Scholar with people directly involved in its development. Exploring how individual library or consortium link resolvers can better use the Library Links program. Integrating Summit and Google Scholar via links to Summit from Google Scholar search results. Linking to Summit Anurag offered several scenarios for achieving the last objective. The two basic questions that our working group presented to him were: How to incorporate a Summit link into the search results presented to consortium library patrons, and

7 Internet Search Engines Working Group Report 7 How Summit data might be harvested by Google to provide this service. Anurag sees several ways to provide Summit links in Google Scholar search results. The choice of techniques depends on our situation. He feels, however, that the best strategy would probably build on the Google Scholar Library Links service. Figure 3. Google Scholar with Library Links for Harvard SFX Service Figure 3 shows Library Links for full-text journals owned by Harvard University. These links connect to the Harvard SFX server. To enable these links, SFX and other commercial or homegrown link resolvers share information from their knowledgebase with Google Scholar. Google activates these links after receiving this information from the library link resolver. Anurag s idea is to create links to Find it in Summit (Google Scholar lets you choose the wording). According to Anurag, the Summit link could be offered to Alliance library users based on IP range. Individual libraries could configure Google Scholar to point to a local OpenURL resolver service (e.g.: SFX) for journal citations and to Summit for books. Journal requests would thus be routed through local resolvers and interlibrary loan services. Links for books would be routed to Summit. Open Worldcat searches would also be available through the Google Scholar Library Search feature as is illustrated in Figure 1.

8 Internet Search Engines Working Group Report 8 On the question of how Summit holdings would be harvested, Anurag recommended MARCXML over simpler formats like Dublin Core or the XML format that Google Scholar recommends for harvesting resolver knowledgebase information. The more structured and comprehensive MARCXML data is used by Google for de-duping and other purposes. Christian and Anurag foresee two ways to generate the URLs that would link from a Google Scholar record to its corresponding record in Summit. Anurag also expresses interest in a supplemental data field that shows the number of libraries that own each item. The MARCXML harvesting technique recommended by Anurag is relatively simple and direct from an Alliance perspective. It only requires exporting MARC records from INN- Reach and converting these records to MARCXML. A number of utilities are available to do this, and the entire process may be automated. Because the MARCXML file could reside on a different server, harvesting should have no direct impact on the INN-Reach system itself. A more detailed conversation with Google Scholar will be needed to determine the best approach to integration with Summit. Google appears willing to explore multiple options. With the lead person behind Google Scholar development ready to work with us, there is reason to think that this will be a worthwhile endeavor, and that our partnership with Google would open doors for other library consortia. III. Making Summit Records Available: Open Archives Initiative Harvesting The ISEWG also recommends investigation of the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). Our research shows that Google, Yahoo, and other search engines have begun to harvest metadata from OAI repositories 3. Google indicates that OAI Dublin Core is insufficiently rich for indexing bibliographic metadata, but OAI still retains some advantages over other harvesting strategies. OAI appears to be a possible long-term strategy for making Summit bibliographic metadata harvestable by other services. Additional reasons for recommending OAI are the following: OAI will permit sharing Summit data without exposing the INN-Reach system to search engine robots. OAI supports incremental harvesting. After an initial harvest, only metadata for changed or deleted records will be harvested. Both open source and commercial OAI-compliant repositories are available. Some of these applications are already used at consortium libraries. OAI supports multiple metadata standards. Dublin Core is used for all OAI transactions and is appropriate for use by Internet search engines. The Dublin 3 A repository is a network accessible server that can process OAI-PMH requests. See for more information.

9 Internet Search Engines Working Group Report 9 Core exported via OAI can also be enriched with MARCXML, MODS, or other metadata standards. OAI might therefore address future needs in addition to those considered by our working group. ISEWG asked Innovative Interfaces if they or other libraries have identified a need for OAI harvesting of INN-Reach systems by search engines. Not surprisingly, the answer was no. Because search engines like Google and Yahoo continuously explore new ways to mine the 'deep web' for content, it is difficult to anticipate future opportunities for integration with Summit. In this rapidly changing context, exploring standard protocol for sharing Summit data with other services appears to be worthwhile. Developing an OAI repository may help Alliance libraries to recognize and seize future opportunities. This is a sample OAI record containing the bibliographic information for an item and the ability to link to a library catalog. The record format is simple Dublin Core: <oai_dc:dc xmlns:oai_dc=" xmlns:dc=" xmlns:xsi=" xsi:schemalocation=" <dc:title xml:lang="en">white teeth : a novel</dc:title> <dc:creator>smith, Zadie</dc:creator> <dc:subject>lcsh: Immigrants -- Great Britain -- Fiction</dc:subject> <dc:publisher> New York : Random Houses</dc:publisher> <dc:date>created: 2000</dc:date> <dc:type>text</dc:type> <dc:identifier>issn: </dc:identifier> <dc:language>english</dc:language> <dc:rights xml:lang="en">copyright 2000</dc:rights> </oai_dc:dc> By harvesting Dublin Core records like the one above, search engines could index and link to Summit records. Access to this data could be limited to external partners and local Alliance library initiatives.

10 Internet Search Engines Working Group Report 10 IV. Increasing Alliance Members Awareness of Search Engine Functions Search Engine Toolbars Yahoo! Toolbar with library searching via WorldCat Internet Explorer Adds "Libraries" to search option, running a Find in a Library search in Yahoo! Also adds links to Netlibrary, Find OCLC libraries page, and OCLC OCLC provides these images and html for promoting the Yahoo! Toolbar with WorldCat on library web sites. Google Toolbar's ISBN!Autolink Feature Internet Explorer, Firefox AutoLink feature detects ISBNs on any web page, changes the toolbar button to "Show Book Info," and auto-converts the ISBNs into links to the Open WorldCat info on that title. In order for ISBNs to link to Open WorldCat, WorldCat must be selected as the default provider (above right) in the Toolbar's!Autolink Settings. AutoLink Settings can be selected either from the Google logo menu (Options > Browsing tab > AutoLink Settings button) or the "Change Default Provider" link on the AutoLink button menu. Albion Booklink Internet Explorer Plug-in for IE. Allows user to highlight an ISBN or some links to books, rightclick & choose "Booklink," and be taken to Open WorldCat record for that ISBN Firefox search extensions (via OCLC) Firefox Adds to Firefox's built-in search box links to WorldCat via Google and WorldCat via Yahoo

11 Internet Search Engines Working Group Report 11 Open URL Resolvers As a way of linking patrons to all types of library holdings, ISEWG encourages Alliance libraries with Open URL resolvers to link their resolvers from Google Scholar. Google describes the process and provides help on their Support for Libraries 4 page. As noted above, Google is interested in working with us to encourage greater participation. Libraries with a commercial link resolver and not linking from Google Scholar: Clark College Mount Hood CC Reed College Eastern Washington Univ. Oregon Inst. of Technology Seattle Pacific University Lane Community College Pacific University Seattle University Linfield College Portland State University Southern Oregon University V. Other Concerns Problems with link from Google Scholar to Open WorldCat Alliance members should be aware that patrons may encounter difficulties when navigating the link between Google Scholar and Open WorldCat. For example, a search in Google Scholar for cold war civil rights leads to a record for a book by the same title. The Library Search link from that record in Google Scholar goes to an Open WorldCat record for a paperback edition seemingly not owned by any Oregon libraries. However, a record for this edition is in Summit. Google Scholar apparently does not link to the hardcover edition that is owned by 18 Alliance libraries, although Open WorldCat contains a record for this edition. ISEWG recommends informing Google and OCLC of these difficulties and encouraging rectification. 4