Citation Analysis: Comparison of Web of Science 1, Scopus TM, SciFinder 1, and Google Scholar

Transcription

1 Journal of Electronic Resources in Medical Libraries, 7: , 2010 Copyright # Taylor & Francis Group, LLC ISSN: print= online DOI: / Citation Analysis: Comparison of Web of Science 1, Scopus TM, SciFinder 1, and Google Scholar JIE LI, JUDY F. BURNHAM, TREY LEMLEY, and ROBERT M. BRITTON University of South Alabama Biomedical Library, Mobile, Alabama, USA In recent years, numerous articles have compared the coverage, features, and citation analysis capabilities of Scopus TM and Google Scholar with Web of Science 1, a Web-based version of Science Citation Index. This article goes a step further and compares the citation analysis potential of four databases: Web of Science, Scopus, SciFinder, and Google Scholar. Each database presents its own strengths and weaknesses, including methods of analysis, differences in coverage, and means of linking references. As an illustration, Web of Science provides coverage back to In contrast, Scopus only has completed citation information from 1996 onward, yet Scopus provides better coverage of clinical medicine and nursing than Web of Science. SciFinder has the strongest coverage of chemistry and the natural sciences, while Google Scholar has the capability to link citation information to individual references. Although Scopus and Web of Science provide comprehensive citation reports, all databases miss linking to some references included in other databases. KEYWORDS citation analysis, database comparison, Google Scholar, Science Citation Index Expanded, SciFinder 1, Scopus TM, Web of Science 1 Received May 4, 2010; revised May 13, 2010; accepted May 20, The authors would like to thank Donna Ladnier who assisted in citation comparison and graphics. Address correspondence to Jie Li, MLS, AHIP, University of South Alabama Biomedical Library, Mobile, AL 36688, USA. jli@jaguar1.usouthal.edu 196

2 Citation Analysis 197 INTRODUCTION For more than 40 years, ISI Citation Indexes, with Science Citation Index as the first citation index, were the only indexes that tracked citing references. ISI Web of Science 1 was launched in 1997 as the Web version of the print ISI Citation Indexes, which began publication in Ever since the introduction of Scopus TM and Google Scholar in 2004, there have been many articles discussing and comparing the features of these databases and their citation analysis capabilities compared with Web of Science. 1 4 SciFinder 1 was a client-based chemistry database launched in late 1997; its citation analysis features were added in Few articles discuss SciFinder s citation analysis features. The purpose of this article is to include all four of these citation databases and compare their features and their compositions. While Web of Science, Scopus, and Google Scholar also include social science and humanities, this article looks at the coverage of three areas: basic medical sciences, clinical medicine, and nursing. LITERATURE REVIEW In 2005, Jacso compared Web of Science, Scopus, and Google Scholar according to subject scope and composition, database size and dimensions, and other features and found that Google Scholar showed lack of competence and understanding of the basic issues of citation indexing. 1 Falagas et al. compared PubMed, Scopus, Web of Science, and Google Scholar and found Scopus s citation analysis was faster and included more articles than the citation analysis of Web of Science, but the citation analysis of Web of Science provided better graphics and was more detailed than the citation analysis of Scopus. 2 Kulkarni et al. compared Web of Science, Scopus, and Google Scholar for articles published in general medical journals and found Google Scholar and Scopus retrieved more citations per article. 3 Whitley compared Web of Science and SciFinder and found that SciFinder consistently had a higher percentage of unique citing publications than Web of Science for chemical literature. However, Whitley pointed out that researchers using only one database would miss some citing references. 4 DATABASE OVERVIEW Web of Science Web of Science consists of seven databases containing information gathered from thousands of scholarly journals, books, book series, reports,

3 198 J. Li et al. conferences, and more. Among them are five citation databases: Science Citation Index Expanded (SCI-Expanded), Social Sciences Citation Index (SSCI), Arts & Humanities Citation Index (A&HCI), Conference Proceedings Citation Index Science (CPCI-S), and Conference Proceedings Citation Index Social Sciences & Humanities (CPCI-SSH). Web of Science also includes two chemistry databases: Index Chemicus (IC) and Current Chemical Reactions (CCR-Expanded). In this article, Science Citation Index Expanded was used to evaluate the database. Science Citation Index Expanded fully indexes more than 6,650 major journals from 1900 to the present, covering more than 150 scientific disciplines. Web of Science has two types of searches: the bibliographic search and cited reference search. The easy-to-use basic, advanced, or author searches may be used to find bibliographic information. The qualifiers in either basic or advanced search help users to make their searches more specific. Numbers of articles in subject areas, document types, authors, source titles, publication years, institutions, funding agencies, languages, and countries are automatically shown on the search result page and any areas may be selected to refine the search or for results analysis (see Figure 1). A citation report may be created from the search results. The citation report provides two bar charts, showing published items in each year and citations in each year. The report also provides a summary of the report showing the search results found, sum of the times cited, average citations per item, and FIGURE 1 Web of Science results analysis.

4 Citation Analysis 199 the h-index number. The h-index measures the productivity and impact of a researcher and is based on the number of citations by a scientist compared with the number of citations they have received. The citation report also provides total citations each year for each individual article (see Figure 2). The Distinct Author Set feature in Web of Science is a discovery tool that shows sets of papers likely written by the same person. Web of Science analyzes citation data, such as subject areas and journals, to create the distinct author sets (see Figure 3). Citation maps may be created on the citing article results page to give a visual result of the citing references. Records that cite or are cited by the target record are first-generation citation maps. Records citing records that cite the target record and records cited by records cited by the targeted record are second-generation citation maps (see Figure 4). The Scientific Web Plus feature allows users to view the Web search results by simply clicking on the icon on the search results page. Cited Reference search is a unique feature in Web of Science that finds articles that have cited a previously published work. Cited Reference Search retrieves citing articles that were published before the subscription years. The research ID feature allows batch uploading of profile information. The Marked List feature has some drawbacks. Users may only save a Marked List to a file, export it to a reference management file, or print. A Marked List may not be analyzed and citation reports cannot be created. FIGURE 2 Web of Science citation report.

5 200 J. Li et al. FIGURE 3 Web of Science distinct author sets. FIGURE 4 Citation map. Web of Science s main features include the following:. 6,650 major science journals (Science Citation Index Expanded). Records from 1900 to the present

6 Citation Analysis 201. Coverage in 150 scientific disciplines in agriculture, astronomy, biomedical sciences, chemistry, physical sciences, materials sciences, and so forth. Journal articles and conference papers. Searchable by chemical structure and by compound=reaction details. Citation report includes published items and citations in each year showing in bar charts. Citation analysis by author, country, document type, institution, language, publication year, source title, subject area and funding information. h-index. Researcher ID batch upload of profile information. EndNote Web Free with Web of Science. Cited reference search, citation report in bar graph, and citation map (unique to Web of Science) Scopus Scopus has a very user friendly interface. Updated daily, Scopus covers 38 million records, including abstracts of more than 18,000 peer-reviewed titles from more than 5,000 publishers, including open-access titles, trade publications, and book series. About 80% of the records contain abstracts. Also searched are about 435 million scientific Web pages through Scirus s Web search, covering the scientific Web and 23 million patents from 5 patent offices worldwide. With its strongest coverage in health sciences and physical sciences, Scopus also includes life sciences, social sciences, engineering, and arts and humanities records, and has a robust international and conference paper coverage. The Author Identifier is one of the strengths of the Scopus database (see Figure 5). Entering an author s name gives an exact match of the author, by last name and initials, as well as how the name exactly appears in the journal articles. The Scopus Author Identifier uses an algorithm that matches author names based on their affiliation, address, subject area, source title, dates of publication citations, and co-authors. Results can be limited by source title, author name, publication year, document type, subject area, language, affiliation, and source type. Results can be sorted by document=article title, author name, date, source title, and cited by. A cited reference list can be obtained by searching a certain author. Then, if any of the articles have been cited, the user may click on the cited by number to get the articles that cited this author s particular work. A Citation Analysis in Scopus allows users to sort by and view the articles that cite the original article. In addition, the use of Citation Tracker allows the tabular display of citing articles by article and by year. Self-citations can be eliminated. These data can be exported into a spreadsheet. The h-index with graph shows the strength of the publication record.

7 202 J. Li et al. FIGURE 5 Scopus author identifier. Scopus also offers a journal analyzer that allows the user to compare journals according to number of citations, articles published, percentage not cited, and gives a trend line. A weakness of Scopus is that it only has references from 1996 to the present. While the author identifier is a plus, multiple detail displays for more than one author=affiliation retrieved cannot be viewed. Scopus s main features include the following:. Indexes 8,700 life and health sciences journals. Referenced records from 1996 to the present; records without references from 1869 to the present. Covers life sciences, health sciences, chemistry, engineering, physical sciences, and social sciences. Includes journal articles, science Web sites, patents, and conference papers. Analyzes citations by author, document type, publication year, source title, and subject area. Shows h-index in graphical format. Exports citation analysis to a spreadsheet, identifies author by tracking publication history with an Unique ID (these features are unique to Scopus)

8 Citation Analysis 203 SciFinder SciFinder contains two reference databases (CAplus and MEDLINE), a structure database (REGISTRY), a reaction database (CASREACT), a commercial source database (CHEMCATS), and a regulatory database (CHEMLIST). Except for MEDLINE, all of the other five databases are produced by Chemical Abstracts Service (CAS). CAplus, with an emphasis on chemistry and related sciences, contains more than 32 million documents and covers 10,000 scientific journals, patents, conference proceedings, technical reports, books, dissertations, reviews, meeting abstracts, Web preprints, and electronic-only journals. MEDLINE contains more than 18 million references and provides coverage for more than 4,800 biomedical journals from 1950 to the present, with 99% of the references being from journal literature. CAplus covers the period from 1907 to the present and is updated with 3,000 references daily, while MEDLINE is updated five times per week. SciFinder may be searched by text, chemical structures, or molecular formulas. There are three basic means of searching SciFinder from the main search page: Explore References, Explore Substances, and Explore Reactions. Clicking on Explore References opens a page with seven additional search options: research topic, author name, company name, document identifier, journal, patent, and tags. The document identifier allows searching by CA accession numbers, patent numbers, or PubMed ID numbers (PMID), among others. Searching by research topic is the broadest and most versatile search option, allowing searches by topic, year, document type, language, and author=company name. SciFinder is a very useful tool in citation analysis. Once an article is retrieved, clicking the Get Citing tab retrieves all articles that cite the retrieved article. In contrast, clicking the Get Cited feature on the toolbar retrieves references contained in the retrieved article. Furthermore, all articles once retrieved may be analyzed by author name, CAS registry number, CA section title, company-organization name, database, document type, index term, CA concept heading, journal name, language, publication year, or supplementary terms. In addition, articles may be sorted by accession number, author, publication year, or title. There is the option to remove duplicate references. SciFinder has many strong points, including a user-friendly interface, with numerous ways to analyze, refine, or sort search results, as mentioned earlier. Citation data can be located in four steps. There is strong coverage both of patents and of physical sciences, with searching methods particularly useful to physical scientists and engineers, such as searching by chemical structure, molecular formula, or substance identifiers. Tags allow the user to label references with descriptive user-defined terms. Data can be exported in several different formats. There is also a Keep Me Posted feature. However, SciFinder also has several weaknesses. One is a noticeably slow search

9 204 J. Li et al. engine. Also, there is no journal analyzer function. Trying to ascertain the total number of citations per journal would be a time-consuming procedure, since citing may only be performed for 500 references at a time. Furthermore, it is not possible to ascertain impact factors, Eigen factors, h-index, or to make any cross-journal citation comparisons. In addition, SciFinder lacks anything resembling the Author Identifier feature of Scopus, although SciFinder does allow the searcher to search for alternative spellings of the author s last name. To illustrate, entering the name Turrens in the Author Name search yielded the results found in Figure 6. To search for the author in question, Julio F. Turrens, the search results quickly allowed the user to select articles by Turrens J, Turrens J F, Turrens Julio, and Turrens Julio F. Retrieving articles by these four entities retrieved 145 results [3 þ 69 þ 4 þ 69], but clicking the Remove Duplicates button reduced the number of hits to 85. SciFinder s main features include the following:. 10,000 scientific and 4,800 biomedical journals. CAPlus records from 1907 to the present; MEDLINE records from 1950 to the present. Covers biomedical journals, chemistry and related science materials, patents, conference proceedings, technical reports, books, dissertations, reviews, meeting abstracts, and Web preprints. Provides information on more than 52 million organic and inorganic substances, 61 million chemical sequences, chemical synthesis information, regulated chemicals, and commercially available substances. Allows searching by chemical structure, molecular formula, and substance identifiers. Citation analysis by research topic, author, company name, document identifier, journal, or patent. Citation analysis may only be performed for 500 references at a time. Not possible to ascertain impact factors, Eigen factors, or h-indexes. User-friendly interface FIGURE 6 SciFinder author search.

10 Citation Analysis 205 Google Scholar Google Scholar went live in November It is multidisciplinary, can be searched across many document formats (including peer-reviewed papers, theses, books, abstracts, articles, conference proceedings, patents, and legal documents), and searched for documents from many types of sources (including academic publishers, professional societies, preprint repositories, universities, and other scholarly organizations). Google Scholar displays other documents that have cited articles appearing in the results list, and suggests related articles. It also displays links to the article via an institution s resources if on campus, and displays locations of physical copies at other libraries. Subject areas covered by Google Scholar can be narrowed to the following: biology, life sciences, and environmental science; medicine, pharmacology, and veterinary science; business administration, finance, and economics; physics, astronomy, and planetary science; chemistry and materials science; social sciences, arts, and humanities; and engineering, computer science, and mathematics. One, some, or all subject areas may be selected. Google Scholar s basic search interface includes only a single search box. From here, one may either search for articles or legal opinions and journals. If searching for articles, there is an additional option of clicking an include patents check box. There are also other preference settings, including interface language, search language, and individual library search. In the advanced search interface, phrases can be searched with all of the words, with the exact phrase, with at least one of the words, and without the words. Terms and phrases can also be limited to the title only. Other search options include author name, publication (including standard publication abbreviations), and date. One drawback to date searching, however, is the inability to select the month level, as only year ranges can be searched. As with the basic search, other preference settings include interface language, search language, and individual library search. The advanced search page also allows for distinct=unique author searches. There is a single box for author searching, wherein one may enter last name only, last name plus initials, or full name. Among the strengths of the Google Scholar database is that its basic search interface is user-friendly and familiar to Google users. Google Scholar also provides direct links to full text in library holdings, casts a very broad net in number of sources searched, includes phrase searching, and contains a unique feature, related articles search. Chief among its weaknesses is Google Scholar s total lack of citation analysis tools, which are widely available in other citation databases. Also, there is no author information provided, and searchers cannot save searches or export citations.

11 206 J. Li et al. Google Scholar s main features include the following:. Multidisciplinary. Searches many document formats. Searches many different types of sources. Retrieves high number of hits. User-friendly and familiar user interface. Directly links to full text article holdings of an institution. Unknown number of journals indexed or coverage dates (proprietary) DATABASE COMPARISON Coverage Comparison This project compared several characteristics of the databases. The database coverage was reviewed along with strengths and weaknesses. As ISI Web of Science is the oldest citation database; it has the longest coverage with both bibliographic data and citation data going back to Before 2004, there was almost no competition among citation databases. Although SciFinder started in 1997, it has been kept in the chemistry world, with little usage by other disciplines. In addition, before 2001, SciFinder was not a Web-based database, and was only available through a client module, so its use was limited. With complete CAPlus included in the database, SciFinder is strongest in chemistry. Besides its citation features which are included in CAplus and MEDLINE, SciFinder also contains a structure database REGISTRY, a reaction database CASREACT, a commercial source database CHEMCATS, and a regulatory database CHEMLIST that are unique to chemistry and related sciences. As mentioned, both Scopus and Google Scholar started in Like Web of Science, Scopus is a commercial database. Elsevier combined its bibliographic databases EMBASE, Compendex, World Textile Index, FLUIIDEX, GeoBase, and BIOBASE, and added MEDLINE to make a huge database and added the citation feature. Scopus s bibliographic data cover journals as early as However, its citation data are limited to 1996 to the present. Google Scholar is free and indicates on its site that it includes articles, theses, books, abstracts, and court opinions from academic publishers, professional societies, online repositories, universities, and other Web sites. Information about Google Scholar coverage is not available. While Google Scholar seems to retrieve more references, this study found duplicate records. For instance, Google Scholar may retrieve the same articles from journals, as well as from university or other Web sites when these articles are listed in Web-based resumes or faculty publications. Google Scholar does not analyze the citations like the other three databases do. See Table 1 for a chart that compares database coverage and features.

12 TABLE 1 Database Coverage and Feature Comparison Database Web of Science Scopus SciFinder Google Scholar Date of inception = = =2004 No. of journals included 9,760 (6,650 SCIE; 1,950 18,242 14,800 No data provided. SSCI; 1,160 A&HCI) Other sources Conference proceedings Books series, conference No data provided. No data provided. proceedings Coverage period 1900-present Bibliographic data, Databases included SCI, SSCI, A&HCI, CPCI-S, CPCI-SSH, IC, CCR Subject areas Life Sciences, Health Sciences, Physical Sciences, Social Sciences, Arts & Humanities Subject strength Strong coverage in medicine, life sciences, physical sciences present Citation data, 1996-present MEDLINE, EMBASE, Compendex, World Textile Index, FLUIIDEX, GeoBase, BIOBASE Life Sciences, Health Sciences, Physical Sciences, Social Sciences, Arts & Humanities Strong coverage in health sciences and physical sciences. Bibliographic data,. CAplus, 1907-present. MEDLINE, 1949-present. Citation data, 1996-present No data provided. CAplus, MEDLINE No data provided. Life Sciences, Health Sciences, and Physical Sciences Extensive coverage of chemistry, more than 49 million substances, and over 61 million sequences. Life Sciences, Health Sciences, Physical Sciences, Social Sciences, Arts & Humanities No data provided. (Continued ) 207

13 TABLE 1 Continued Database Web of Science Scopus SciFinder Google Scholar Types of publications Peer-reviewed journals, conference proceedings, patents, and Web pages (Scientific WebPlus). Database strengths User-friendly interface, distinct author set, results analysis, and citation report. Database weaknesses Marked list does not have the features as the search results have. Peer-reviewed journals, open access journals, trade publications, book series, conference proceedings, patents, and Web pages (Scirus). User-friendly interface. Ability to group authors and affiliations together for more complete search. Only includes citations back to Cannot view details of more than one author= affiliation retrieved. Journals, book chapters, patents, conference proceedings, technical reports, meeting abstracts, and dissertations. User-friendly interface, easy to refine search. Allows searching by chemical structure, company, molecular formula, and substance identifiers. Slow search engine. Only includes citations back to Update frequency Weekly Daily CAplus daily; MEDLINE 5 times per week. Linking to full text Yes Yes Yes Yes Journals, theses, books, conference proceedings, and patents, and from professional societies, online repositories, universities and other Web sites. Basic search interface is simple. Casts a very broad net in number of sources searched. No citation analysis, no author information, cannot save searches or export citations. No update information provided. The print SSCI started in

14 Citation Analysis 209 Database Feature Comparison This study also analyzed database features (see Table 2). All of the databases compared in this article are user friendly. Both Web of Science and Scopus have fill-in search boxes with drop-down menus that make it easy for users to select search fields and to limit the years searched. Both databases allow more limits to be applied after the initial search. SciFinder has three search tabs: Find References, Find Substances, and Find Reactions. Find Substances and Find Reactions are used to find chemical substances and reactions and are not available in the other databases compared. In this article, the Find References features were explored. SciFinder s access is not only restricted by Internet Protocol (IP) but also by username and password. Google Scholar is very much like Google in that it is simple and easy-to-use and casts a wide net. However, searches with Google Scholar can retrieve large amounts of irrelevant results. Only Scopus and Web of Science provide a unique author search by identifying the subject, author affiliation, and journals to help identify variations on the same author. Scopus can even track the author history when the author has moved from institution to institution. Web of Science s cited reference search is a unique feature that cannot be found in any other databases. Web of Science citation maps give a visual result of the citing references. First-generation and second-generation citation maps may be displayed. The research ID feature allows batch upload of profile information. Scopus s h-index in graphic format provides a visual picture. As mentioned, SciFinder is the only database that users may search for substances and reactions, but for purposes of comparison this article does not discuss these features. As for Google Scholar, it uses the same simple and easy interface that is familiar to many Web searchers. Citation Analysis Comparison Finally, this study compared the citation analysis capabilities of each resource (see Table 3). All databases may link the search results to citing references. Web of Science can analyze the citations by author, country, funding agency, grant number, document type, institution name, language, publication year, and source title. Its citation report provides two bar charts, showing published items in each year and citations in each year. The report also provides a summary of the report showing the search results found, sum of the times cited, average citations per item, and the h-index number. It also provides total citations each year for each individual article (see Figure 2). Scopus Citation Overview may be displayed by year and citation with the option of excluding self citations with h-index, which can be displayed as a graph. The citing references may be analyzed by source title, author, year, and result type. SciFinder analyzes the citing references by author, CAS registry number, CA section title, organization, database, document type, index term, CA

15 TABLE 2 Database Search Feature Comparison Database Web of Science Scopus SciFinder Google Scholar Search types and features Basic search, advanced search, and cited reference search. Search by topic, title, author, editor, group author, publication name and year, conference, funding agency, grant number, organization, address, and funding agency. Unique author search Web of Science Distinct Author Set feature shows sets of papers likely written by the same person based on the subject areas and publications. Unique features Cited reference search, citation report with bar charts, citation map, and linking to researcher ID. Basic search, author search, affiliation search, advanced search. Search by author, title, keyword, affiliation, language, ISSN, CODEN, DOI, references, conference, chemical name, and CAS number. Scopus Author Identifier uses an algorithm that matches author names based on their affiliation, address, subject area, source title, publication dates, and co-authors. Journal analyzer allows user to compare journals in same subject area. Analysis includes total citations, articles published, trend line, and % not cited. Reference search, substance search, and reaction search. Search by research topic, author name, company name, document identifier, journal, patent, tags, publication year, document type, and language. Basic search and advanced search. Search by text, title, author, publication, date, and patent. Additional limits to subject areas, legal opinions, and specific state. Not provided. Not provided. Substances and reaction searches. Use same simple and easy Google search interface. 210

16 TABLE 3 Citation Feature Comparison Database Web of Science Scopus SciFinder Google Scholar Linking to citing references How results analyzed Citations may be analyzed by author, country or territory, funding agency, grant number, document type, institution name, language, publication year, or source title. Citation report options Yes Yes Yes Yes Citation report shows published items in each year and citations in each year, sum of the times cited, average citations per item, h-index, and number of citations for each article each year. Citation Tracker, h-index, Journal Analyzer. Articles retrieved can be sorted by # of citations. Provides citation tracker to track number of citations over time. Provides h-index graph and number. Citations may be analyzed by author, CAS registry #, CA section title, company, database, document type, subject heading, journal name, language, publication year, and supplementary terms. No citation report available. No citation analysis available. No citation report available. 211

17 212 J. Li et al. concept heading, language, publication year, and subject. SciFinder does not have an overview citation report. Google Scholar does not provide citation analysis or citation reports. METHODS Three journals were selected for this study: New England Journal of Medicine in general medicine, Cell in basic medical sciences, and Nursing Research in nursing. These journals have the highest impact factor (excluding review journals) in their own subject areas and are covered by all four databases. For comparison, articles were selected using the following criteria:. all 51 articles labeled Original Articles in New England Journal of Medicine Jan Mar 1999, volume 340, issues all 57 articles labeled Articles (original research articles) in Cell Jan Mar 1999, volume 96, issues 1 6. all 16 articles labeled Articles, Methodology, or Brief Reports in Nursing Research Jan April 1999, volume 48, issues 1 2 The searches were done in the four databases within a 10-day time frame, from December 14 23, The total number of citing references from each journal retrieved from Web of Science, Scopus, SciFinder and Google Scholar was compared. In addition, five articles were randomly chosen from each journal to compare publication type and language of the citing articles. Systematic sampling was used to select five citing articles from each journal. The method used to select the articles was that all citing articles were sorted by ascending date, page number, and every nth article (where n ¼ total number of articles divided by five). The publication type of each article was identified (original article, review, editorial, conference proceeding, and other document types, which included news items, short surveys, notes, and letters). For languages, the articles were divided by English and non-english. This comparison was done only in Web of Science, Scopus, and SciFinder because of the difficulty of doing so in Google Scholar. Since there was a big difference in the citing references in New England Journal of Medicine, five articles were selected that had the highest percentage of difference in Scopus and Web of Science. RESULTS Table 4 shows the total number of citing references retrieved from all four databases for articles published in New England Journal of Medicine, Cell, and Nursing Research for the selected period. In the journal Cell, there was not much difference. Google Scholar had a higher number for citing

18 TABLE 4 Citing References for Articles in Three Journals Citation Analysis 213 Total number of citing references Journal Title, Volume, Issue, Date Articles cited Web of Science Scopus SciFinder Google Scholar NEJM 340(1 12), Jan Mar ,455 20,973 6,645 25,445 Cell 96(1 6), Jan Mar ,681 17,844 17,367 19,563 Nursing Research 48(1 2), Jan Apr references. For New England Journal of Medicine and Nursing Research, Google Scholar also had the highest number. However, a closer examination of the citing references from Google Scholar revealed numerous duplicate references. Another interesting observation was not the difference in the number of citing references from Cell in all databases was as large as the other two journals. Combined from the SciFinder coverage, it was determined that SciFinder had fewer clinical medicine and nursing journals than Web of Science and Scopus. Google Scholar did not specify its coverage of subjects or journals. Since there is no method to decide the publication type in Google Scholar, the study compared the publication types only in Web of Science, Scopus, and SciFinder. In the five articles randomly selected from each journal, the number and percentage of document types in the citing references are shown in Table 5. As was noticed in Cell, the total citation numbers from the four databases did not show a great deal of difference. However, in New England Journal of Medicine, some articles had a big difference in citing references. As there is no way to download citations from Google Scholar, and SciFinder s stated coverage is mainly in chemistry and basic medical science, it was decided to compare this in Web of Science and Scopus. The difference in one article was as high as 18% between Web of Science and Scopus. Five articles were selected for further analysis: Terán-Santos, Jiménez-Gómez, and Cordero-Guevara 1999; 5 Lau et al. 1999; 6 Darouiche et al. 1999; 7 Jorenby et al. 1999; 8 and Vroomen et al These articles had the highest percentage differences in citing references in Web of Science and Scopus. The difference of these five articles ranged from 13.36% to 18.57%. The results of the differences are shown in Table 6 and Figure 7. DISCUSSION While Web of Science and Scopus index many of the same publications, they give quite different results for citing reference searches. In this study, while searching the basic medical sciences, it was found that there was not much difference in retrieving citing references from Web of Science, Scopus, and

19 TABLE 5 Publication Types of Citing References Journal with cited articles Database Article Review Editorial Conference proceedings Other document type Total citing references Randomly selected 5 articles from Cell Randomly selected 5 articles from New England Journal of Medicine Randomly selected 5 articles from Nursing Research Web of Science Citing references Scopus Citing references SciFinder Citing references Web of Science ,144 % 70.37% 24.13% 1.84% 3.23% 0.44% 100% ,139 % 70.24% 25.55% 0.00% 2.37% 1.84% 100% ,154 % 61.61% 33.88% 0.00% 4.07% 0.43% 100% Citing references % 61.10% 18.39% 6.98% 9.51% 4.02% 100% Scopus Citing references % 59.04% 30.88% 2.08% 4.96% 3.04% 100% SciFinder Citing references Web of Science % 65.96% 28.72% 0.00% 2.13% 3.19% 100% Citing references Scopus Citing references SciFinder Citing references % 70.00% 17.14% 4.29% 8.57% 0.00% 100% % 63.79% 28.45% 0.86% 5.17% 1.72% 100% % 75.00% 25.00% 0.00% 0.00% 0.00% 100% 214

20 Citation Analysis 215 TABLE 6 Citing References in Teran-Santos, Lau, Darouiche, Jorenby, and Vroomen Articles Teran-Santos Lau Darouiche Jorenby Vroomen Citing in both Web of % % % % % Science and Scopus Citing in Scopus only % % % % % Citing in Web of Science only % % % 57 5% 8 8.5% Total % % 1, % 1, % % SciFinder. However, when searching in clinical medicine, Scopus included more articles than Web of Science. This study found that SciFinder was not an adequate tool for analyzing clinical medicine citing references. The study also found that Web of Science, Scopus, and SciFinder all missed references from publications that they cover. Google Scholar did not disclose which publications were indexed and the results were not downloadable, so it was difficult to determine which citations were missing. Also Google Scholar included citations from Web sites and therefore duplicated some citing references. For article types, while the majority of the citing references cited in both Web of Science and Scopus had exact matches, some of them matched everything except publication types. A reference called original article in one database may be called a review article in another. At the same time, a conference proceeding in one database, may be called an article in another. The problem of cited articles not linked to a target source may be caused by author error, mapping error, internal page, or journal translation. 10 In conclusion, Scopus covered more journals and was capable of doing citation analysis. However, only Web of Science provided access to cited FIGURE 7 Citing references in five New England Journal of Medicine articles.

21 216 J. Li et al. references back to When conducting a citation search for a basic science topic or journal, a search of Web of Science, Scopus, or SciFinder would be sufficient. However, a thorough citation search on a nursing or clinical topic or journal would require the use of both Web of Science and Scopus. ABOUT THE AUTHORS Jie Li, MLS, AHIP (jli@jaguar1.usouthal.edu) is Assistant Director for Collection Management; Judy F. Burnham, MLS, AHIP (jburnham@jaguar1. usouthal.edu) is Library Director; Trey Lemley, MLIS, JD, AHIP (lemley@ jaguar1.usouthal.edu) is Information Services Librarian=Evening Supervisor; and Robert M. Britton, MLS (rbritton@bbl.usouthal.edu) is Electronic Resources=Collection Development Librarian, all at the University of South Alabama Biomedical Library, Mobile, AL REFERENCES 1. Jacso, P. As We May Search Comparison of Major Features of the Web of Science, Scopus, and Google Scholar Citation-based and Citation-enhanced Databases. Current Science 89, no. 11 (2005): Falagas, Matthew E.; Pitsouni, Eleni I.; Malietzis, George A.; and Pappas, Georgios. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and Weaknesses. FASEB Journal 22, no. 2 (February 2008): Kulkarni, Abhaya V.; Aziz, Brittany; Shams, Iffat; and Busse, Jason W. Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals. JAMA: The Journal of the American Medical Association 302, no. 10 (September 9, 2009): Whitley, Katherine M. Analysis of SciFinder Scholar and Web of Science Citation Searches. Journal of the American Society for Information Science & Technology 53, no. 14 (December 2002): Terán-Santos, J.; Jiménez-Gómez, A.; and Cordero-Guevara, J. The Association Between Sleep Apnea and the Risk of Traffic Accidents. Cooperative Group Burgos-Santander. New England Journal of Medicine 340, no. 11 (March 18, 1999): Lau, J.Y.; Sung, J.J.; Lam, Y.H.; et al. Endoscopic Retreatment Compared with Surgery in Patients with Recurrent Bleeding After Initial Endoscopic Control of Bleeding Ulcers. New England Journal of Medicine 340, no. 10 (March 11, 1999): Darouiche, R.O.; Raad, I.I.; Heard, S.O.; et al. A Comparison of Two Antimicrobial-Impregnated Central Venous Catheters. Catheter Study Group. New England Journal of Medicine 340, no. 1 (January 7, 1999): Jorenby, D.E.; Leischow, S.J.; Nides, M.A.; et al. A Controlled Trial of Sustained-release Bupropion, a Nicotine Patch, or Both for Smoking Cessation. New England Journal of Medicine 340, no. 9 (March 4, 1999):

22 Citation Analysis Vroomen, P.C.; de Krom, M.C.; Wilmink, J.T.; Kester, A.D.; and Knottnerus, J.A. Lack of Effectiveness of Bed Rest for Sciatica. New England Journal of Medicine 340, no. 6 (February 11, 1999): Buchanan, Robert A. Accuracy of Cited References The Role of Citation Databases. College & Research Libraries 67, no. 4 (July 2006):