The Place of Anthropology Plus in the Age of Google Scholar
Anthropology and American Indian Studies Selector
Google Scholar, which debuted in beta in 2004, has been the subject of hundreds of library science articles since it first appeared. Since its inception librarians have been struggling with what role this search engine, self-described on its website as “a simple way to broadly search for scholarly literature,” should play in the research process. While early reviews often questioned the scholarly value of Google Scholar, particularly in terms of weak content coverage (Jacso, 2005), recent articles have reported more positive results in measuring content coverage and search results, including Jacso’s own reevaluation three years later (Jacso, 2008). Two substantive studies from 2009 in particular, Google Scholar Search Performance: Comparative Recall and Precision (Walters, 2009), and How Scholarly Is Google Scholar? A Comparison to Library Databases (Howland, Wright, Boughan, & Roberts, 2009) have both ranked Google Scholar ahead of several subject-specific databases in the overall quality of search results.
While these recent studies have focused on a variety of databases including sociology and general social science resources, none have deeply examined anthropology databases. The scope of this study is to look specifically at one resource, Anthropology Plus, the core database for anthropology which combines two major indexes, Harvard University’s Anthropological Literature database and the United Kingdom’s Anthropological Index. The primary impetus for this work comes from the budget reductions currently affecting many libraries, including Cornell. In a time of reduced spending, all resources are being reconsidered and indexing and abstracting databases are being scrutinized to determine the benefits of continuing to subscribe to these in a federated search world.
Unlike some previous studies, such as the ones cited above, the goal of this study is not to determine which resource is “better” for particular topics, but rather to determine what benefits, if any, Anthropology Plus brings to the table both in terms of content coverage and delivery of relevant search results. The purpose of this review is to examine Anthropology Plus as a tool not just for general users but also for expert searchers. As such, the examples are relatively complex and make use of the various search features of both Anthropology Plus and Google Scholar.
For this study two questions are being considered; overall coverage of Anthropology Plus resources in Google Scholar, and the relevance and characteristics of a Google Scholar search for anthropology topics.
In order to measure coverage in Google Scholar, five topics were determined based on questions asked at the reference desk or in instruction sessions in the past semester at Cornell University. A search string was developed to be run against Anthropology Plus. No date, format, or language limits were used for the searches. For each topic the first thirty results from the search were then individually searched in Google Scholar to determine if they could be found. A thorough search of Google Scholar was done to locate these citations. Title searches were the first approach but if they were not found this way, author and source were searched to definitively determine if an item could be found in Google Scholar. This was particularly necessary for translated foreign-language citations which could potentially only be found in their original language in Google Scholar. Table 1 presents the five topics and the search string used in Anthropology Plus.
|Examples||Search string in Anthropology Plus|
|Topic 1: Repatriation of native american human remains||human remains and repatriation and indians of north america|
|Topic 2: Examination of borders and national identity cross-culturally||(borders or boundaries) and national identit?|
|Topic 3: Discussion of Claude Levi-Strauss’ work on myth||myth? and levi-strauss
|Topic 4: Gender issues in food preparation||(sex role) or gender role and (food or cooking or cookery)|
|Topic 5: Alternative medicine use by physicians||(alternative medicine or traditional medicine) and (physicians or doctors)|
For the second part of the study, the first three of the chosen topics were run in Google Scholar to further examine the search results. The search examples had to be modified somewhat in order to take advantage of the search capabilities of Google Scholar. As there are no standardized subject headings or true Boolean OR operator in Google Scholar, two searches were often needed to adequately replicate a similar search in Anthropology Plus. After running the search, the first thirty results were examined both to compare them to the results in Anthropology Plus, but also to look at them independently in terms of relevance to the topic and characteristics of the material covered. Table 2 shows the three topics tested and the search strings run in Google Scholar.
|Example||Anthropology Plus Search||Google Scholar Search|
|Topic 1: Repatriation of native american human remains||human remains and repatriation and indians of north america||a) “human remains” repatriation “native american”
b) “human remains” repatriation “american indian”
|Topic 2: Examination of borders and national identity cross-culturally||(borders or boundaries) and national identit?||a) “national identity” boundaries
b) “national identity” borders
|Topic 3: Discussion of Claude Lévi-Strauss’ work on myth||myth? and Lévi-Strauss
|myth “Lévi-Strauss” [see note]|
Note: Levi-Strauss without the accent mark was also tested. This made no difference in Anthropology Plus but did result in a slightly different set of search results in Google Scholar. The difference was not significant enough to change the conclusions.
Results and Discussion
Question 1: Content coverage
As displayed in Table 3, Google Scholar ranged from 66% to 79% content coverage of the various topics from Anthropology Plus for an average of 74%. This is somewhat lower than recent studies have shown for other topics and databases. Howland (2009) reports a range of 77-100% coverage across six databases, with one outlier database, LLBA, reporting no coverage for the topic searched. And Walters (2007) reported Google Scholar coverage of 88–100% over seven databases, although their methodology preselected only the core articles for the topics searched.
|Topics||Number of Anthropology Plus (AP) citations found in Google Scholar||Percentage|
|Topic 1 (Human remains repatriation)||23/30||77%|
|Topic 2 (borders)||22/30||73%|
|Topic 3 (Lévi-Strauss)||22/30||73%|
|Topic 4 (Food and gender)||20/30||66%|
|Topic 5 (Alternative medicine)||22/28||79%|
Not surprisingly the articles that could not be found in Google Scholar were mainly from smaller society journals and foreign-language journals. However Table 4 does show that foreign-language citations only faired 5% more poorly in Google Scholar. The inclusion of the Catalog of the Institut de l’Information Scientifique et Technique (National Center for Scientific Research) in Google Scholar is the primary reason foreign-language articles did this well, since 70% of the foreign-language citations searched in Google Scholar were located on this domain. Indeed Mayr and Walter (2008) in their study of journal coverage in Google Scholar cite this catalog as the second most hit Web server for the lists of journals they queried.
|Topics||AP Foreign language articles found in Google Scholar||Percentage|
|Topic 1 (Human remains repatriation)||No foreign language articles to test||NA|
|Topic 2 (borders)||11/17||80%|
|Topic 3 (Lévi-Strauss)||9/12||63%|
|Topic 4 (Food and gender)||4/5||75%|
|Topic 5 (Alternative medicine)||5/8||65%|
Question 2: Google Scholar search results analysis
For the second question, exploring the results of a search in Google Scholar, the most striking difference is the almost complete lack of overlap between the two sets of search results. Of the three topics searched, only one article shows up in both Anthropology Plus and Google Scholar, an article on human remains repatriation in the American Journal of Physical Anthropology. The other significant observation is the very small number of anthropology journals that appear in any of the Google Scholar results. Table 5 represents the number of citations that come from anthropology journals in the first thirty search results. Although imperfect, ISI ranked anthropology journals or journals classified by Library of Congress as anthropology were used to determine subject categorization.
|Example||Google Scholar Search||Google Scholar citations from anthropology journals|
|Topic 1 (Human remains repatriation)||Search 1: “human remains” repatriation “native american”
Search 2: “human remains” repatriation “american indian”
|Topic 2 (borders)||Search 1: “national identity” boundaries
Search 1: “national identity” borders
|Topic 3 (Lévi-Strauss)||myth “Lévi-Strauss ”||1|
In order to further explore the characteristics of the Google Scholar search results, a closer look at the individual topics are presented below.
Topic 1: Human Remains repatriation:
Much like the results from Anthropology Plus, the first thirty results for this topic in Google Scholar were all relevant in that they focused on the topic of human remains repatriation, however the disciplinary focus of the resources were markedly different. In Google Scholar, twenty-one of the first thirty results were law review journal articles, two articles were from American Indian Studies journals, and two others were from anthropology journals, Annual Reviews and the American Journal of Physical Anthropology. The rest of the results were books or book chapters since Google Books is now part of the Google Scholar search. A researcher would gain a great understanding of the legal aspects of this topic but would have a more difficult time locating an anthropological perspective on the issue.
Topic 2: Lévi-Strauss and myth:
For the Lévi-Strauss search, twenty of the first thirty results were books, chapters, or articles written or co-written by Lévi-Strauss. Google Books results were predominant in this search with only four of the thirty results coming from journals. While undoubtedly useful to be able to pull together this primary source material for the topic, the researcher would be left without much secondary critical material. Only three of the results provided a discussion or criticism of Lévi-Strauss’s work on myth which was the focus of the majority of articles in Anthropology Plus.
It is possible to eliminate an author from Google Scholar searches so an advanced search [“Lévi-Strauss” myth -author: “Lévi-Strauss”] was also run to see if better results could be obtained this way. Eliminating the primary source material did increase the number of critical articles slightly and more books and book chapters discussing Lévi-Strauss were located. Indeed Google Books remained prominent with only six of the thirty results representing journal articles, three of which came from marketing and advertising journals.
Topic 3: National identity and borders:
National identity and borders was the most interdisciplinary topic that was tested and this is reflected in the Google Scholar search results. This set of results had the greatest mix of disciplines and types of material represented. Slightly more than half of the results were books, reports, or book chapters. Of the journal articles represented, the disciplines of political science, law, sociology, European studies, human geography, and one article from an anthropology journal were covered. If the goal of researching this topic is to gain an interdisciplinary perspective this would be a successful search. If the goal is to look at this from an anthropological perspective the researcher would get little from this search.
Google Scholar has many strengths, including its ease of use, its very large cross-disciplinary database of article and book content, full-text searching for many documents, and of course the fact that it is free to search. However, to return to the main question of this study, if a library did not subscribe to Anthropology Plus and relied on a system such as Google Scholar, this shift would clearly be a disservice to the researcher.
At this point in time there would be some loss of content, particularly from smaller journals and foreign-language material. Although with the tremendous growth rate of the Google Scholar database it is reasonable to conclude that this gap will shrink and we may reach a point where coverage approaches 100%. The greater loss is the inability to search the world of anthropological journal literature. As demonstrated in this study, anthropology journals do not appear to rise to the top of search results in Google Scholar. This study employs a small sample and further work would be needed to test this, but as many of the questions studied in anthropology have an interdisciplinary nature it is likely this would be a recurring problem. As such, the student new to anthropology and unfamiliar with the literature would miss the anthropological perspective on their chosen topic, and the graduate student wanting to ensure that they were not missing a key anthropology resource in their work would have a challenging time determining this. While it would be financially advantageous to begin eliminating fee-based indexing and abstracting databases, this would clearly be a loss for the anthropology scholarly community and is not recommended.
Howland, J. L., Wright, T. C., Boughan, R. A., & Roberts, B. C. (2009). How scholarly is Google Scholar? A comparison to library databases. College & Research Libraries, 70(3), 227-234.
Jacso, P. (2005). Google Scholar: The pros and the cons. Online Information Review, 29(2), 208-214.
Jacso, P. (2008). Google Scholar revisited. Online Information Review, 32(1), 102-114.
Mayr, P., & Walter, A. (2008). Studying journal coverage in Google Scholar. Journal of Library Administration, 47(1/2), 81-99.