As an academic librarian, I always caution patrons about properly using different types of sources (popular vs. scholarly, peer-reviewed, grey literature, and news sources). With regards to Wikipedia, I don’t discourage them from going there, as I use it all the time. Wikipedia, as a successful crowdsourcing project, has been surrounded by controversy due to its open editable model. In spite of the level of skepticism it has been receiving from scholarly communities for decades, Wikipedia still remains one of the popular choices for online information users. The perception that Wikipedia articles lack the quality control rigor of academia is now shifting, since now more users are relying on it for fact-checking and background information. Even academic institutions, such as libraries, are contributing to Wikipedia with content.
Wikipedia has shared some fundamental principles, called the Five Pillars:
- Wikipedia is an encyclopedia
- Wikipedia is written from a neutral point of view
- Wikipedia is free content that anyone can use, edit, and distribute
- Wikipedia’s editors should treat each other with respect and civility
- Wikipedia has no firm rules
Out of all these pillars, the second one (in boldface) is of interest to me due to the aspects of currency, accuracy, and reliability, which are so important when determining if sources are good or not.
Selwyn and Gorard (2016) conducted a study on the treatment of Wikipedia by academia, pointing out a key finding regarding its usage: “Wikipedia mainly plays an introductory and/or clarificatory role in students information gathering and research.” But in spite of the great news this represents for students, one must practice caution when using Wikipedia for research papers and other forms of scholarship. The fact that anyone can write or contribute to a Wikipedia article…anonymously, should give you enough reason to do your due diligence as a scholar-to-be to make sure that the article you are reading can be verified from other reliable sources. `
Here’s a guide to help you evaluate a Wikipedia article, librarian-style. And how do we librarians assess the content of a source? By doing the CRAAP test. Yes, because librarians like myself do give a CRAAP about Wikipedia and see some value in what it offers to students in their initial phases of research.
B. The CRAAP Test
CRAAP is an acronym that stands for Currency, Relevance, Authority, Accuracy, and Purpose. This test was created by Sarah Blakslee (U of California, Chico), to serve as a way to assess sources’ accuracy and reliability. All five components are key in assessing any type of source you encounter, whether it comes from Google, Wikipedia, a governmental website, or in grey literature (e.g. pamphlets, conference proceedings, annual reports, etc.). Let’s take a quick look at each one of these components and the type of questions you could pose when reading a source, like a Wikipedia article.
Currency – The timeliness of the information
Questions we should pose when checking for Wikipedia article’s currency:
- How was this page first developed?
- How has it evolved?
- How has its content been revised or updated?
- Does this topic require current information?
Relevance – The importance of the information
for your needs
Questions we should pose when checking for Wikipedia article’s relevance:
- How informative is this page?
- Is there a diversity of contributors?
Authority –The source of the information
Questions to pose when checking for levels of authority present in Wikipedia article’s authority:
- Who are the key contributors to this page?
- Are they qualified to create and update content? If so, in what ways?
- Has a contribution or a revision to the Wikipedia article been made anonymously?
Accuracy – The reliability, truthfulness and
correctness of the content
Questions to pose when checking for Wikipedia article’s accuracy:
- How informative is the content presented?
- Can the content be supported by evidence?
- Has this content been reviewed or refereed by qualified authors?
- Does the article provide with additional sources that verify its content?
Purpose – The reason the information exists
Questions to pose when determining why Wikipedia article was written:
- Why was it produced?
- What have been some of the main points of discussion among contributors and readers of this DH page?
- How is this information going to be used?
C. The Digital Humanities Wikipedia page: Does it pass the CRAAP test?
A reliable source needs to pass all five criteria items. Just because an article is current and accurate, it does not mean that it is authoritative. Just like an article that is authoritative, it does not mean that is still holds currency if it was written decades ago and new findings have proven it to have obsolete ideas. So let’s give a CRAAP about Wikipedia and see if the following article passes the test:
The Digital Humanities Wikipedia Page (See Figure 1)
Use the following set of activities as a guide to deconstruct a Wikipedia article with the use of the CRAAP test. There are no right or wrong answers, and so the answers you will generate through these exercises do not have to match the ones I have provided as a guide. These exercises will only provide you with a partial way to determine if the DH Wikipedia page abides by the five criterion that comprise the CRAAP test.
Every Wikipedia article has a page history section that lists its previous iterations, each one with its corresponding time stamp contributor’s username, and a brief rationale statement. Article revisions are shown from newest to oldest, each one displayed one line at a time. Exploring the revision history of a Wikipedia article is key in understanding how its content has been revised and kept up-to-date. The following three activities will allow you to partially determine the currency of the DH Wikipedia page.
- When was the latest modification made?
- Who was the contributor?
- Why was the modification made?
Take a few moments to find the answer, then compare them with the following response:
The latest modification, was the reincorporation of three sources. This modification was made on November 2019, by username internateArchiveBot. (See Figure 2)
Relevance can be determined by finding out where the information comes from. Does the Wikipedia article include citations? Who is the intended audience? Is the content relevant for DH professionals, as well as for students interested in DH, or grant institutions? Is it diverse enough to capture the attention of different communities?
Relevance Activity #1: How informative and diverse is this page?
Browse through some of the different versions this page has experienced in the past 13 years. You may do this by visiting the “View History” tab of the DH page, selecting an edit line of interest, and clicking the (cur prev) links provided compare versions. To compare an old version with the current version, visit the old version then click cur. To a version with its previous iteration, click on prev. (See Figure 4 to view differences between editions)
As you do this with several versions, take a look at the diversity of contributors and contributions. Any surprising findings? Take a few moments to explore the different versions , then compare notes with the following response:
This page has information value in terms of the contributors’ cautious approach in defining DH and their acknowledgment of the difficulty in defining this field. The section on Values and Methods, although strangely included without any reference links (except for the last item), showcased this definitional complexity of DH. The two strongest sections of this page are Projects and Criticism.
The Projects page is very helpful in showcasing the amplitude of efforts undertaken by scholars in diverse contexts. It focuses not so much on the tools, but on the capabilities of DH initiatives in understanding humanistic phenomena and on the collaborative nature of these types of projects.I also really appreciate how responsive this page has been in addressing tough issues appearing in subsections like, Negative Publicity, Black Box, Diversity, and Issues of Access.
The most surprising part of this page is the lack of content under the Tools page, which is indicative of a focal shift from a technology-centric conceptualization to a more cultural-centric approach to DH. It also helps to have an introductory sentence or paragraph that explains the rationale behind each section.
If you have trouble finding the different versions of a Wikipedia article, please refer to the following source, which uses the example of the entry for “Heavy Metal Umlaut” to explain the evolution a of Wikipedia article:
In order to learn about the contributors of a Wikipedia page, you should become familiar with the User’s Page. If the edit was not made anonymously, we can check the User’s Page to see user’s biographical information, including interests, education, and professional experience.
Authority Activity #1: Find the User’s page of a contributor
Go back to the View History tab and go to the oldest page of the history log (back to 2007), and locate the individual by the username “Geertivp”. Click on “Geertivp” and identify the following:
- Name, Work Experience, Education, Contributions
- Can you locate the user’s website? Is this person appearing in social media outlets? Which ones? Could you contact this person if you needed to?
Compare notes with Figure 5:
Authority Activity #2 Who were the early contributors of the DH Wikipedia page?
The very first contributor was Dr. Elijah Meeks. Contributors from 2007 include: Dr. William McCarty (External links, Objectives), Dr. Gabriel Bodard (Objectives, See Also), SimonMahony (Objectives, Bibliography), and Unsworth (Software). Towards the end of 2008, GoldCorrie and Ms. Erin Belle contributed content (External Links) From 2011, this page has contributions from Teagrant and ARK (Terminology), and from Alf7e (Degree Programs, Organizations and Institutes). Sophia Chang, was also a contributor to this page (Centers and Institutes, Journals and Publications), until blocked in 2018.
Authority Activity #4: Describe a biographical profile of some of the main contributors. What can we learn from their biographies?
It’s always a good exercise to learn a bit about contributor’s biographical profiles, as they provide a window into their projects and approaches to topics, in this case, the discipline of Digital Humanities.
Take a few moments to visit some of the contributors’ homepages or websites (external to Wikipedia article page) and learn more about about their biographies. Compare notes with the following response:
The originator of the Wikipedia DH page is Dr. Elijah Meeks, who currently serves as the Digital Humanities Specialist at Stanford University, specializing in developing humanistic research questions through spatial analysis, topic modeling, and visualizations. Meeks has been involved in key DH projects, including: The Digital Gazeteer of the Song Dynasty, ORBIS, and Mapping the Republic of Letters, among many other endeavors. Dr. Simon Mahony currently serves as Director of the UCL Centre for Digital Humanities, with research and teaching interests in the application of digital technologies in learning communities and collaborative environments.
Dr. Gabriel Bodard is a digital classicist (School of Advanced Studies, U. London), interested in methodologies and approaches of digital humanities in the subjects of ancient history, archeology, and classics. Mr. John Unsworth is the current Dean of Libraries at the University of Virginia. His research focus has been cyberinfrastructures for the humanities and he is credited for helping develop the HathiTrust Research Center. Dr. Willard McCarty is Professor of Humanities Computing (King’s College) and Digital Humanities (U. Western Sydney), with research foci in interrelations between DH and humanistic disciplines, interdisciplinary research, as well as computing historiography.
Matthew Kirchenbaum, who currently teaches and directs the Digital Studies Certificate Program at the Maryland Institute for Technology in the Humanities, claims to have successfully cohabited both the humanities and technology realms throughout his life and career. Erin Bell, MLIS, is the Technology Director of the Center for Public History + Digital Humanities at Cleveland State University. Other notable contributors include Rudolf Ammann, researcher at the UCL Centre for Digital Humanities; Amanda French, research and consultant with expertise in DH grant writing.
Authority Activity #5: Determine how these biographical profiles might influence they way these editors understand the topic?
Just to exemplify this influence, I would like to highlight a the possible influence of contributors, such as Bodard and McCarty, in expanding portions of the page, like the See Also section (2007) with the addition of topics (e.g. Digital Classicist and Digital Medievalist).
This CRAAP criterion can be partially correlated to one of Wikipedia’s Five Pillars,Wikipedia is written from a neutral point of view , which states:
“We strive for articles in an impartial tone that document and explain major points of view, giving due weight with respect to their prominence. We avoid advocacy, and we characterize information and issues rather than debate them. In some areas there may be just one well-recognized point of view; in others, we describe multiple points of view, presenting each accurately and in context rather than as “the truth” or “the best view”. All articles must strive for verifiable accuracy, citing reliable, authoritative sources, especially when the topic is controversial or is on living persons. Editors’ personal experiences, interpretations, or opinions do not belong.”
Let’s see how trustworthy and informative the DH Wikipedia page is, by (1) taking a look at the links provided, and by reading about the diversity of DH constructs provided by scholars.
Accuracy Activity #1: Indicate if the DH Wikipedia page can be trusted
Take a few moments to assess the accuracy of the DH Wikipedia page, then compare notes with the following response:
Part of defining the purpose of information is by studying the major debates involving an issue and they ways in which consensus is reached among contributors and readers to resolve such issues. In the case of Wikipedia articles, it’s important to read about the different types of modifications that have taken place and the discussions that these generated. Do contributors provide with a rationale as to why modifications were needed? Were issues resolved for particular topics? Are there any biases or underlying motives that influenced the way the article content is presented?
There are two Wikipedia pillars that speak of how these discussions contribute to the open, editable model by which an article should be created and curated: Wikipedia’s editors should treat each other with respect and civility AND Wikipedia is free content that anyone can use, edit, and distribute
One of the ways to encourage contributors to work towards a common purpose in a respectful manner is the “Talk Page” feature. Every Wikipedia page includes “talk page”, where editors discuss improvements to articles and debate what stays and what goes. David Auerbach describes (and questions) the purpose of this “talk page” in his article “Encyclopedia Frown”:
“In the best case, which does occur reasonably often, spirited debate on the “talk” page of an article results in ongoing negotiations and refinement to an article until it is truly high quality. The editors who can work harmoniously in pursuit of this ideal goal of neutral ‘consensus’ can make editing Wikipedia a wonderful and productive experience.”
The following activities will help determine why content is created or modified, how it is contested by the Wikipedia DH community of discussants, and how disagreements have been resolved in ways that can sustain the main purpose of the DH page.
Purpose Activity #1: Study a major issue that generated a great deal of controversy on the DH page?
Take a few moments to identify topics discussed or debated regarding content revisions for this page. To do so, go through the edition history and select an edit line (which includes a date stamp, username, and type of edit). Click on the associated talk page, by clicking on the “talk” link. Compare notes with the following response:
D. The Verdict: Does the Digital Humanities Wikipedia article pass the CRAAP test?
Your response is as correct as mine and I welcome your comments below. Here’s my verdict:
The Wikipedia Digital Humanities article has the potential for passing the test, in its currency, relatedness to the topic of Digital Humanities, the high qualification of its contributors, and the accuracy of its content.
The high number of edits has been made to this page, its diversity of collaborators, and the way the page has evolved, I can say have contributed an increase in its quality. In their study on Wikipedia . Wilkinson and Huberman (2007) established the following relationship between degree of collaborative editing and high-quality articles: “We examined all 50 million edits made to the 1.5 million English-language Wikipedia articles and found that the high-quality articles are distinguished by a marked increase in number of edits, number of editors, and intensity of cooperative behavior, as compared to other articles of similar visibility and age.” This correlation seems to be happening with the DH Wikipedia page. This is an article that has gone through 13 years of edits, curated by a group of high-caliber contributors, becoming an interesting venue of interactions and debates. The DH page is a testament on how the process of reaching consensus in such venue can help improve and move the field of Digital Humanities forward.
Feel free to use the CRAAP criteria when reading other articles of interests, regardless of the type of source. And speaking of sources, here are the sources I used for this analysis.
Auerbach, David. “Encyclopedia Frown.” Slate, December, 11, 2014.
Selwyn, N., & Gorard, S. (2016). Students’ use of Wikipedia as an academic resource—Patterns of use and perceptions of usefulness.
Wilkinson, D. M., & Huberman, B. A. (2007, October). Cooperation and quality in wikipedia. In Proceedings of the 2007 international symposium on Wikis (pp. 157-164). ACM.
“Evaluating Information – Applying the CRAAP Test Meriam Library California State University, Chico”. https://library.csuchico.edu/help/source-or-information-good