Skip to main content

Interrogate & Evaluate Resources — Evaluation Toolkit

Evaluation Exploration Toolkit

Lego Hiker

This is a toolkit for you to rummage around in, taking questions to ask and ways to think from each. No one set of questions has all the answers - it's up to you, the intrepid information explorer, to assemble your expeditionary tools, as you evaluate the information you find.

Information is created for a reason. Nothing slipped & fell and ended up on the internet, or in print, or on TV. It takes an act of human intentionality to create and publish information. This toolkit will give you a starting point to pull back the curtain on what you read, watch, listen to, or experience, and think about how you interact and respond to the information you find.

Four Moves

  • Check for previous work: Look around to see if someone else has already fact-checked the claim or provided a synthesis of research.
  • Go upstream to the source: Go “upstream” to the source of the claim. Most web content is not original. Get to the original source to understand the trustworthiness of the information.
  • Read laterally: Read laterally. Once you get to the source of a claim, read what other people say about the source (publication, author, etc.). The truth is in the network.
  • Circle back: If you get lost, hit dead ends, or find yourself going down an increasingly confusing rabbit hole, back up and start over knowing what you know now. You’re likely to take a more informed path with different search terms and better decisions.

 

In general, you can try these moves in sequence. If you find success at any stage, your work might be done.

When you encounter a claim you want to check, your first move might be to see if sites like Politifact, or Snopes, or even Wikipedia have researched the claim (Check for previous work).

If you can’t find previous work on the claim, start by trying to trace the claim to the source. If the claim is about research, try to find the journal it appeared in. If the claim is about an event, try to find the news publication in which it was originally reported (Go upstream).

Maybe you get lucky and the source is something known to be reputable, such as the journal Science or the newspaper the New York Times. Again, if so, you can stop there. If not, you’re going to need to read laterally, finding out more about this source you’ve ended up at and asking whether it is trustworthy (Read laterally).

And if at any point you fail–if the source you find is not trustworthy, complex questions emerge, or the claim turns out to have multiple sub-claims–then you circle back, and start a new process. Rewrite the claim. Try a new search of fact-checking sites, or find an alternate source (Circle back).

Created by Mike Caulfield.

Want to learn why he came up with this?

Six Questions

What type of content is this?

  • Recognize first what kind of content you’re looking at.
    • Is it a news story? Or is it an opinion piece? Is it an ad or what some people call native advertising produced by a company? Is it a reaction to someone else’s content?

Who and what are the sources cited and why should I believe them? 

  • News content usually cites sources for the information provided. These are the people quoted, or the documents or reports or data being referred to.

Evidence: What's the evidence and how was it vetted?

  • Evidence is the proof that the sources offer for what they know. It overlaps with how close someone is to an event. But even highly credentialed sources may begin to speculate sometimes. They may be guessing.
    • So, first, identify the evidence that any source is offering. Circle it. Write it down. Do it as an exercise a couple times. It becomes easy to recognize.

Interpretation: Is the main point of the piece proven by the evidence?

  • Most media content offers a thesis, or main point, of some kind [...]  ask whether this main point makes sense, and whether the conclusions are supported by the evidence offered.

Completeness: What's missing?

  • Most content should lead to more questions. An important step in being a critical, questioning consumer is to ask yourself what you don’t understand about a subject. Look back at the piece. Did you miss something? Or was it not there?

Knowledge: Am I learning every day what I need?

  • Think about what media you consumed yesterday. What did you learn about? What did you read about?

WAIL Your Sources!

Word Choice

  • What do you notice about the choice of words? For example, are the words formal or informal?

Adjectives & Adverbs

  • How do the adjectives & adverbs shape meaning and bias?

What's Included?

  • What information does the source make sure to include?

What's Left Out?

  • What information has been left out of the source? Is there important information missing?
Based on these questions, what conclusions can you draw about your source?

BEAM: How to Use a Source

This method was created to help you think about different sources and how you might put them all together for your final product.

Background Sources

  • Sources accepted as unquestionable fact, undisputed information
  • Examples: dictionaries, encyclopedias, and other reference materials

Exhibits or Evidence Sources

  • Sources used for explication, analysis, or interpretation
  • Examples: often primary sources like newspapers, primary research articles, data

Argument  Sources

  • Sources from other authors you are agreeing with, disagreeing with, or building upon
  • Examples: scholarly articles or books

Method Sources

  • Sources an author follows to determine how they are doing their research. Key terms, a specific procedure, model or perspective
  • Examples: textbooks, literature reviews with commentary on research

 

Bizup, Joseph. "BEAM: A Rhetorical Vocabulary for Teaching Research-Based Writing." Rhetoric Review 27, no. 1 (2008): 72-86. doi:10.1080/07350190701738858