Grain of Salt

Evaluating sources: how to check where a claim came from

Evaluating sources

Struck diagram on assay stock: Evaluating sources

Evaluating sources of information is the work of deciding how much weight a claim deserves by tracing it back to where it came from and asking what that origin can actually support. It is a judgment about a specific claim in a specific source, not a permanent grade awarded to a publication. The same newspaper, the same agency and the same textbook can be authoritative on one sentence and worthless on the next, so the question is never "is this a good source" but "is this a good source for this".

Most claims reach you two or three steps from their origin. A friend repeats a post, the post summarizes an article, the article paraphrases a report, and the report describes a study that nobody in the chain has opened. Source evaluation is the habit of walking back up that chain until you reach something that can be checked against the world, and then noticing what changed at each step on the way down.

What evaluating information sources actually asks

Evaluating information sources asks 3 questions, in a fixed order, and the order is what makes it work.

  1. Who is making this claim, and how would they know? Access matters more than title. A shipping clerk who counted the pallets knows the count; a professor of logistics who read a summary does not.
  2. What is the claim resting on? Every claim rests on something: a measurement, a document, a witness, a model, or another claim. If you cannot name what it rests on, you have not reached the source yet.
  3. Who else could have checked this, and did they? A claim that no independent party was ever in a position to contradict carries less weight than one that survived people trying.

Notice what is not on that list. Whether the page looks professional, whether the author is confident, whether the conclusion matches what you already believed. Those are the signals people actually use, and they are the reason source evaluation has to be a deliberate procedure rather than an impression.

Credibility, reliability and relevance are three separate tests

Credibility, reliability and relevance get used as synonyms and they measure different things. Separating them is most of the skill.

Credibility

Credibility is the grounds a source has for being believed on this subject: direct access to the evidence, training in the method used, accountability if it turns out to be wrong, and disclosure of who paid. A trade body's report on its own industry is not disqualified by the funding, but the funding is a fact about the report and belongs in your reading of it.

Reliability

Reliability is consistency over time. Would this source produce the same answer if the same question were asked again, and what happens when it gets something wrong? A source with a visible corrections record is more reliable than one with a spotless record, because the spotless record usually means nobody is checking. Look for the correction, not the absence of one.

Relevance

Relevance is whether the source addresses your actual question. A rigorous 1994 survey of commuting patterns in a single city is a strong source and an irrelevant one if your question is about freight in 2026. Relevance failures are the most common and the least noticed, because a good source feels like an answer even when it is answering something else.

Five checks that survive contact with the web

These five checks survive contact with the web because none of them depends on how the page looks. They come out of research on how professional fact checkers read, notably the Stanford History Education Group's work on lateral reading, which found that experienced checkers leave a page almost immediately to find out what other sources say about it, while students stay on the page and study its design. The digital literacy researcher Mike Caulfield packaged four of these moves as SIFT: stop, investigate the source, find better coverage, and trace claims to the original.

  • Stop before you pass it on. The single highest yield habit is the pause. Almost nothing you read needs to be relayed within the minute.
  • Read laterally. Open a new tab and find out what independent parties say about this source. Never rely on the source's own About page, which is written by the interested party.
  • Trace the claim to the original. Follow the citation back until you reach the study, the dataset, the transcript or the photograph. The distinction between what a document records first hand and what it repeats from elsewhere is set out on Primary vs secondary sources, and it is the distinction that makes tracing worth doing.
  • Find better coverage. Ask whether anyone with independent access reports the same thing. Ten outlets carrying the same wire copy are one source, not ten.
  • Check the date and the version. Web pages are edited silently. The Internet Archive's Wayback Machine will often show you what a page said last year, which is how you discover that a figure was quietly revised.

A YouTube video deserves a specific note, because video defeats several of these checks by feeling like direct evidence. A video is not a source, it is a container. The source is whoever is speaking, and whatever they are holding up to the camera. Ask what the video would look like if the claim were false, and you will usually find it would look identical.

Evaluating information sources in print and on the web

Print and web sources differ on 5 dimensions that matter, and on none of the dimensions people usually cite. Print is not more truthful; it is harder to change after the fact. The web is not less rigorous; it is less filtered before publication and far more traceable after it.

DimensionPrint sourceWeb source
Filtering before publicationUsually some, from an editor, publisher or academic pressRanges from none to the full peer review process
FixityHigh. A printed page cannot be silently altered once distributedLow. Text, figures and dates can be changed without notice
DatingReliable. Edition and printing are statedOften ambiguous. A displayed date may be publication, update or today
Traceability of the chainSlow. Footnotes lead to a library, an interlibrary loan and a waitFast. A digital object identifier can take you to the original in seconds
Correction visibilityPoor. The correction appears somewhere the reader is notGood when done well, invisible when not

The practical consequence is that print and web require the same questions with different tactics. For a printed book you check the publisher, the edition, the bibliography and whether the author had access to the archives cited. For a web page you check the archive snapshot, the date stamp, the outbound citations and what other sites say about the domain. Libraries sit across both: a university library's catalog and its subscription databases exist precisely to give you a filtered slice of the academic literature, and a librarian is the single most underused resource in this whole procedure.

A source evaluation worksheet you can run in four minutes

Run these 7 questions on any source before you cite it, quote it or forward it. The worksheet is deliberately short, because a checklist you complete is worth more than a rubric you abandon.

  1. Name the claim in one sentence, in your own words, with any numbers in it.
  2. Name the source of the claim, meaning the author and the outlet, not the person who sent it to you.
  3. Find the date the claim was made, and the date of the evidence behind it. These are frequently years apart.
  4. Follow one link back, and then one more, until you reach something that is not itself a summary.
  5. Search the source's name in a new tab and read what two independent parties say about it.
  6. Ask what is missing: the denominator, the control group, the sample, the years excluded, the alternatives not tested.
  7. Write down your confidence in one word: solid, plausible, or unverified. Most things are the third.

The classroom ancestor of this is the CRAAP test, developed at the Meriam Library of California State University, Chico, which scores a source on 5 criteria: Currency, Relevance, Authority, Accuracy and Purpose. It is a good teaching device with one weakness worth knowing: it keeps you on the page. Every criterion can be satisfied by a source that is carefully designed to satisfy it. Mastery here is not memorizing more criteria, it is the reflex of leaving the page to check.

Where source checking stops and reasoning begins

Source checking stops at the point where the evidence is established and the argument begins, and this border is where most disputes actually live. A perfectly credible source can be used to support a conclusion it does not reach. This is the territory of Logical fallacies, and two of them are direct hazards to anyone who has just learned to evaluate sources.

The first is the genetic fallacy: concluding that a claim is false because of where it came from. A funded study is a reason to check the methods, not a reason to believe the opposite of its result. The second is a symmetrical error, treating a strong source as a guarantee for every sentence it contains, including the parts outside its competence and the parts where it is summarizing someone else.

Both errors have the same shape and the same cure. A soil survey commissioned by the developer who wants to build on the site is a source with an obvious interest, and the correct response is to read its method and commission a second survey, not to assume the ground is unstable because the wrong people paid for the test. Evaluating sources of information is meant to end in a check, not in a verdict on whoever produced it.

So here is the test to carry into tomorrow, and it takes about ten seconds. When a claim arrives, ask yourself: what would I have to see to conclude this source is wrong about this particular claim, and how hard would it be to go and look? If you cannot answer the first half, you have not understood the claim yet. If the second half turns out to be easy, you have no excuse. The chain from a study to the headline you actually read is traced step by step on Media literacy, and it is usually shorter and leakier than it looks.

Where to go next

Proudly powered by WordPress | Theme: Amber Blog by Crimson Themes.