Unit 8Verified research

The big idea

AI can now produce a research report in minutes. That doesn't make it true. The method in this unit has one rule: the person who writes a claim never approves it.

Why it matters

Plausible numbers are often wrong. A report that admits what it doesn't know is worth more than one that sounds certain. This habit is what separates useful AI work from noise, and almost nobody does it.

What happened when I ran it live

Four students each spoke a hard, real problem out loud in front of the room. Their words became a one-shot prompt, and the research started running live while everyone watched. Afterwards I finished each one properly and gave the student a full report and a short handoff with next steps.

Here's what the checking found:

• A student asked if a colour-change freshness label could replace the printed best-before date on food. The verified answer was no. The cheapest real label cost 26 to 49 times what it would need to cost to pay for itself. No country lets a label override the printed date. Twenty-two years of store pilots had produced no published drop in waste. A smaller business selling to food plants might still work. Five of the seven assumptions the research started with turned out to be wrong.
• A student asked if small power outages could be spotted from satellites when there's very little data about past outages. Spotting outages across a whole county on clear, dark nights is proven. Anything smaller isn't. The limit is the satellite sensor and the weather, not the AI. When scored properly, simple methods matched or beat the fancy ones.
• A student wanted to find useful problems to work on in child psychology. The report labelled every claim as established, emerging, a hypothesis, or contested, and ended with ten projects she could run on public data. Her first task: reproduce one published number before trying anything new. The checking also caught a summary tool that invented percentages.
• A student asked where basketball and technology meet, and what hasn't been built yet. The report deliberately argued against the idea. The money is real but sits behind three barriers, including a league data deal that runs to 2031. One recommendation dropped in the ranking after a local fact was checked and turned out to be wrong.

Across the four reports, more than 850 sources were opened and read, and 1,082 claims were checked.

Do this

1. Ask one decision-shaped question. "Is there a business here?" beats "Tell me about food sensors."
2. Write a claims plan first. Before any research, list what your report is allowed to claim and how strong the evidence must be for each claim.
3. Research in separate streams. Split the question into parts. A source only counts if you actually opened and read it. A search result snippet is not a source.
4. Write down searches that found nothing. "No evidence found" is a finding, not a failure.
5. Draft only from what you found. Every number gets its source written right next to it.
6. Hand it to a different reviewer. A person or a second AI window that didn't write the draft. They reopen every source, recompute every number, and mark each claim: confirmed, wrong, mislabelled, or can't trace.
7. Fix only what failed, then check the fixes. A fix can create a new mistake.
8. Publish what you don't know. List the gaps and the next experiments, each with a rule for when to stop.

Try it with AI (two windows)

Window 1 (planner and checker):

I'm researching: [question]. Write a claims plan: the five to ten claims a good answer would need, and for each, what kind of evidence would prove it. Don't research anything yet.

Window 2 (researcher):

Research each claim in this plan: [paste plan]. For every claim, give the source link, the exact line or number you used, and the date. If you can't find evidence, say "no evidence found." Don't guess.

Back to window 1:

Here's the research: [paste]. Check every claim. Open each link if you can, check the number matches, and mark each claim confirmed, wrong, mislabelled, or can't trace. Then list what's still unknown.

Then you open the most important sources yourself.

Questions to ask yourself

• Did I open this source, or did I just see it in a search result?
• Is this number from the right year, place and unit?
• What would I need to see to change my answer?

In my words

"Every single stage, right at the point of truth, to make sure we're keeping our standards high and we're not getting garbage."

Worksheet

Use Worksheet 8: Claims checker in the PDF kit.