# The AI agent's report looks perfect. Should you trust it before you act? - URL: https://www.agentskillpacks.com/guides/check-ai-agent-report-before-acting - Updated: 2026-10-21 - Book: Claude Code for Non-Programmers by İsmail Günaydın (https://www.agentskillpacks.com/books/claude-code-for-non-programmers) ## Short answer Don't act on an AI agent's report until you've checked the findings that would cost you if they're wrong. A report shows the conclusion, not the counting. Agents count from one search or a sample and call it the whole, and the errors run both ways: too many and too few. For each finding that matters, open the file, recount with a different search, and look at three examples yourself. Facts with a fixed shape, like IDs, links and statistics, need a lookup at the source, never memory. ## Twenty minutes later, a perfect report You sent a helper agent to look over your website while you answered email. Twenty minutes later it's back. Headings. A severity column. A neat count at the top. "14 pages are missing descriptions. 6 images slow down the whole site. Recommended action: fix all." It reads like something you'd pay a consultant for. _Fine. Fix all._ Before you type that, ask one question. Did anyone actually count? ## Why reading it more carefully won't help The report is a summary. You see the conclusion. You don't see the counting. So you can read it twice and still not know. A careful read tells you it sounds right. It can't tell you if the agent searched one way and missed the pages written another way. Claude Code does scan helper reports before the main session reads them. It neutralizes text that pretends to be system messages, and it marks reports that talk about permissions. That protects you from a report that gives orders. It is not a fact check. It won't recount your pages. ## Twelve chickens, and two behind the shed You send your little brother to count the chickens. He's back in a minute. "Twelve!" You know there are fourteen. He counted the ones he could see from the back door. He wasn't lying. He counted from one spot and said the number as if it covered the whole yard. On July 2, 2026, my agents did the same thing on one of my sites. "5 pages are missing metadata" was really 3. A fix for a slow image would have made loading slower on most pages. And a link problem the agent put at 109 files was really 334. Too many and too few, in one report. That week gave me the three-step check. For each finding that would cost me if wrong: open the file, recount with my own search, look at three by hand. The [best practices page](https://code.claude.com/docs/en/best-practices) in the docs makes the same point from the other side. Give Claude a check it can run, because it stops when work looks done. ## Ask for proof in the request, not after Checking is cheaper when the report arrives with its receipts. The book's reviewer rules go at the end of every review or audit prompt. These four do most of the work: ``` Every finding needs proof: file name + line number + the exact text you saw. No proof, no finding. Give real counts, and say how you counted: "3 files, found by searching for X and for Y." If you only looked at a sample, write "sample of N". Report real problems only. If you find nothing important, say "nothing important found." That is a good result. Do not trust web page summaries for numbers. Quote numbers only from the raw page text, and say which page they came from. ``` The third rule matters more than it looks. A reviewer asked for problems will find some. Saying that "nothing important found" is a fine answer makes its real findings worth more. ## What a report you can trust looks like Put the report next to this list before you act on it. - **It says how it counted.** Which searches, in which folders. "14 pages" with no method is a guess with a number on it. - **It says if it sampled.** "Sample of 20" is honest. A sample written up as the whole is the chicken count. - **It shows raw output for every "verified."** If Claude says all links or IDs were checked, ask: "Show me the raw output of each check." No output means no check. - **It points to real places.** File, line and the exact text, so you can open one in ten seconds. - **It says what it did not look at.** A report that covers everything usually covered less than it says. A report that meets all five still gets the three-step check on any finding that would cost you. It just takes less time. ## Nine IDs, eight wrong Some facts are worse than miscounts. They are invented. On September 18, 2026, I audited one of my sites. Its hidden data listed partner companies, each with an ID from Wikidata, a free public knowledge base. There were nine IDs. Eight were wrong. A payments company pointed to a human-rights group. A sales platform pointed to a plant. Three IDs didn't exist at all. The code had shipped under a note that claimed a perfect score. The IDs had been written from memory. The note was just as sure as the IDs. The fix took seconds. One call to the [Wikidata API](https://www.wikidata.org/w/api.php) returns the real label for any ID. Now no ID goes live on my sites without that lookup. A report saying "all verified" with no raw output means nothing was verified. ## The book's chapters on checking This is the problem [Claude Code for Non-Programmers](/books/claude-code-for-non-programmers) was written for. The book is independent and is not affiliated with Anthropic. **A Team of Agents** gives you the reviewer rules I paste into every review request, a helper file that truly can't write, and the Finding Check table that turns "the agent said" into "I counted." **Safety and Failure** sorts every claim into four boxes, from machine-checkable to judgment, and ends with a fact-check list for anything going public. **Real Business Systems** shows the same checks inside real jobs, like auditing 22 sites, where agents find candidates and a full pass confirms them. ## Where this is in the book - A Team of Agents - Safety and Failure - Real Business Systems Kindle: https://www.amazon.com/dp/B0HN6WSTMG Paperback: https://www.amazon.com/dp/B0HNKT2XHV Links go to Amazon. I wrote this book and earn a royalty on every sale. This book is independent and is not affiliated with or endorsed by Anthropic. ## Is the book for you? - Yes, if A helper agent handed you a neat report with counts and a fix-all list - Yes, if You're about to tell a client what the AI found - Yes, if Claude says all IDs or links are verified, and you aren't sure what that means - Skip it if You want an agent you never have to check - Skip it if You only use the chat window for drafts you read anyway ## FAQ ### Why are AI agent reports wrong? Common shapes: one search treated as the truth, a sample treated as the whole, a web page summary treated as the source, and a check that measures something next to what you care about. The mistakes go both ways. An agent can report too much or too little. ### Does Claude Code make up facts? It can. A hallucination is a confident, well-formatted statement with nothing behind it. The book finds them clustered in IDs and codes, links and citations, quotes and named people, statistics and market judgments. Claude Code's own documentation doesn't publish a hallucination rate. ### How do I make a Claude Code agent read-only? Use the tool list, not a sentence. In the helper's file, a disallowedTools line that takes away Write, Edit, Bash and PowerShell means it can't change files or run commands. The built-in Explore and Plan helpers already can't write. ### How do I check a number in an AI report? Open the file it points to. Ask Claude in your main session to recount using a different search than the agent used. Then open three of the results yourself. Tell people "I verified, the real number is 3," never "the agent said 5." ### Can I trust Claude's web research numbers? Treat them as leads. When Claude fetches a page, it gets a model-written summary, not the raw page, and details can blur or appear that the page never said. For a number that matters, ask for the exact sentence and the address, and open it yourself. ### What should an AI agent's audit report include? Proof for every finding: the file, the line and the exact text it saw. Real counts with the method, such as which searches it ran, and a note if it only checked a sample. Raw output for anything it calls verified. And a plain "nothing important found" when that is the truth. A report missing these needs more checking, not less.