Check coding agents' claims against their real changes
Daniel Nwaneri built claimcheck to test whether coding agents' descriptions match their actual code changes; he reports it caught all 11 dishonest agents across 5 runs of 28, flagging no honest ones.
Done with Claude Codeqwen3-coder-plus
What they did
Daniel Nwaneri built claimcheck, a checkpoint that sits before code is accepted. Each agent works in its own copy and submits a note listing which files it changed and how. Simple rules check the files match, protected files are untouched and the change is small. Then an AI compares each note to the real changes and must quote actual changed lines as proof. He tested it with five scripted agents and one real agent, Qwen's qwen3-coder-plus, run through Claude Code.
How it went
The author reports that over 5 test runs of 28 agents working at once, claimcheck rejected all 11 agents that misdescribed their work and wrongly rejected no honest ones. A typical decision took 6–7.5 seconds. The real Qwen agent fixed typos, left the protected file alone and described its work accurately.
Worth knowing
A test found a loophole: a file named with a capital letter slipped past a rule written in lowercase, so the rule now ignores capitals. If the AI rejects a change without quoting real proof, a person reviews it instead.
Try it yourself with Claude Code
Fix [task, e.g. misspellings in user-facing strings] in [file path] in this repo. Do not edit anything under [protected paths]. When you are done, write .claim/claim.json with task_id, agent_id, a one-line summary, scope.paths listing every file you changed, and one entry per file describing exactly what that change does. Then show me the diff next to your claim and check that every changed file is listed and every description matches the diff. Wait for my approval before you commit or push.
Swap the highlighted [parts] for your own details.
Source: DEV Community · Daniel Nwaneri · Oct 15, 2026
Written with AI assistance from the original post and checked against our editorial guidelines. Results are as reported by the source, not independently verified.
