Skip to content

The work and its checks can share the same mistake

This is one result from the free test Is Evidence Keeping Up With Your Output? You are checking. The checks pass. The reason they pass is that the check and the work descend from the same starting point, so a mistake in that starting point is invisible to both. This is the profile where a green result is the least informative thing on your screen.

Two checks are not two checks when they share a parent

We treat independent confirmations as multiplying. If one check passes there is some chance of error, and if two pass there is much less. That arithmetic only works when the two checks can fail for different reasons.

When both derive from the same brief, the same prompt, the same source document, or the same model in the same session, their errors are correlated. Ten passes are not ten pieces of evidence. They are one piece of evidence, repeated ten times, at ten times the cost and ten times the confidence.

This is why the profile is about shared mistakes rather than missing checks. You are not skipping the work. You are doing the work in a way that cannot produce a surprise.

The circle has been measured

The machine-learning field ran into this directly when it started using models to score model output. Research at NeurIPS 2024 found that language models recognize their own generations and rate them more favorably, and that the strength of the self-preference bias tracks how well the model recognizes its own text.

The deeper objection is structural, not statistical. When generation and evaluation happen in the same space, the evaluation is justified by the same kind of judgment it is meant to audit. Improving the model improves both halves of the circle at once.

You inherit this whenever the checker sees the artifact that produced the work: the spec you wrote, the transcript above, the summary the tool already gave you. It is not specific to models. A reviewer handed your reasoning along with your conclusion is checking your arithmetic, not your premise.

Three shared parents

A test suite generated from the same description as the implementation. Both encode the requirement as "sorted ascending." The requirement was descending. Every test is green.

An analysis and its sanity check both drawing on one export. The export filtered out canceled orders. Revenue is overstated in the analysis and confirmed by the check.

A second model asked to review the first model's answer, in a thread where the first answer is visible. It anchors on what it can see and produces a fluent endorsement with three minor suggestions.

In all three, the reviewing step is skilled and the reviewing step is downstream of the mistake.

The trap is that adding more checking makes it worse

This is the difference that matters between this profile and the others. Elsewhere, more effort buys something. Here, more effort buys confidence without buying information, and confidence is what lets the work through.

Teams in this state often describe a specific failure afterwards: "we had reviews, we had tests, it still shipped wrong." That sentence is usually accurate. The reviews and tests were real. They all pointed downstream of the same bad assumption.

Break the inheritance, do not add a layer

The principle: the value of a check is the chance it fails for a reason the work could not anticipate. Judge a check by its independence from the work's origin, not by its thoroughness.

Concretely, for the next piece of work:

  1. Write down the answer before you look. Take the one claim the decision rests on and get its correct value from a source that had no part in producing the work. Record it first, so it cannot be adjusted after you see the output.
  2. Withhold the reasoning from the reviewer. If a person checks it, give them the claim and the decision it supports. Do not give them the prompt, the brief, or the model's explanation. Those are the shared parent.
  3. Use a different tool, a different session, or a different starting document. Same model, fresh context, no visibility of the first answer is meaningfully better than a review inside the thread. It is not as good as a source outside the model.

You know it worked when a check can fail while the work follows its starting instructions exactly. If failure requires the instructions to have been disobeyed, the inheritance is intact.

Where this is not the right worry

Some work has no external truth to appeal to. Tone, structure, taste, and framing are judged against intent, and intent lives in the brief. Checking those against the brief is correct, and asking for independence there is a category error.

Reserve this for claims with a fact of the matter: numbers, states of systems, policy, dates, behavior, outcomes. Those are the claims where a shared parent turns into a shared error someone acts on.

If your situation is that no external answer exists at all yet, rather than that your checks inherited one, start at Nothing outside the work can prove it wrong. That page is about building the first falsifier. This one is about a falsifier that turned out to be a relative.