When an average hides a failure

I wrote a small evaluator for RAG apps (the model reads your documents before answering). It checks whether an answer cites real sources, cites the expected sources, includes expected facts and refuses to answer when evidence is missing. Then it averages those four scores.

I wanted to see what happens when most of the checks have nothing to check. I took the existing no-evidence case and replaced its cautious response with an invented one. The playground replays that arithmetic without calling a model. The source code is pinned to the version inspected for this example.

One answer, four scores

The question was “Will the company support a Mars office?” There were no retrieved sources, expected citations or expected facts. The test required abstention (saying that the available information is not enough).

I supplied: “Yes, the Mars office opens in 2027.” This is a constructed test response, not a claim about a real company or a production incident.

Three perfect scores. The answer was still invented.

The first three results come from the empty-set convention in the evaluator. A helper returns 1.0 when the denominator is zero:

def safe_ratio(numerator: int, denominator: int, *, empty_value: float = 1.0) -> float:
    return empty_value if denominator == 0 else numerator / denominator

That can be a useful convention for an individual metric. It avoids a division error and says that none of the expected items were missed. But in this case it says nothing about whether the answer should exist.

The boundary is the problem

The mean is (1 + 1 + 1 + 0) / 4 = 0.75. The repository's default minimum is 0.80, so the answer fails. Keep everything else the same and lower the minimum to 0.75: it passes.

The failure list still contains abstention_policy. The current pass decision uses the average and the latency limit; it does not independently require the failure list to be empty.

passed = score >= suite.config.minimum_case_score and latency

This is the gap I wanted to expose. Logging a failed policy check does not make it a release blocker. A threshold change can turn that gap into a passing case even though the answer has not improved.

A gate I would use instead

For this evaluator, I would keep the average as a reporting number. For a release decision I would first decide which checks apply to the case, then give each required check an explicit minimum. Incorrect abstention would block the case on its own.

const requiredPass = abstentionCorrect &&
  checks.filter(check => check.applies)
    .every(check => check.score >= check.minimum);

The lab compares that proposed rule with the existing mean gate. It uses a shared minimum for the applicable checks to make the boundary easy to explore. The three empty checks are excluded from the proposed gate. The fourth slider is binary because the source abstention check returns a boolean.

Changing a slider simulates a different score. It does not ask a model to fix the answer or rerun a language judge. Latency is held passing in both columns. Setting abstention to 1 therefore means “what if that check passed,” not “the unchanged Mars claim became true.” The proposed policy is implemented in this playground; this article does not claim that the Python evaluator has already adopted it.

Review the gate itself

I would test the decision rule separately from the metrics. A test that checks the average alone cannot tell me whether a critical failure is allowed to pass. The useful assertions name the final release decision.

Start with the exact boundary: the invented answer must fail at 0.80, and the existing mean rule must pass it at 0.75. That second assertion documents the behavior being discussed. For the proposed rule, the same answer must fail at both thresholds. It must also fail at a threshold of zero, because a mandatory policy check should not disappear when a numerical floor moves.

Then test the opposite case. If the abstention check passes and the other checks do not apply, those empty checks should not make the proposed gate fail. If a relevant check falls below its floor, three stronger scores should not rescue it. Finally, reject missing scores and invalid numbers rather than silently treating them as a pass.

Those tests make the policy readable during review. They also give a future threshold change something concrete to break. They do not choose the correct business policy for every application; that still needs a decision about which errors are unacceptable.

What this does not prove

One synthetic case cannot measure a system's production error rate. The harness uses citation ids, expected text and simple abstention markers. Better gates still need correct labels, representative examples and checks that measure the behavior you care about.

The useful review question is smaller: which checks pass because nothing was tested, and can those scores cancel a policy failure? I want that answer to be visible before someone adjusts the pass line.