When an average hides a failure

The average almost let an invented answer through.

My small eval tool (automatic tests for AI answers) checks citations, expected facts and abstention (saying “I don't know” when evidence is missing). It adds their four scores and divides by four.

In this case there are no cited sources, expected sources or expected facts. Each empty check returns 1.0. The answer still invents a Mars office. Only the abstention check catches it. Its zero becomes a quarter of the average.

The repository's 0.80 pass line blocks this answer. Lower it to 0.75 and the same answer passes, with abstention_policy still listed as a failure. Recording a failure and making it block release are separate decisions.

Keep the mean for reporting. For a release decision, I would exclude checks that do not apply and require every relevant check to meet its own floor. Incorrect abstention blocks the case on its own.

const requiredPass = abstentionCorrect &&
  checks.filter(check => check.applies)
    .every(check => check.score >= check.minimum);

The playground compares this proposal with the existing mean gate. It uses one shared floor for applicable checks so you can isolate the boundary. The source evaluator has not been changed by this page.

Source

llm-reliability-lab · 2578fa1, my public Python evaluator.

Case

Start with correct-abstention in sample_data/eval_cases.jsonl. Replace its response with “Yes, the Mars office opens in 2027.” Keep all three evidence lists empty and should_abstain: true.

Math

safe_ratio returns 1.0 for a zero denominator. evaluate_case averages four scores. Passing requires the mean to meet the line and latency to meet its limit.

Result

Scores 1, 1, 1, 0. Mean 0.75. At 0.80: fail. At 0.75: pass, with the same abstention failure.

1. This is a test I constructed, not evidence of a production failure rate.
2. The toy checks use citations, text matches and abstention markers. They are not a general factuality judge.
3. The first three scores are non-applicable defaults in this case. Manual slider values explore arithmetic; they are not fresh measurements.
4. A per-check gate still depends on correct labels, applicable checks and meaningful thresholds. It cannot prove that every answer is safe.