The Third Umpire · Lesson 11 — The Balance ← Course

The balance — F1 and the trade-off.

You can't max both. Turn the trigger finger and precision and recall move opposite ways — so choose by what a mistake costs.

Precision and recall are not independent dials — they are two ends of one. The umpire's eagerness to give OUT is that single control: turn it up and it catches more real edges (recall rises) but calls more wrong dismissals (precision falls). Turn it down and the reverse. You are always trading one against the other.

So there is no “best” setting in the abstract — it depends on which mistake hurts more. When a wrong dismissal is the disaster (acting on a false alarm — blocking a good payment, deleting real mail), favour precision. When a missed dismissal is the disaster (letting a real one through — a disease screen, a security check), favour recall.

And when you want a single number that balances the two — one that punishes ignoring either — you use F1, the combined score of precision and recall. It peaks when the two are in balance and sags the moment one collapses.

Slide the trigger finger from cautious to eager and watch precision, recall, and F1 move against each other.

◂ cautiousthe trigger fingereager ▸
Precision OUT calls that were right0%
Recall real outs that were caught0%
F1 precision & recall, combined ◂ at its peak0%
Accuracy for reference0%
favour precision when…

a wrong dismissal is the costly error — flagging fraud, deleting spam. Better to let a few slip than to punish the innocent.

favour recall when…

a missed dismissal is the costly error — a disease screen, a security scan. Better a false alarm than a real one getting through.

One dial, two mistakes, your call.

Precision and recall pull against each other on a single control — buying one spends the other. F1 gives you one balanced score, but it cannot tell you which mistake you can least afford; only you can. Choose the balance by the cost of being wrong, then measure it and hold the line.

That completes the toolkit: accuracy for the headline, the confusion matrix for the two mistakes, precision and recall to measure each, and F1 to balance them. Now you can say — with numbers, not vibes — whether the umpire is any good.

Next: Lesson 12 →
▸ Go deeper — F1, thresholds, and weighting optional

Why F1 is a harmonic mean

F1 = 2 · (precision · recall) / (precision + recall). It uses the harmonic mean, not a plain average, on purpose: an umpire with 100% precision and 0% recall averages 50 but scores an F1 of 0. The harmonic mean refuses to reward one number while the other collapses — you must do reasonably on both.

The dial is really a threshold

Most models output a score — a probability of “out” — not a yes/no. A threshold turns that score into a call, and sliding it is exactly the trigger finger here. Plotting precision against recall as the threshold moves gives the precision–recall curve, the full picture of a model's trade-off.

Weighting one over the other

When one mistake really is worse, Fβ lets you weight recall or precision more heavily instead of treating them equally. F1 is just the balanced case where β = 1.

The choice is domain, not maths

No formula tells you whether a false alarm or a miss is worse — that comes from the real cost in your setting. The metrics measure the trade-off honestly; deciding where to sit on it is a judgement about consequences.