Glossary

Calibration

Calibration is the correspondence between stated confidence and observed frequency of being correct: judgments called “70% likely” should be right about seven times in ten across a well-defined class of cases. It measures the quality of confidence statements, not whether a single decision had a good outcome.

Why it matters

Organizations often reward confident narratives. Poor calibration — chronic overconfidence or underconfidence — distorts portfolio bets, hiring, and forecasting. Improving calibration improves the inputs to expected-value and Bayesian updates; it does not by itself choose the right strategy.

Calibration is assessed over many comparable judgments, not one win. A correctly calibrated 70% forecast will still be wrong 30% of the time. Decision quality of the process and quality of the outcome must stay distinct: good calibration can coexist with bad luck, and lucky outcomes can coexist with miscalibrated bravado.

Example (enterprise): A revenue leadership group tracks forecast confidence bands for deals. When “high confidence” closes only half the time, they retrain qualification criteria rather than celebrate the occasional surprise win as proof of process excellence.

Mechanisms

  • Express uncertainty as explicit probabilities or confidence classes on comparable questions.
  • Track outcomes against those classes over a meaningful sample.
  • Feedback and adjust — correct systematic over/underconfidence in future judgments.

Boundaries

  • Calibration ≠ accuracy of a single forecast; it is long-run confidence-frequency match.
  • Calibration ≠ Bayesian updating; one can update coherently yet remain miscalibrated.
  • Calibration ≠ decision quality overall; it is one attribute of judgment hygiene.

Related topics

None linked yet.

Related concepts & frameworks

Further research

Calibration — Definition for Decision Quality | The Strategy Stack