How to Detect Overconfidence
Detecting overconfidence means noticing when assigned credence outruns evidential support.
Watch for Certainty Markers
Overconfidence often announces itself through language: obviously, impossible, no serious person, everyone knows, case closed. These phrases may be justified sometimes, but they should trigger a calibration check.
Use the Rival Test
Ask whether a reasonable rival could hold a different credence without being stupid or corrupt. If yes, your certainty should probably come down. The existence of competent disagreement does not make all views equal, but it often widens warranted uncertainty.
Check Track Record
Overconfidence becomes visible over time. Keep a small prediction log. If your 90 percent claims are true only 60 percent of the time, the problem is not bad luck; it is miscalibration.
Look for Missing Error Bars
Overconfidence often shows up as a claim with no error bars. The speaker reports a conclusion but not the uncertainty, alternatives, source limits, or conditions under which the conclusion would weaken.
When you notice that pattern in yourself, add the missing structure: "I am confident because..., my main uncertainty is..., and the evidence that would lower my confidence is..." This turns confidence back into something inspectable.
Adjacent References
These are starting points rather than exhaustive citations. They connect this page to nearby work in probability, calibration, cognitive bias, and social epistemology.
- |Daniel Kahneman and Amos Tversky, "Judgment under Uncertainty: Heuristics and Biases" (1974).
- |Sarah Lichtenstein, Baruch Fischhoff, and Lawrence D. Phillips, "Calibration of Probabilities" (1982).