JBS Calibration Training Tool

Overconfidence has been called the most “pervasive and potentially catastrophic” of all the cognitive biases to which human beings fall victim. It leads many of us to decisions which we later regret and has been blamed for lawsuits, strikes, wars, and stock market bubbles and crashes. So what can you do about it? Well the first and most important thing to do, is to find out where you stand on the overconfidence scale. The trick is to get calibrated and this app will help you do exactly that.

Calibrated probability assessments are subjective probabilities assigned by individuals who have been trained to assess probabilities in a way that historically represents their uncertainty. In other words, when a calibrated person says they are "80% confident" in each of 100 predictions they made, they will get about 80% of them correct. Likewise, they will be right 90% of the time they say they are 90% certain, and so on. Calibration training improves subjective probabilities because most people are either "overconfident" or "under-confident" (usually the former). By practicing with a series of trivia questions, it is possible for subjects to fine-tune their ability to assess probabilities.

For example, a subject may be asked:
True or False: "A hockey puck fits in a golf hole"
Confidence: Choose the probability that best represents your chance of getting this question right... 50% 60% 70% 80% 90% 100%

If a person has no idea whatsoever, they will say they are only 50% confident. If they are absolutely certain they are correct, they will say 100%. But most people will answer somewhere in between. If a calibrated person is asked a large number of such questions, they will get about as many correct as they expected. On the other hand, an uncalibrated person who is systematically overconfident may say they are 90% confident in a large number of questions where they only get 60% or 70% of them correct. Calibration training generally involves taking a battery of such tests. This is one such test. Feedback is provided at the end and the idea is to help you match your level of accuracy with your level of confidence.