We have started adding “Accuracy” numbers to emails. For example:
What does that mean? The short answer is that it’s a transform of the familiar Brier score, which we have mentioned in several blog posts. Where the Brier measures your error (low is good), Accuracy measures your success (high is good). This is more intuitive … except when it’s not.
Users who have LIKED this post:
The market is better calibrated than we thought, but not perfect. In our previous calibration post, each question counted once. In the chart below, each forecast counts once, which is the usual method.
Users who have LIKED this post:
The following shows an example of a Scaled or Continuous question:
Instead of estimating the chance of a particular outcome, you are asked to forecast the outcome in natural units like $. Forecasts moving the estimate towards the actual outcome will be rewarded. Those moving it away will be penalized. As with probability questions, moving toward the extremes is progressively more expensive: we have merely rescaled the usual 0%-100% range and customized the interface.
