Tag Archives: scaled

SciCast Accuracy Calculations

We have started adding “Accuracy” numbers to emails. For example:

  • Your average accuracy on this question was 83.
  • SciCast’s average accuracy on this question was 90.

What does that mean?  The short answer is that it’s a transform of the familiar Brier score, which we have mentioned in several blog posts.  Where the Brier measures your error (low is good), Accuracy measures your success (high is good). This is more intuitive … except when it’s not.

Continue reading


Users who have LIKED this post:

  • avatar

Scaled Continuous Question

The following shows an example of a Scaled or Continuous question:


Instead of estimating the chance of a particular outcome, you are asked to forecast the outcome in natural units like $.  Forecasts moving the estimate towards the actual outcome will be rewarded. Those moving it away will be penalized.  As with probability questions, moving toward the extremes is progressively more expensive: we have merely rescaled the usual 0%-100% range and customized the interface.

Continue reading