You may have noticed some upgrades to the SciCast interface that should make your experience even better! For instance, during Incentive Weeks (when you can win prizes) we’ve added new Leaderboard views. You can now toggle to see the leaders for each prize day.
Following is a graph of users’ chances of winning gift cards on Thursday last week. It is based on the expected points for forecasts on a single day. Expected points calculated over more days are the basis of the Thursday and Friday leaderboards.
Additional Updates
Newly registered forecasters are immediately given a leadoff question to get started.
And, all forecasters are offered recommended questions based on previous forecasts. Enjoy, and let us know what you think!
have you considered expanding the leaderboard to show the top 20? there are more active users this days, and perhaps more exciting movement going on in the 10-20 range.
We’ll consider it.
Do you intend to update the “Accuracy” boards again? They didn’t change for last week’s activity. My forecasting on Thursday was pretty accurate, enough to earn over 1000 points and put me (@Tender) in the Top 10 overall, but not enough to win an Amazon gift card. I would really like to see the top performers from Thursday and learn what is required to qualify for these accuracy gift cards!
The awards don’t have any correlation whatsoever with your total points gained that day, or (unlike Tuesdays) how hard you “worked.” I won four gift cards and did not even log in that day. Two recurring bets (Tesla <35k and Mars Curiosity) were apparently enough to put me at the top of the heap.
@tendernow/Tender, thank you for your inquiry and for being one of our more accurate and active users.
We don’t update the leaderboards every day, only when we run the related lotteries. That means that the Thursday leaderboard is updated once each Friday. The expected points are calculated at random times, and the update follows that as soon as possible. Just in case the leaderboard had something wrong, I updated it a few minutes ago using the most recent expected points calculation. Tender is higher on the list now.
Selecting who wins the 20 gift cards each day they’re available is based on a lottery, so our best forecasters are sometimes missed. That is far less likely to happen at the end, when we award three-times as many gift cards based on longer term accuracy with a more stable expected value calculation.
@kennyth0, this does seem a bit dubious. I actually checked the Leaderboards over the weekend and they were definitely not updated as of Sunday morning. The overall “Point Leader” board hadn’t changed since mid-week, and the old Thursday board included names that hadn’t made a forecast since July. Your update today was the first in quite a while. As it stands, I am (like @Tender) ahead of @Phoenix, and also received nothing. Even assuming that you took the contest snapshot one second after his recurring forecasts, it seems mathematically unlikely that two forecasters in the current Top 9 received nothing, while the current #10 received four.
Is it possible that whatever backend problem prevented the Leaderboard from updating for several days may have also corrupted your Thursday lottery? Would it be possible to see the distribution from which you randomly selected the winners?
It may be pathetic to be so motivated by gift cards, but I am a pathetic person!
@relax852014/Relax, I’ve looked into the issue further. I don’t manage the main leaderboard, so perhaps the problem with leaderboards was unrelated to the lottery. Our main engineer says that there is a possibility that leaderboards did not update on Friday and Saturday, but we have no way to know. Nevertheless, the Thursday leaderboard would not reflect people’s chances of winning the one lottery because the leaderboard is based on all forecasts from Thursdays during the four-week special period and the lottery is based on forecasts from just one Thursday. The leaderboards are meant to provide a rough sense of how users will fare in the final lotteries when more is at stake. I’ll post the distribution of the chances of winning for Thursday last week (for all users with chance greater than zero). When I recalculated the chances based on the most recent expected points calculations, the distribution was similar. The strange outcome of the lottery appears to have been random.
Your statement about motivation by gift cards is interesting because the accuracy prizes are part of our research. Are there other non-monetary prizes you can imagine that would motivate you as strongly as the chance to win gift cards? When the gift cards stop, what will keep you in the market?
@phoenix548/Phoenix, estimating long-term accuracy when questions haven’t resolved is messy. We’re currently considering contests that run longer (into the next tax year!) and use other methods so that our estimates of accuracy would be more valid. We have a limited budget, but suggestions for improvement are welcome.
See the histogram added to the main post.
personally i’m more motivated by the recognition of high placement on the leaderboard than the gift certificates. they’re nice, but pursuing them isn’t even close to financially rational for me.
more leaderboards comparing participants by more metrics (high/mean/median/low score over last X days, score increase over X days, etc) would give me more places where i can do better, which would be appealing.
i get a small kick out of the simple little badges, too. adding more, like “positive winnings on X questions in a row”, “earned more than 1000 points on a single question”, “took a >1000 point risk on a question, and didn’t lose” or “maintained >Xth place for 30/60/90 straight days” would be nice (and free). see stackexchange for a sense of the variety of badges that could be awarded (and shorter, more clever names that could be used.)
Those are good badge ideas. Thanks.
Thanks for the feedback, @kominek.