The final SciCast annual report has been released! See the “About” or “Project Data” menus above, or go directly to the SciCast Final Report download page.
SciCast has seen over 11,000 registrations, and over 129,000 forecasts. Google Analytics reports over 76K unique IP addresses (suggesting 8 per registered user), and 1.3M pageviews. The average session duration was 5 minutes.
Users who have LIKED this post:
SciCasters:
Thank you for your participation over the past year and a half in the largest collaborative S&T forecasting project, ever. Our main IARPA funding has ended, and we were not able to finalize things with our (likely) new sponsor in time to keep the question-management, user support, engineering support, and prizes running uninterrupted. Therefore we will be suspending SciCast Predict for the summer, starting June 12, 2015 at 4 pm ET. We expect to resume in the Fall with the enthusiastic support of a big S&T sponsor. In the meantime, we will continue to update the blog, and provide links to leaderboard snapshots and important data.
Through the course of this project, we’ve seen nearly 130,000 forecasts from thousands of forecasters on over 1,200 forecasting questions, and an average of >240 forecasts per day. We created a combinatorial engine robust enough to allow crowdsourced linking, resulting in the following rich domain structure:
Near-final question structure on SciCast, with most of the live links provided by users. (Click for full size)
Some project highlights:
MIT and ANU researchers studied SciCast accuracy and efficiency, and were unable to improve using stock machine learning — a testimony to our most active forecasters and their bots. [See links for Della Penna, Adjodah, and Pentland 2015, here.]
What’s Next?
Prizes for the combo edits contest will be sent out this week, and we will be sharing a blog post summarizing the project. Although SciCast.org will be closed, this blog will remain open as well as the user group. Watch for announcements regarding future SciCast.
Once again, thank you so much for your participation! We’re nothing without our crowd.
Contact
Please contact us at [email protected] if you have questions about the research project or want to talk about using SciCast in your own organization.
So Long, and Thanks for All the Fish
Users who have LIKED this post:
Please join us for a live webinar tomorrow, May 7 at 12PM EST. SciCast, Cybersecurity Markets and the Near & Far Future of AI is the second installment of TechCast Global’s webinar series. In the course of one hour, we will feature three thought-provoking segments and give attendees an opportunity to ask questions and interact with our panelists.
Users who have LIKED this post:
Exciting new questions are live on SciCast! Check them out, listed by category and make your predictions. Continue reading
The SciCast Accuracy Incentives contest has come to an end. The final accuracy leaderboard will be selected at a random time in the next few weeks. This will allow more questions to resolve as well as randomize the selection time to reduce the impact of attempts to inflate scores on unresolved contest questions.
Read more about the contest.
We have started adding “Accuracy” numbers to emails. For example:
What does that mean? The short answer is that it’s a transform of the familiar Brier score, which we have mentioned in several blog posts. Where the Brier measures your error (low is good), Accuracy measures your success (high is good). This is more intuitive … except when it’s not.
Users who have LIKED this post:
Today we disabled 7 accounts for blatant gaming behavior, particularly this time for dumping points from a shill account to a main account. When accounts are disabled, the account owners receive an email, if they have provided a valid email address. Otherwise they will see a note when they try to log in. We will not name the accounts until the owners have had a chance to respond to us. Continue reading
Users who have LIKED this post:
Our sponsor has made the sensible request that we discount forecasts after the answers are known — even if it took us awhile to pause the question. We have now instrumented SciCast to record and show ‘Date Known.’ Forecasts after ‘Date Known’ will still be part of the historical record, but they will neither gain nor lose points. Remember to check this during the mandatory 48-hour comment period following a proposed resolution. (In the future it may be possible to soften the cutoff, or create a fixed bonus for first correct report of a resolution.)
SciCast has been featured in a Wall Street Journal article about crowdsourced forecasting in the U.S. intelligence community. We’re excited to share that SciCast now has nearly 10,000 participants, a 50% increase in the last two months - an important achievement for a crowdsourced prediction site.
Have you ever wondered what will be the next ‘big thing’ in technology? What if you could garner collective wisdom from your peers - those who are interested in the same topics as you – with global reach?
Don’t miss two unique opportunities to learn more about how you can do this on SciCast (www.scicast.org), the largest known science and technology-focused crowdsourced forecasting site.
SciCast will be the featured topic in a Reddit Science AMA and an American Chemistry Society webinar this week! Don’t miss these opportunities to share your SciCast expertise and weigh in on the discussion. We also encourage you to share the information with your friends and colleagues.