Tag Archives: Awards

Color_Logo

SciCast Final Report (Public)

The SciCast 2015 Annual Report has been approved for public release. The report focuses on Y4 activities, but also includes a complete publication and presentation list for all four years.  Please click “Download SciCast Final Report”  to get the PDF.  You may also be interested in the SciCast anonymized dataset.

Here are two paragraphs from the Executive Summary:

We report on the fourth and final year of a large project at George Mason University developing and testing combinatorial prediction markets for aggregating expertise. For the first two years, we developed and ran the DAGGRE project on geopolitical forecasting. On May 26, 2013, renamed ourselves SciCast, engaged Inkling Markets to redesign our website front-end and handle both outreach and question management, re-engineered the system architecture and refactored key methods to scale up by 10x – 100x, engaged Tuuyi to develop a recommender service to guide people through the large number of questions, and pursued several engineering and algorithm improvements including smaller and faster asset data structures, backup approximate inference, and an arc-pricing model and dynamic junction-tree recompilation that allowed users to create their own arcs. Inkling built a crowdsourced question writing platform called Spark. The SciCast public site (scicast.org) launched on November 30, 2013, and began substantial recruiting in early January, 2014.

As of May 22, 2015, SciCast has published 1,275 valid questions and created 494 links among 655 questions. Of these, 624 questions are open now, of which 344 are linked (see Figure 1). SciCast has an average Brier score of 0.267 overall (0.240 on binary questions), beating the uniform distribution 85% of the time, by about 48%. It is also 18-23% more accurate than the available baseline: an unweighted average of its own “Safe Mode” estimates, even though those estimates are informed by the market. It beats that ULinOP about 7/10 times.

You are welcome to cite this annual report.  Please also cite our Collective Intelligence 2014 paper and/or our International Journal of Forecasting 2015 paper (if it gets published — under review now).

Sincerely,

Charles Twardy and the SciCast team

1+

Users who have LIKED this post:

  • avatar

Announcing the Combo Edits Contest!

SciCast’s unique feature is the ability to make conditional edits, also called assumptions or combinatorial edits. Now you’ll have a chance to compete for big prizes for your making at least at least 25% of your forecasts conditional forecasts. combo edits! Continue reading

1+

Users who have LIKED this post:

  • avatar

Help Our Research Efforts and Be Entered to Win an Amazon Gift Card

Please give us a hand with our research and take a moment to complete the latest surveys. On April 10, to thank you for your help, we’ll be conducting random drawings for $1,000 total in Amazon Gift Cards. You earn one ticket for each of these five surveys completed sensibly. Previous completions count. We will not ask for any personally identifying information and we strictly follow human subjects in research protocols.

Surveys we’d like you to complete are available in your dashboard.

They are:

  • Quantitative Analysis (NEW)
  • Demographic Information
  • Professional Information
  • Motivation and Thinking Style
  • Knowledge and Reasoning

 

0