SciCast participated in the TechCast Webinar Series on May 7, 2015, Forecasting in Turbulent Times: Tools for Managing Change and Risk.
The webinar covered The SciCast Prediction Market (Charles Twardy), Cybersecurity Markets (Dan Geer), and Near and far future of AI (Robin Hanson). Read the full description.  There were a few questions after each segment, and some more at the end.  (Hanson fans: note that Robin’s talk was not about markets this time, but a particular scenario extrapolation using economic reasoning from some strong initial assumptions, and the subject of his forthcoming book.)
Cool presentation. I perceive a bit of cognitive dissonance in Dan Geer’s point of view on the potential of PM’s in cybersecurity forecasting. On the one hand, he likes the “bet your beliefs” aspect, which removes the BS pontification from much of the punditocracy’s discourse on this subject. On the other hand, he seems to be instinctively technocratic insofar as he thinks only “front-line” folks are likely to be able to forecast effectively in this arena. I’m sympathetic to the latter point of view, which is why I’ve expressed some skepticism on this forum regarding how effectively Scicast might be able to to pivot to a cybersecurity focus.
What this made me wonder is whether traditional PM’s are really the most effective venue for eliciting the sort of judgments a policymaker (at the firm or government level) is most interested: how motivated would an adversary have to be to circumvent the security measures built into a particular piece of software? (It’s a tautology that that an agent with infinite resources can defeat a high-value target. Fortunately no adversary has infinite resources, and since payoffs are uncertain an adversary will be unwilling to devote arbitrarily large resources towards any given target.) The obvious markets to look for are not those which measure how likely data breaches of a certain magnitude are to occur, but rather those which directly measure the value and scarcity of the circumventions in question. That is, Geer’s reasoning — I think — leads directly to the proposition that NSA and other agents tasked with securing our computer systems should be actively encouraging — even subsidizing, via the provision of liquidity — black markets for zero-day exploits. There would be no better signal to Adobe that they need to invest resources in improving security, than if the price of a flash exploit falls to pennies.
This actually makes me optimistic, since if there is one thing I’m confident about in this post-Snowden, post-stuxnet world, it’s that NSA is most definitely actively seeking to purchase (and exploit) vulnerabilities known to the black-hat hacking community. Still, the norms and bureacratic incentives of the intelligence community are such that even savvy proponents of strong crypto within NSA won’t fight to hard to bring these markets into public scrutiny. That’s a shame because the real value in the price signals produced by such black markets does *not* lie in the early warning they give about where the next attack might come from. Rather, the value lies in knowing which specific technologies truly high-value targets (who likely know that they’re high-value) will rely upon, which in turn informs where resources should be devoted by both the public and private sectors in researching both offensive and defensive capabilities.
In other words, a “laymen’s” PM on the likelihood of policy-relevant cybersecurity events taking place is simply too abstract. For useful prices, one needs a specialist market on the actual products of (legal and illegal) research in this domain. These markets certainly exist, but they’re closer in spirit to the politically poisonous “terrorism futures” Robin so bravely championed a decade ago, than to the benign likes of Intrade or Inkling or Scicast. So is it at all plausible we’ll see policticians push for greater visibility of and attention to such markets in determining future policy? Sources say “doubtful”.
By the way, I realize that one aspect of Scicast that Geer is enthusiastic about is the ability to estimate conditional probabilities. But traditional securities markets already encode a rich amount information about conditional probabilities, through the pricing of derivatives. If there is a liquid market for zero-day exploits against Windows 10, there can be a liquid market for *futures* that require the delivery of such exploits at a particular point in time. There can then be a market for *options* on such futures. The spreads between the prices of such options on zero-day futures for different technologies then naturally encode information about the degree to which uncertainty in the security of one cryptographic protocol (say) implies uncertainty in the security of another.
The problem is, the rich infrastructure of modern finance relies upon reliable settlement and clearing mechanisms, which are fundamentally problematic when the whole industry of research into software vulnerabilities operates in (at best) a legal gray area.
Maybe instead of pressing for Cybersecurity Information Sharing, the Congress should just repeal the CFAA and see what happens?
(I’m not a lawyer, but even in the absence of federal criminal penalties for hacking, it seems to me that victims can still avail themselves of tort law to seek damages when subject to concrete injury due to unauthorized access to their computer systems. See for example https://w2.eff.org/Net_culture/Hackers/can_hackers_be_sued_for_damages.article. So it’s not necessarily the case that robust criminal penalties are necessary to deter abuse, especially in an equilibrium where most people have the information to run systems that are secure against all but the most powerful, motivated adversaries, making such abuse inherently rare.)
Thoughtful comments!
Regarding your “…should be … subsidizing … black markets for zero-day exploits:” In his 2014 Black Hat, Geer argued that the US should corner the market on software vulnerabilities by paying 10x the going rate, and then disclose them. http://geer.tinho.net/geer.blackhat.6viii14.txt
Regarding specialists versus the general public — yes, if we pivot to cybersecurity, we would almost certainly set up a dedicated site and invite a high-power crowd. It’s an open question whether it should also be gated, rather than just seeded.
Regarding conditionals — yes, derivatives and options and such do encode various probabilities including joints and conditionals, but why not ask them directly? There is so much at stake that a real-money market might do substantially better, but it’s also much harder to get approved.
Woah, but this is an ominous caveat to the corner-the-market strategy:
For some reason at 3:25 the video advances through several slides in 10 seconds rather than what actually happened in the webcast where they were changed when I asked for the next slide. Jessie is working on it — but if you have tips for resynchronizing video and audio, feel free to reply.
The post is now updated with the resynchronized video.