Marc and I ended up interpreting my forecast and resolving it in different ways, though with the same ultimate conclusion. I looked at the above figure and tried to eyeball a point on the Rainbow curve that was about 5% of the way through the 200 million frames - that is, at 10 million frames, a little bit after the dotted vertical at 7%. Based on this eyeballing the Rainbow figure, I concluded that the rainbow-colored line was above the horizontal line (roughly corresponding to A3C’s final performance) by the time 10 million frames is reached - somewhere around the time that the rainbow-colored line is yellow.

Marc, on the other hand, started from A3C’s score (reported elsewhere) and tried to figure out how many frames it used, which was surprisingly difficult (sidenote: Marc is a coauthor on a paper which touches on related evaluation issues). After some back and forth, he wrote:

“I was going by the assumption that A3C consumed about 320M frames of data, which from squinting at the Rainbow graph would put us just short of the 5% mark (but around 7-8%). However, I can't find that # frames figure in the paper. The UNREAL paper has # frames for A3C, but reports mean scores, not median scores. However, if we cross reference the UNREAL learning curve with the A3C mean score reported in Vlad Mnih's 2016 paper, we get a result that suggests about 1 billion frames for A3C, or 180 frames per second -- which sounds about right for the 4 days of training reported in Mnih et al. So you might be right, & I'll adjust my statement to...yes [for this forecast].”

Score: True
_____

Brier Score Calculation

To introduce Brier scores, the means by which I grade my forecasts, I’ll start by excerpting Wikipedia at length:

“The Brier score is a proper score function that measures the accuracy of probabilistic predictions. It is applicable to tasks in which predictions must assign probabilities to a set of mutually exclusive discrete outcomes. The set of possible outcomes can be either binary or categorical in nature, and the probabilities assigned to this set of outcomes must sum to one (where each individual probability is in the range of 0 to 1). It was proposed by Glenn W. Brier in 1950.[1]
...the Brier score measures the mean squared difference between:

• The predicted probability assigned to the possible outcomes ...
• The actual outcome ...

Therefore, the lower the Brier score is for a set of predictions, the better the predictions are calibrated. Note that the Brier score, in its most common formulation, takes on a value between zero and one, since this is the largest possible difference between a predicted probability (which must be between zero and one) and the actual outcome (which can take on values of only 0 or 1).”

Brier scores are calculated in a few different ways, but here I will use the common formulation mentioned above, in which 0 is the best, 1 is the worst, and .25 means you’re essentially forecasting randomly.

Using the common formulation of the Brier score and the scores above, we get:

*drum-roll*

.22

The calculation I plugged into Wolfram Alpha was, if you want to try variations on it:

((.8-1)^2+(.6-1)^2+(.6-1)^2+(.95-1)^2+(.7-0)^2+(.6-1)^2+(.7-0)^2+(.6-0)^2+(.6-0)^2+(.9-1)^2+(.6-0)^2+(.7-1)^2) /12

So, I did better than chance, but barely. Sad!  If you changed a few forecasts, it would push it over the line in the other direction (towards being systematically wrong), and it would take several forecasts being flipped in my direction for me to be very (meta-)confident about the future of AI.

Note that Brier scores give an incomplete picture of how good one is at forecasting, however, so being close to .25 doesn't mean I did a horrible job, though I might have. Two other factors are key.

First, some domains are harder to forecast than others - without more analysis of past data and past forecasts, we don’t know how random or predictable AI progress is. I'd be curious to see people do this with datasets like the AI Index and the EFF AI Progress Measurement Project, and I'd like to see more people make and evaluate more forecasts.

Second, Brier scores are about the forecasts you actually made, not other possible forecasts. I could have made many easy forecasts to boost my score. Arguably, the only one was the "no human-level or superintelligent AI" one. With many virtually certain forecasts that turned out correctly, my score would have been closer to 0.

In forecasting tournaments, it's typical to have a range of difficulty levels for questions, and to have many people answer the same questions. That way, you can know whose forecasts are more or less impressive. It also helps to have some base rates of predictability for the domain in question, which, again, we (/I) don't have yet.

Perhaps more importantly than the overall Brier score and the considerations above, though, is what can be learned from the errors.

Patterns

Over the course of the year, as I started to see how forecasts would resolve, I reflected a bit on the sources of errors, and I’ve noticed a few patterns.

First, all of my errors were in the direction of overestimating short-term AI progress. This isn’t super (meta-)surprising to me, since I consider myself pretty bullish about AI, and would be surprised to find myself in the opposite camp. And it is also at least potentially consistent with (Roy) Amara’s Law: “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”

I am not sure I should radically update in response to this, though, because first, there is an asymmetry in some of my forecasts such that it would have been impossible to end up underestimating progress (because I didn’t give an upper bound, e.g. with the “superhuman” forecasts for Labyrinth and Montezuma’s Revenge); and second, I’m generally of the belief that there are bigger risks from underestimating technical progress than overestimating it, so if there is an inevitable false positive/false negative tradeoff, I'd like to be slightly on the overestimating side of things. Most of the things we should do in response to expectations of rapid future progress are pretty robustly beneficial (e.g. more safety, ethics, and policy research, or improving social safety nets). But there are cases where that isn’t true, and I made some actual errors that aren’t explainable by the asymmetry point, so I will try to update at least somewhat in response to this pattern and move towards a more calibrated estimate of progress.

Second, in the case of speech recognition and Montezuma’s Revenge, I deferred too much to a small number of relevant experts who were (and are) bullish about AI progress. I suspect part of the reason this happened is that I spend a lot of time talking to people who are concerned about the societal implications of AI, and some such people arrive at their concern based on views about how quickly the technology is progressing relative to society’s speed of preparation (though not all people concerned about societal implications can be described that way or are bullish). I’ll try to take this into account in the future, and more deliberately seek out and associate with skeptics about progress in particular technical areas and with regard to AI in general.

Third, I generally did better in areas that I am familiar with relative to areas I am unfamiliar with (e.g. Atari=familiar, speech recognition=unfamiliar). This isn’t surprising, but worth reflecting more on. Sometimes these factors overlapped (e.g. speech recognition is also an area I’m not that familiar with, and I deferred to someone on it). So I’ll try to limit my forecasts in the future to areas I’ve spent a non-trivial amount of time understanding, like deep reinforcement learning applied to control problems like Atari. I was thinking about making some forecasts for deep reinforcement learning for continuous control problems (like robotics) this year, and now am not so sure about that, because I’ve read such papers less carefully and extensively than Atari-related discrete control papers.

Fourth, the forecasts I made based on quantitative trends did well relative to my forecasts based on other factors like (small n) expert judgment, my gut feelings, and my perception of the hotness of a given research area. There is some evidence from other forecasting domains that quantitative forecasts often fare better than expert judgment. And it’s (in retrospect, at least) unsurprising that hotness isn’t a great signal, since often people work on problems that aren’t just important and timely but also difficult and intellectually interesting, like transfer learning.

Sixth, the problems with evaluating AI progress, which is notoriously difficult, are also applicable to forecasting AI progress. Issues with underreporting of evaluation conditions, for example, made it tricky to determine whether some of the forecasts were right. Some of the ambiguities in my forecast resolutions could have been avoided had I been more specific and spent more time laying out the details, but that would have been more likely to happen in the first place if widely agreed upon evaluation standards were used in AI, so I can’t take all the blame for that.

Conclusion

Overall, I found this retrospective analysis to be humbling, but also helpful. I encourage others to make falsifiable predictions and then look back on them and see what their error sources are.

I’ve definitely grown less confident as a result of doing this Brier score calculation, though I also have become generally more skeptical of other people’s forecasts, since, first, most people don’t make many falsifiable public predictions based on which they can be held accountable and improve their calibration over time, and second, several of my failures were based in part on trusting other people’s judgments. As noted above in the discussion of Brier scores, it's not clear how good or bad my score was, because we (or at least I) don't know what the underlying randomness is in AI progress, and I didn't give myself many freebies.

However, I continue to be impressed, as I was last year, with the power of simple extrapolations, and may make more of these in the future, perhaps using methods derived from the Baidu paper mentioned above, or along the lines I discussed but didn’t follow up on in last year’s post, or using other approaches.

I’m not sure yet whether or when I will write a similar blog post of forecasts for 2018. If I do, I’ll try harder to be more specific about them, since, despite trying hard to make my forecasts falsifiable in 2017, there was still a fair amount of unanticipated wiggle room.

To be continued*!

*confidence level: 70%

____

Recommended Reading

In addition to the various links above, I’m including pretty much the same reading list that I gave last year, for those who want to know more about issues of technological forecasting in general, AI progress measurement in particular, and related issues. I’m adding an additional reference by Morgan on the promise and peril of using expert judgments for policy purposes, and the aforementioned expert survey by some of my colleagues on the longer-term future of AI.
____

Anthony Aguirre et al., Metaculus, website for aggregating forecasts, with a growing number of AI events to be forecasted: http://www.metaculus.com/questions/#?show-welcome=true

Stuart Armstrong et al., “The errors, insights and lessons of famous AI predictions – and what they mean for the future”:  www.fhi.ox.ac.uk/wp-content/uploads/FAIC.pdf

Miles Brundage, “Modeling Progress in AI”: https://arxiv.org/abs/1512.05849

Jose Hernandez-Orallo, The Measure of All Minds: Evaluating Natural and Artificial Intelligence: https://www.amazon.com/Measure-All-Minds-Evaluating-Intelligence/dp/1107153018

Doyne Farmer and Francois Lafond, “How predictable is technological progress?”: http://www.sciencedirect.com/science/article/pii/S0048733315001699

Katja Grace et al., “When Will AI Exceed Human Performance? Evidence from AI Experts,” https://arxiv.org/abs/1705.08807

Katja Grace and Paul Christiano et al., AI Impacts (blog on various topics related to the future of AI):  http://aiimpacts.org/

M. Granger Morgan, “Use (and abuse) of expert elicitation in support of decision making for public policy,” PNAS, http://www.pnas.org/content/111/20/7176

Luke Muehlhauser, “What should we learn from past AI forecasts?”: http://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/what-should-we-learn-past-ai-forecasts

Alan Porter et al., Forecasting and Management of Technology (second edition): https://www.amazon.com/Forecasting-Management-Technology-Alan-Porter/dp/0470440902

Tom Schaul et al., “Measuring Intelligence through Games”: https://arxiv.org/abs/1109.1314

I have a long-standing interest in understanding how predictable AI progress is, and occasionally make my own forecasts. In this post, I’ll review some of my previous forecasts, make new ones for 2017, and suggest ways that I and others could make better predictions in the future.  The purpose of this post is to gather all of the forecasts in one place, keep myself honest/accountable regarding AI forecasting (which was part of the point of making the forecasts in the first place), see what if anything can be learned so far, and encourage others to do more of the above.

For those most interested in the stuff on Atari (my more quantitative forecasts) and my new predictions, and less interested in how my other miscellaneous forecasts fared, just read this blog post. If you want to know more about the process I went through to review all of my other forecasts, and to see how I did on predicting non-Atari things, see this supplemental page.

Atari Forecasts

I’m focusing on these because they’re forecasts about which I’ve thought a lot more than the ones linked to above, and about which I've made more specific data-based forecasts. I also have a lot of data on Atari performance, which will be made public soon.

In early 2016, I made a simple extrapolation of trends in mean and median Atari performance (that is, the best single algorithm’s mean and median score across several dozens games). For both mean and median performance, I made a linear extrapolation and an exponential extrapolation.

I also said:

As the use of the word “sidenote” suggests, this wasn’t that rigorous of a forecast. I just took the available data and assumed the trend was either linear or exponential, and that the future data would be between those two lines. I’ll mention some ways I could have done this better later in this post. But it turned out to be fairly accurate, which I find interesting because it’s often claimed that progress in general AI is nonexistent or impossible to predict. In contrast, I think that general game playing is one of the better (though still partial) measures of cross-domain learning and control we have, and it’s fairly steady over time.

Here is the same plot, with two recent scores added, along with the range I forecasted. The light blue ovals are the data I had as of April, the lines are the same as those I plotted in April, the dark blue stars are very recent scores, and the red oval is the range I forecasted before. Two of DeepMind’s ICLR submissions are roughly at the bottom and top of the range I expected based on eyeballing the graph in April. Obviously, this could have been done more rigorously, but it seems to have been about right.

Note that there is eventually some upper bound due to the way evaluation is done (with a fixed amount of time per game), but it may not be reached for a while. And other metrics can be developed (e.g. learning speed, perhaps explored in a future post) which allow for other measures of progress to be projected, even if final scores max out, so I don't see any reason why we couldn't keep making short-term forecasts of benchmarks like this.

Based on the recent data, I think that that we might be seeing an exponential improvement in median scores. The range I gave before was agnostic regarding linear vs. exponential, and recent data points were in the ballpark of both of those lines, but only the higher one really counts since we’re interested in the highest reported score. Using the same sort of simple extrapolation I used before, I pretty strongly (80% confidence) expect that median scores will be between 300 and 500% at the end of 2017 (a range that covers linear progress and a certain speed of exponential progress), and not quite as strongly (60%) expect it to be at the higher end of this, that is, 400-500%, reflecting exponential progress before some eventual asymptote well above human performance.

For mean scores (which I didn’t make a prediction for before), here is what the most recent data (the PGQ paper submitted for ICLR) looks like when added to the graph from April.

It turns out that mean scores could have been more accurately predicted by an exponential curve—and more specifically, a faster exponential curve than I had come up with. It makes sense that mean scores would grow faster than median scores, but I’m somewhat surprised by how fast mean progress has been. I didn’t make a forecast in April, though, so I’ll rectify that now: by the end of the year, I weakly (60% confidence) expect mean scores to be between 1500% and 4000%. Obviously, that’s a pretty wide range, reflecting uncertainty about the exponent, but even at the low end, it’d be a lot higher than where we are today (877.23%).

Finally, note that these are pretty simple extrapolations and it might turn out that scores asymptote at some level before the end of the year. It seems plausible to me that you could figure out a rough upper bound based on detailed knowledge of the 57 games in question, but I haven't done this.

Conclusion re: Past Forecasts

Overall, I think my forecasts for Atari and the other domains covered (see the supplemental post for more examples of forecasts) were decent and reasonably well-calibrated, but I’m perhaps biased in my interpretation. I haven’t calculated a Brier score for my previous forecasts, but this would be an interesting exercise. Among other things, to do this, I’d have to quantify my implicit levels of confidence in earlier predictions. Perhaps I could have others assign these numbers in order to reduce bias. Since I’m giving confidence levels for my forecasts below, it will be easier to calculate the Brier score for my 2017 predictions.

Also, I think that the success of the median Atari forecast, and the plausibility that the mean forecast could have been better via e.g. error bars, suggests that there may be high marginal returns on efforts to quantify and extrapolate AI progress over the short-term.

Finally, it was a pain to find all of my old forecasts, so in the future, I’ll be putting them in blog posts or using a specific hashtag to make them more easily discoverable in the future.

Present Forecasts

Below are forecasts that I’ve either thought about a lot already, or just came up with in a few minutes for the purpose of this post. These are labeled “present” forecasts because while they’re about the future, they’re relatively weak and shoddy compared to what I or others might do in the future, e.g. with theoretically well-motivated error bars, more rigorous data collection, a wider range of tasks/domains covered, etc. I’ll say a bit about such future forecasts later, but for now I’ll just list a bunch of present forecasts.

First, I’ll just repeat what I said above about Atari.

Best median Atari score between 300 and 500% at end of 2017

Confidence level: 80%

Best mean Atari score between 1500% and 4000% at end of 2017

Confidence level: 60%

No human-level or superintelligent AI

By the end of 2017, there will still be no broadly human-level AI. No leader of a major AI lab will claim to have developed such a thing, there will be recognized deficiencies in common sense reasoning (among other things) in existing AI systems, fluent all-purpose natural language will still not have been achieved, etc.

Confidence level: 95%

Superhuman Montezuma’s Revenge AI

I don’t think this is that provocative to those who follow Atari AI super closely, versus how it may seem to those who are casual observers and have heard that Montezuma’s Revenge is hard for AIs, but I think by the end of the year, there will be algorithms that achieve significantly greater than DeepMind’s “human-level” threshold for performance on Montezuma’s Revenge (75% of a professional game tester’s score). Already, there are scores in that ballpark. By superhuman, let’s say that the score will be over 120%.

Confidence level: 70%

Superhuman Labyrinth performance

Labyrinth is another environment that DeepMind uses for AI evaluation, and which affords human-normalized performance evaluation. Already, the UNREAL agent tests at 92% median and 87% mean. So I’ll use the same metric as above for Montezuma’s Revenge (superhuman=120%) and say that both mean and median will be superhuman for the tasks DeepMind has historically used. I’m not as familiar with Labyrinth as Atari, so am not as confident in this.

Confidence level: 60%

Impressive transfer learning

Something really impressive in transfer learning will be achieved in 2017, possibly involving some of the domains above, possibly involving Universe. Sufficient measures of “really impressive” include Science or Nature papers, keynote talks at ICML or ICLR on the achievement, widespread tech media coverage, or 7 out of 10 experts (chosen by someone other than me) agreeing that it’s really impressive.

Confidence level: 70%

I also weakly predict that progressive neural networks and/or elastic weight consolidation (thanks to Jonathan Yan for suggesting the latter to me) will help with this (60%).

Speech recognition essentially solved

I think progress in speech recognition is very fast, and think that by the end of 2017, that for most recognized benchmarks (say, 7 out of 10 of those suggested by asking relevant experts), greater than human results will have been achieved. This doesn’t imply perfect speech recognition, but better than the average human, and competitive with teams of humans.

Confidence level: 60%

No defeat of AlphaGo by human

It has been announced that there will be something new happening related to AlphaGo in the future, and I’m not sure what that looks like. But I’d be surprised if anything very similar to the Seoul version of AlphaGo (that is, one trained with expert data and then self-play—as opposed to one that only uses self-play which may be harder), using similar amounts of hardware, is ever defeated in a 5 game match by a human.

Confidence level: 90%

StarCraft progress via deep learning

Early results from researchers at Alberta suggest that deep learning can help with StarCraft, though historically this hasn’t played much of if any role in StarCraft competitions. I expect this will change: in the annual StarCraft competition, I expect one of the 3 top performing bots to use deep learning in some way.

Confidence level: 60%

Professional StarCraft player beaten by AI system

I don’t know what the best metric for this is, as there are many ways such a match could occur. I’m also not that confident it will happen next year, but I think I’d be less surprised by it than some people. So partly because I think it’s plausible, and partly because it’s a more interesting prediction than some of the others here, I’ll say that it’ll happen by the end of 2018. I think it is plausible that such an achievement could happen through a combination of deep RL, recent advances in hierarchical learning, scaling up of hardware and researcher effort, and other factors soon-ish, but it's also plausible that other big, longer-term breakthroughs are needed.

Confidence level: 50%

More efficient Atari learning

I haven’t looked super closely at the data on this, but I think there’s pretty fast progress in Atari learning happening with less computational resources. See e.g. this graph of several papers hardware type-adjusted score efficiency (how many points produced per day of CPU, with GPUs counting as 5 units of CPU).

The big jump is from A3C, which learned relatively quickly using CPUs, vs. days of GPUs on earlier systems. Moreover, the UNREAL agent learns approximately 10x faster than A3C. So by the end of 2017, I’ll say that learning efficiency will be twice as good as that: an agent will be able to get A3C’s score using 5% as much data as A3C. Considering how big a jump happened with just one paper (UNREAL), this seems conservative, but as with the mean score forecast above, it’s still a big jump over what exists today so is arguably a non-trivial prediction.

Confidence level: 70%

Future Forecasts

There is a lot of room for improvement in the methodology and scale of AI forecasting.

One can use error bars based on the variance of technological progress rates and the number of data points available, as suggested by Farmer and Lafond (that paper is included in the list of resources below).

There are also many more tasks for which one could gather data and make forecasts. For example, one area that I think is worth looking at is progress in continuous control. It’s an area of real-world importance (specifically, robotics for manufacturing and service applications), and there’s a lot of data available for tasks in MuJoCo in terms of scores, data efficiency, etc.  That’s a case where further research and forecasting/subsequent evaluation of forecasts could be valuable not only for improving our knowledge of AI’s predictability, but also our early warning system for economic impacts of AI. Likewise for some NLP tasks, possibly, but I’m less familiar with the nature of those tasks.

A lot of my forecasts are about technical things rather than the social impact of AI, and the latter is also ripe for well-grounded forecasting. Right now, the people making forecasts of AI adoption are people like Forrester Research, who sell $500 reports, and aren’t transparent about methods (or at least, I don’t know how transparent they are since I can’t afford their reports). It might be useful to have better vetted, and/or crowdsourced, free alternatives to such analyses. Topics on which one could make forecasts include AI adoption, publication rates, relative performance of labs/companies/countries, dataset sizes, job displacement scales/types, etc.

The literature on AI forecasting is pretty sparse at the moment, though there are many resources to draw on (listed below). A lot of things can be improved. But in the future, besides growing this literature on its own terms, I think it’d be good for there to be stronger connections between AI forecasts and the literature on technological change in general. For example, Bloom et al. had a very interesting paper recently called “Are Ideas Getting Harder to Find?” which suggested that fast technological improvement has occurred alongside fast growth in the inputs to that improvement (researcher hours). One could ask the question of how much AI progress we’re getting for a given amount of input, how much those inputs (researchers, data, hardware, etc.) are growing, and why/under what conditions AI progress is predictable at all.

Recommended Reading:

Stuart Armstrong et al., “The errors, insights and lessons of famous AI predictions – and what they mean for the future”:  www.fhi.ox.ac.uk/wp-content/uploads/FAIC.pdf

Miles Brundage, “Modeling Progress in AI”: https://arxiv.org/abs/1512.05849

Jose Hernandez-Orallo, The Measure of All Minds: Evaluating Natural and Artificial Intelligence: https://www.amazon.com/Measure-All-Minds-Evaluating-Intelligence/dp/1107153018

Doyne Farmer and Francois Lafond, “How predictable is technological progress?”: http://www.sciencedirect.com/science/article/pii/S0048733315001699

Katja Grace and Paul Christiano et al., AI Impacts (blog on various topics related to the future of AI):  http://aiimpacts.org/

Anthony Aguirre et al., Metaculus, website for aggregating forecasts, with a growing number of AI events to be forecasted: http://www.metaculus.com/questions/#?show-welcome=true

Luke Muehlhauser, “What should we learn from past AI forecasts?”: http://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/what-should-we-learn-past-ai-forecasts

Alan Porter et al., Forecasting and Management of Technology (second edition): https://www.amazon.com/Forecasting-Management-Technology-Alan-Porter/dp/0470440902

Tom Schaul et al., “Measuring Intelligence through Games”: https://arxiv.org/abs/1109.1314

Acknowledgments: Thanks to various commenters on Twitter for suggesting different considerations for new forecasts, various people for encouraging me to keep doing AI forecasting and writing it up (sorry it took so long to make this post!), and Allan Dafoe for comments on an earlier version of this post.

Warning: less well-written than main post

Methodology for Past Forecast Review

I downloaded a CSV file with all of my tweets and searched for all tweets with the strings forecast*, predict*, extrapolat*, state of the art*, SOTA*, and expect*.  This may have missed a few predictions, and there are some forecasts that I’ve made in places other than Twitter, but this method has probably covered the vast majority of predictions, as I’m pretty tweet-prone.  

It turns out that there were a lot more than I thought (I forgot about a lot of the less rigorous ones), and the forecasts have different implicit (and sometimes explicit) confidence levels and focuses (e.g. quantifiable technical achievements vs. social adoption of/responses to AI).   

For each of the forecasts below, which are arranged in chronological order, I’ll reproduce the text of the tweet, and then say something about how it fared. I didn’t reproduce every single forecast-y tweet here because some are extremely vague or otherwise uninteresting, but here is a link to the spreadsheet on which this blog post was based if you’re interested/want to check (and my entire tweet history if you’re super skeptical about data missing from that curated spreadsheet).  

Annotated List of Forecasts 

I expected that CMU, a NASA-related team, and the Institute for Human and Machine Cognition (IHMC) would do well in the first (virtual) round of the DARPA Robotics Challenge:

This was a decent forecast (much better than chance under some interpretations of what I meant, though I was pretty vague), as IHMC got first place out of 28 teams and a JPL-related team got fifth place out of 28. This is definitely better than a random prediction. The DARPA Robotics Challenge website is no longer live, so I am having trouble verifying how CMU did in this round. I assume they weren’t in the top six based on what I later tweeted:

I had previously done an internship at IHMC and had personally seen that they were putting a lot of effort into the DRC, so I probably don’t deserve much credit for this forecast. I also didn’t put much work into making it.

Later, I doubled down on this IHMC-boosterism:

They got second place, but due to events I did not predict (SCHAFT, the winner, being bought by Google and dropping out), this retroactively improved:

Again, I don’t think I get much credit for this.

In early 2015, I said some things about DeepMind's likely work in 2015:
​

I don’t know what evidence I based this on, if any, or what counts as a “mind blowing learning thing in a 3D world,” but I think this basically happened a bit later than I expected: the A3C paper showing early impressive results in Labyrinth came out in early 2016. Fortunately, this was within my vague confidence interval:

In early 2015, I had a vague and pretty incorrect model of what DeepMind and others were trying to do with games – roughly, move forward through time/game complexity space (with newer games generally being harder for AI) and show impressive learning across a wide variety of games for that point in time/complexity space. Based on this, I said:

This model of what DeepMind and others are up to turned out to be a bit misguided, since they’re still publishing a lot of results with (old) Atari games, and making brand new environments that don’t easily map onto the metric above (since the environments have highly variable difficulty/complexity). I was wrong for thinking that they’d try to move on before having more definitively solved Atari and games that are not well captured in that metric (e.g. Go – thousands of years old, but still pretty hard). Nevertheless, if you wanted to be generous and take “DeepMind” to refer to the broader AI community, you could say that OpenAI’s Universe covers a lot of Flash games from the early 2000s, some of which deep reinforcement learning (RL) works pretty well on. But overall, I’d say this was a misguided and vague forecast.  I did caveat it a bit:

Regarding non-game stuff, I said:

DeepMind has since applied deep RL to data center energy management and deep learning to healthcare. They have also used non-game domains for benchmarks in research (e.g. MuJoCo). But this was a pretty uninteresting/banal prediction (it’s pretty obvious they would have done something non-game-related eventually).

As far as I know, this was correct, unless you count StarCraft 2 as a “current” video game.

Arguably, this was ultimately true of AlphaGo – its learning was kickstarted with a dataset of human play, though they have said they’ll explore learning from scratch in the future.

This was based on the early results from Maddison et al. (including some DeepMind authors) in late 2014 that seemed to suggest to me that they might work more on it in the future and that deep learning could help a lot.

IHMC got second (same number of points as the winner, KAIST, but with a slower time) and the Chinese team did poorly.

Regarding speech recognition, in late 2015, i said:

There wasn’t a clear metric for this. There was a lot of coverage of speech recognition in the tech press, and some impressive (nearly) human-level results, but I’m not sure whether 2016 represented any sort of shift in terms of wide adoption. Anecdotally, it seems more widely used in Beijing than in Western countries, but I don’t know for sure.

As part of a longer rant in 2016, I said:

And:

This was pretty vague, and the timeline in question is still ongoing, so I can’t evaluate it yet.

Regarding hardware and neural network training speeds, I said:

(sorry for the bad formatting here)

I’m still pretty confident that hardware is speeding up and will speed up neural net training a lot, but we’ll have to wait until early 2019 to evaluate the 100x thing. I’ll try to specify it a bit better now: for a multiple expert-suggested set of 10 benchmarks in image recognition and NLP, you will be able to achieve the same the same performance, using new hardware and algorithms, in 100x less training time (wall time) vs. results reported in early 2016 on at least 8 of those benchmarks. This is a rough, intuitive guess, so I have less confidence in it than some of my more quantitative extrapolations of Atari results discussed below.

Regarding AlphaGo’s success against Lee Sedol, I said in the middle of the match:

This reference to a prior prediction was based on a Facebook comment I made before the match began, which in turn elaborated on views expressed in a blog post I wrote. The comment (not publicly linkable, unfortunately), on March 1, said:

 I think my reasoning at the time was essentially correct, deep RL is in fact very effective and scalable for well-defined zero-sum games, and that my conclusions in the aforementioned blog post (about the importance of hardware in this case and humans occupying a small band in the space of possible intelligence levels) are still correct. But lots of people thought AlphaGo would win, and I wasn’t extremely confident, so don’t get much credit for this.

Regarding dialogue systems:

I still believe this, but “a few years” haven’t yet passed so don’t have much more to say about this right now, other than that it is probably too vague.

Regarding Google’s business model for AI:

Again, it’s early for this, but this seems pretty plausible to me.

Atari Forecasts

See main blog post.

[Note: these comments assume some familiarity with OpenAI and the motivations behind it – see e.g. their website, this article, and this detailed interview].
 
The announcement of OpenAI has justifiably gotten a lot of attention, both in the media and in the relevant expert community. I am currently flying home from NIPS 2015, a major machine learning conference, and right after OpenAI was announced, I noticed several people around me checking out the website on their phones and computers to learn more, and I expect that OpenAI’s information/recruiting session at NIPS later today will be extremely popular. Here, I will summarize some preliminary, sleep-deprived thoughts on why this is getting so much attention and why that is appropriate.
 
The AI Arms Race
 
Over the past few years, many billions of dollars have been poured into AI and robotics research and development. OpenAI is in fact only the second of two billion dollar research centers announced very recently (the other being Toyota’s). Stuart Russell, lead author of the main textbook in the field, has claimed that more money has been invested by corporations in AI in the past few years than was invested by governments in the field’s entire prior history. This is largely a result, as pointed out in the open letter on AI signed by thousands of researchers and other interested parties, of AI methods (especially but not limited to deep learning) having just recently reached levels of performance that make them commercially lucrative, in turn leading to more funding to reach higher levels of performance, and so forth.
 
This has led to an extremely competitive race between technology companies to snatch up talent in especially “hot” sub-fields of AI. Walking around at NIPS, one would see badges listing the names of not just stereotypical “tech” companies like Google and Facebook but also miscellaneous hedge funds and other large companies seeking to cash in on the AI bonanza. Tellingly, NIPS, I saw a flyer for an academic job that started with “If you’re still interested in academic jobs…” Leaving aside the question of whether all this investment constitutes a bubble (I don’t think so), superficially, at least, things are extremely “hot” in areas like deep learning – for example, at NIPS, the workshop on deep reinforcement learning (a combination of deep learning and reinforcement learning that has led to impressive results in various domains recently) was so popular that people were sitting on the floor, standing along the walls, and getting shut out entirely for fire hazard concerns (including Rich Sutton, author of “the book” on reinforcement learning. I gather he eventually got in, though).
 
This arms race is not just a matter of accelerated deployment of AI technologies, or massive spending by tech giants, though that is happening—it’s also a battle for the hearts and minds of AI researchers, the most accomplished of which are being sought after like star athletes. Already, several years ago, AI researchers at the likes of Google were making plenty of money for a reasonable person’s purposes. Now, with talk of six or more figure salaries being offered to researchers, we may be at a point of diminishing returns for fat paychecks—many researchers also would ideally like the ability to publish (and be able to talk openly about) all their research and to work towards a mission (such as OpenAI’s, to benefit humanity through AI) that resonates with them. In this competitive context, OpenAI is stepping in with a third alternative beyond partially-secretive industry and often bureaucratic and under-resourced academia. It also comes at a critical time in discussions of the future of AI and the prospects for benefiting all of humanity through it. Speaking of arms races…

AI Safety and Ethics
 
In recent years, concerns about the short- and long-term implications of AI, and safety concerns in particular, have gone from being a fringe topic mostly researched outside the AI community to a mainstream topic of discussion. On Thursday at NIPS, there was a major symposium on the social impacts of machine learning attended by hundreds of researchers. Speakers such as Nick Bostrom (whose book Superintelligence helped catalyze the particular form of current concerns around the future of AI) and Erik Brynjolfsson (whose co-authored book The Second Machine Age helped spark discussions on the impact of AI on the economy) and panelists from industry and academia discussed short and long term issues with AI and its social implications, as well as what the AI community can do about it.
 
While a range of opinions was expressed on the urgency of different issues, there was no dispute in the symposium about the enormous consequences that AI is likely to have. Even Andrew Ng, a top deep learning researcher who leads AI research for Chinese search engine Baidu and who has compared concerns about existential risk from AI (of the sort Bostrom writes about, and Musk has echoed) to concerns about overpopulation on Mars, backpedaled his comments to some extent when he said that he thinks it makes sense for at least some people to be working on understanding potential long-term AI safety risks. The last panel of the symposium went into some detail about research priorities in that area, an area that Musk has previously invested $10 million in through the Future of Life Institute. There is certainly still much disagreement on the topic of long-term AI risks, as evinced by the diverse audience reactions at the symposium, some clapping at Ng downplaying long-term risks, and others responding favorably to DeepMind’s Shane Legg comparing AI safety research to engineers on the Apollo Project investigating ways that the mission could go wrong as a matter of common sense and simple responsibility. However, there has been a noticeable change in the tenor and number of people involved in these discussions in recent years.
 
In this context, OpenAI represents not only a big investment in AI safety (since this is among the areas Musk et al. have indicated the institution will study). It is also a sign of the mainstreaming of the issue, given the talent and prestige of those affiliated with the new organization, including Ilya Sutskever of Google deep learning fame. Along with safety research in particular, the funders and chairs of OpenAI have indicated that the organization will be concerned with shorter term issues such as the economic implications of AI, the need for AI to be designed in a way that is usable and complementary to human skills, and, of course, as the name implies, openness. This latter factor is particularly notable, not just for recruiting purposes as discussed above, but also because it represents a significant decision on the part of Musk and the funders. As Musk’s recommended book Superintelligence attests, diffusing AI technologies as widely as possible is not straightforwardly preferable over scenarios in which, say, it is more concentrated in the hands of governments or corporations with particular sets of safety constraints. Indeed, Musk’s latest comments (for example, in the Medium interview linked to above) suggest a shift in his thinking from when he compared AI to nuclear weapons and implied favoring a less open approach to AI safety. It will be interesting to see how the growing AI safety research community reacts to this move and to the specific arguments put forth by Musk et al., and to see how OpenAI will interpret the noteworthy last word in the statement of their mission on their website of  ensuring that AI technology is “as broadly and evenly distributed as possible safely” (emphasis added).  Thus, OpenAI has a lot of symbolic significance, but how will its mission be realized, if it will be?
 
Institutional Innovation
 
It’s not yet clear how exactly OpenAI will operate, but its status as a non-profit is of particular significance. This means that there will be no pesky shareholders questioning the return on investment of their research, or the particular balance struck between emphasis on AI performance progress and safety (a delicate issue again dealt with at some length in Superintelligence), in the event that OpenAI begins to focus more on the latter. Additionally, standing outside of traditional corporations will give OpenAI the ability to think more outside the box about potential applications of its AI approaches, instead of working around existing value streams as, e.g. Google and Facebook do to an extent. This, among other factors, will likely lead to a big influx of talented researchers interested in applying their talent to grand challenges, potentially leading OpenAI to grow to hundreds of people (or, someday, even more). It will also be interesting to see how OpenAI differentiates itself from a comparable (at least in stated ambition if not, yet, committed funds) organization, the Allen Institute for Artificial Intelligence, a non-profit funded by Paul Allen, which seems to have focused so far on applying AI to improving scientific productivity and on particular methods like natural language processing. I hope that OpenAI will differentiate itself by tackling truly difficult challenges like applying AI to, e.g. addressing poverty, mental health, or sustainability, or if they focus more on research, that they give very deep thought to what human-complementary AI means and what sorts of research won’t otherwise be funded by industry. 
 
Scale is one of the ways that OpenAI may be distinct from academia, though it is not obvious all of the ways in which it intended to be different. Mark Riedl noted on Twitter that academia has many of the characteristics OpenAI is purported to have—though, I would add, academic research too often fails to have maximum positive impact and is too often bogged down in grant applications and other paperwork. Relative to industry, I’d expect OpenAI salaries to be on the order of 50-90% or more lower than what some of the best researchers could make, so the organization will need to stay true to its stated principles if it will continue recruiting successfully over the long term. To continue to recruit the best talent, I suspect, as was the case with another instance of institutional innovation I’m familiar with (ARPA-E), that the top leadership of OpenAI will want to not only pore through the hundreds or thousands of applications they will soon receive, but also aggressively recruit top talent in the field and bring in the likes of Musk to help with such recruitment.
 
The role of data at OpenAI also raises interesting questions from an institutional perspective, as Beau Cronin and others have noted on Twitter. With Amazon’s investment and Tesla’s indirect involvement through Musk, it would seem that OpenAI will potentially have access to a lot of the data needed to make deep learning and other AI approaches work well. But these are different sorts of datasets than Google and Facebook have, and may lend themselves to different technical approaches. They also raise the question of proprietary data – how will OpenAI balance its push for openness with this proprietary access? Will it release code but not data? How will its experiments be replicable if they are tailored to particular data streams? How will privacy be addressed? Some of these questions are unique to OpenAI and others aren’t. However, they’re the sorts of questions OpenAI will need to answer.
 
Conclusion/Open Questions
 
As I and others have written elsewhere, there are many open questions on the future of AI in general and how to innovate responsibly in this area, AI and the future of work, how AI progress may proceed, etc. and these precede OpenAI. The launch of OpenAI, however, makes them particularly concrete. Here I suggested some preliminary formulations of these questions in the context of a particular organization that ostensibly will address many of them. I look forward to thoughts people have on whether I formulated these issues correctly, how OpenAI could realize its goals, and what ripple effects its launch could have on the broader AI arms race/safety/ethics conversation.