Source: theverge.com
Trying to get a handle on the progress of artificial intelligence is a daunting task, even for those enmeshed in the AI community. But the latest edition of the AI Index report — an annual rundown of machine learning data points now in its third year — does a good job confirming what you probably already suspected: the AI world is booming in a range of metrics covering research, education, and technical achievements.
The AI Index covers a lot of ground — so much so that its creators, which include institutions like Harvard, Stanford, and OpenAI, have also released two new tools just to sift through the information they sourced from. One tool is for searching AI research papers and the other is for investigating country-level data on research and investment.
Most of the 2019 report basically confirms the continuation of trends we’ve highlighted in previous years. But to save you from having to trudge through its 290 pages, here are some of the more interesting and pertinent points:
- AI research is rocketing. Between 1998 and 2018, there’s been a 300 percent increase in the publication of peer-reviewed papers on AI. Attendance at conferences has also surged; the biggest, NeurIPS, is expecting 13,500 attendees this year, up 800 percent from 2012.
- AI education is equally popular. Enrollment in machine learning courses in universities and online continues to rise. Numbers are hard to summarize, but one good indicator is that AI is now the most popular specialization for computer science graduates in North America. Over 21 percent of CS PhDs choose to specialize in AI, which is more than double the second-most popular discipline: security / information assurance.
- The US is still the global leader in AI by most metrics. Although China publishes more AI papers than any other nation, work produced in the US has a greater impact, with US authors cited 40 percent more than the global average. The US also puts the most money into private AI investment (a shade under $12 billion compared to China in second place globally with $6.8 billion) and files many more AI patents than any other country (with three times more than the number two nation, Japan).
- AI algorithms are becoming faster and cheaper to train. Research means nothing unless it’s accessible, so this data point is particularly welcome. The AI Index team noted that the time needed to train a machine vision algorithm on a popular dataset (ImageNet) fell from around three hours in October 2017 to just 88 seconds in July 2019. Costs also fell, from thousands of dollars to double-digit figures.
- Self-driving cars received more private investment than any AI field. Just under 10 percent of global private investment went into autonomous vehicles, around $7.7 billion. That was followed by medical research and facial recognition (both attracting $4.7 billion), while the fastest-growing industrial AI fields were less flashy: robot process automation ($1 billion investment in 2018) and supply chain management (over $500 million).
All this is impressive, but one big caveat applies: no matter how fast AI improves, it’s never going to match the achievements accorded to it by pop culture and hyped headlines. This may seem pedantic or even obvious, but it’s worth remembering that, while the world of artificial intelligence is booming, AI itself is still limited in some important ways.
The best demonstration of this comes from a timeline of “human-level performance milestones” featured in the AI Index report; a history of moments when AI has matched or surpassed human-level expertise.
The timeline starts in the 1990s when programs first beat humans at checkers and chess, and accelerates with the recent machine learning boom, listing video games and board games where AI has came, saw, and conquered (Go in 2016, Dota 2 in 2018, etc.). This is mixed with miscellaneous tasks like human-level classification of skin cancer images in 2017 and in Chinese to English translation in 2018. (Many experts would take issue with that last achievement being included at all, and note that AI translation is still way behind humans.)
And while this list is impressive, it shouldn’t lead you to believe that AI superintelligence is nigh.
For a start, the majority of these milestones come from defeating humans in video games and board games — domains that, because of their clear rules and easy simulation, are particularly amenable to AI training. Such training usually relies on AI agents sinking many lifetimes’ worth of work into a single game, training hundreds of years in a solar day: a fact that highlights how quickly humans learn compared to computers.
Similarly, each achievements was set in a single domain. With very few exceptions, AI systems trained at one task can’t transfer what they’ve learned to another. A superhuman StarCraft II bot would lose to a five-year-old playing chess. And while an AI might be able to spot breast cancer tumors as accurately as an oncologist, it can’t do the same for lung cancer (let alone write a prescription or deliver a diagnosis). In other words: AI systems are single-use tools, not flexible intelligences that are stand-ins for humans.
But — and yes, there’s another but — that doesn’t mean AI isn’t incredibly useful. As this report shows, despite the limitations of machine learning, it continues to accelerate in terms of funding, interest, and technical achievements.
When thinking about AI limitations and promises, it’s good to remember the words of machine learning pioneer Andrew Ng: “If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.” We’re just beginning to find out what happens when those seconds are added up.
