For all the promise and peril of artificial intelligence (AI), there’s one big obstacle to its seemingly relentless march: The algorithms for running AI applications have been so big and complex that they’ve required processing on powerful machines in the cloud and data centers, making a wide swath of applications less useful on smartphones and other edge devices.
Now, that concern is quickly melting away, thanks to a series of breakthroughs in recent months in software, hardware, and energy technologies that are rapidly coming to market.
That’s likely to drive AI-driven products and services even further away from a dependence on powerful cloud-computing services and enable them to move into every part of our lives—even inside our bodies. In turn, that could finally usher in what the consulting firm Deloitte late last year called “pervasive intelligence,” shaking up industries in coming years as AI services become ubiquitous.
By 2022, 80% of smartphones shipped will have AI capabilities on the device itself, up from 10% in 2017, according to market researcher Gartner. And, by 2023, that will add up to some 1.2 billion shipments of devices with on-device AI computing capabilities, up from 79 million in 2017, according to ABI Research.
A lot of startups and their backers smell a big opportunity. According to Jeff Bier, founder of the Embedded Vision Alliance, which held a recent conference in Silicon Valley, investors have plowed some $1.5 billion into new AI chip startups in the past 3 years—more than was invested in all chip startups in the previous 3 years. Market researcher Yole Développement forecasts that AI application processors will enjoy a 46% compound annual growth rate through 2023, when nearly all smartphones will have them, from fewer than 20% today.
And it’s not just startups, as a series of announcements at the Computex computer conference in Taipei will reveal. On 26 May, Intel previewed its coming Ice Lake chips, which among other things have “Deep Learning Boost” software and other new AI instructions on a graphics processing unit. The company Arm also unveiled a series of processors aimed at AI applications, including for smartphones and other edge devices. A few hours later, Nvidia announced its first AI platform for AI devices.
“Within the next 2 years, virtually every processor vendor will be offering some kind of competitive platform for AI,” Tom Hackett, principal analyst at IHS Markit, said at the alliance’s Embedded Vision Summit. “We are now seeing a next-generation opportunity.”
Those chips are finding their way into many more devices beyond smartphones. They’re also being used in millions of “Internet of things” devices such as robots, drones, cars, cameras, and wearables. Among the 75 or so companies developing machine-learning chips, for instance, is Israel’s Hailo, which raised a $21 million funding round in January. In mid-May it released a processor that’s tuned for deep learning, a branch of machine learning responsible for recent breakthroughs in voice and image recognition.
More compact and capable software that could pave the way for more AI at the edge is coming, judging from new research indicating the size of neural networks could be reduced tenfold and still achieve similar results. Already, some companies are managing to compress the size of software needed for AI.
Google, for instance, debuted its TensorFlow Lite machine-learning library for mobile devices in late 2017, enabling the potential for smart cameras to can identify wildlife or imaging devices to can make medical diagnoses even where there’s no Internet connection. Some 2 billion mobile devices now have TensorFlow Lite deployed on them, Google staff research engineer Pete Warden said at a keynote presentation at the Embedded Vision Summit.