On May 6, 2010, at 2:32 p.m. Eastern time, nearly a trillion dollars evaporated from the US stock market within 20 minutes—at the time, the fastest decline in history. Then, almost as suddenly, the market rebounded.
After months of investigation, regulators attributed much of the responsibility for this “flash crash” to high-frequency trading algorithms, which use their superior speed to exploit moneymaking opportunities in markets. While these systems didn’t spark the crash, they acted as a potent accelerant: When prices began to fall, they quickly began to sell assets. Prices then fell even faster, the automated traders sold even more, and the crash snowballed.
The flash crash is probably the most well-known example of the dangers raised by agents—automated systems that have the power to take actions in the real world, without human oversight. That power is the source of their value; the agents that supercharged the flash crash, for example, could trade far faster than any human. But it’s also why they can cause so much mischief. “The great paradox of agents is that the very thing that makes them useful—that they’re able to accomplish a range of tasks—involves giving away control,” says Iason Gabriel, a senior staff research scientist at Google DeepMind who focuses on AI ethics.
“If we continue on the current path … we are basically playing Russian roulette with humanity.”
Yoshua Bengio, professor of computer science, University of Montreal
Agents are already everywhere—and have been for many decades. Your thermostat is an agent: It automatically turns the heater on or off to keep your house at a specific temperature. So are antivirus software and Roombas. Like high-frequency traders, which are programmed to buy or sell in response to market conditions, these agents are all