Superintelligence: Paths, Dangers, Strategies

“The possibility of us developing human-level artificial intelligence, and from there superintelligence—an artificial agent that is considerably more intelligent than we are—is at least a contender for the most important issue in the next two centuries.” Read more...

The best books on Effective Altruism

Will MacAskill, Nonprofit Leaders & Activist

“He is a philosophy professor at Oxford University and runs the Oxford Martin School’s Future of Humanity Institute. He’s said, for a long time, that Kurzweil is half right. If we get AGI, the outcome could be absolutely wonderful. But it could also be terrible. He warns about the possibility not so much of a superintelligence going rogue—like Skynet, or HAL in 2001—but more simply of an immensely powerful entity that would not set out to damage us but have goals that could do us harm…He uses what he calls a ‘cartoon’ example: The first AGI turns out to be developed by someone who owns a paperclip manufacturing company. The AI has the goal of maximizing the production of paperclips. After a little while, it realizes, ‘Well these humans, they’re made of atoms, they could be turned into paperclips.’ So it turns us all into paperclips. Then it turns its gaze towards the stars and thinks, ‘Well there’s an awful lot of planets out there and I can turn all those into paperclips!’ So it develops a space program and travels around the cosmos and turns the entire universe either into paperclips or something that makes paperclips. It’s an absurd idea, but it shows the possibility of inadvertent damage. An AI doesn’t have to hate humans in the way Hollywood often shows them disliking us. It can just have goals that do us damage.” Read more...

The best books on Artificial Intelligence

Calum Chace, Novelist