Comments on ‘Robots could murder us out of kindness’

CHEERLEADING FOR BETTER OUTCOMES

I was recently asked to discuss how machines can better understand humans at the Media Evolution Conference in Malmö, Sweden. The conference collected a broad cross-section of the technology and media world.

A few weeks before, I had attended The Effective Altruism Summit in Berkeley (and a related retreat I had been invited to give a talk at, held around the same time). There was much discussion of existential risk, particularly risk from AGIs (Artificial General Intelligences), specifically on the threat from ‘Unfriendly AI’.

I have long been intrigued by the question of how to manage the emergence of advanced machine intelligences, and the extreme risks and rewards that come with them. A cluster of organisations dedicated to mitigating existential risk were represented at the Summit, including The Machine Intelligence Research Institute, The Centre for the Study of Existential Risk, The Future of Humanity Institute, and The Future of Life Institute. Between them, there was a blend of fascinating perspectives on the best way to manage humanity’s road ahead.

Returning to Europe inspired, I decided to close my next talk with a brief discussion of existential AI risk, since the topic was Human-Machine interactions. Below is the talk itself:

I didn’t mention ‘robot murder’ per se -- simply that even a truly benevolent AGI could plausibly conclude it would be ethical to end civilisation as we know it. I never expected my heartfelt soundbites to be picked up the way they were — by Wired, Daily Mail, The Independent, CNET, and others. 

It is societally useful to spread awareness of the need for Friendly AI research, especially since there are perhaps 40 serious AI researchers in the world, and only half a dozen have committed themselves to working on Friendly AI.

Machine Intelligence is in many ways humanity’s greatest gambit: tremendous risk, but also potentially incalculable reward.

A super-intelligent Artificial General Intelligence truly friendly to our best interests could be a digital Second Coming. An Unfriendly AGI (or AI swarm) could be like the devil incarnate. It’s very difficult to tell what we’re getting when it comes out of the Box.

But one must balance awareness of risk against the danger of spreading unnecessary fear of science itself. Caution is helpful; panic is not.

I’ve received a lot of mail and comments, which I would like to address in subsequent posts.

Concerned about developments within Machine Intelligence and would like to learn more? You may find the following organisations of interest:

• The Machine Intelligence Research Institute

• The Centre for the Study of Existential Risk

• The Future of Humanity Institute

• The Future of Life Institute

• The Lifeboat Foundation

• OpenCog

• OpenAI