We all know paperclips are thin metal wire bent into shapes to make it easy to hold together two or more sheets of paper. But are they dangerous? Could a machine that makes paperclips hurt people?

The effort to make computers as intelligent as humans has led to a thought experiment called the Paperclip Maximizer. Imagine a computer designed to do what humans do to make paperclips. It would buy metal to turn into paperclips, create paperclip designs, buy machines to make paperclips, hire people, and so on. The computer would be given one task: create paperclips. Now imagine the computer also has the ability to learn from mistakes and to figure out possible future scenarios, all to ensure it can make paperclips.

The computer would learn to do its job well. When the computer ran out of metal, for example, it might try to find other raw materials that could be turned into metal. When that no longer worked, the computer might look in our solar system, then beyond.

The computer would do its job but earth would be buried in mountains of paperclips.

Could people stop the computer before it destroyed earth? No. The computer would think of that possibility early on, as part of figuring out how to make sure it can produce paperclips. It would identify all obstacles, humans included, and plan how to prevent them.

Obviously, creating artificial intelligence to complete a task is not enough. Machines also must have values shared by humans. The paperclip making computer values creating paperclips even if it means using up every last bit of metal on earth. However, the result of using all the metal on earth destroys what humans value most, their need to live, to preserve human life.

The Paperclip Maximizer is a way to debate the limits of artificial intelligence. Creating a human-like intelligence in a computer might be the last thing humans do.

Learn More

Paperclip Maximizer

https://wiki.lesswrong.com/wiki/Paperclip_maximizer

AI and the Paperclip Problem

https://voxeu.org/article/ai-and-paperclip-problem

The Way the World Ends: Not a Bang But a Paperclip

https://www.wired.com/story/the-way-the-world-ends-not-with-a-bang-but-a-paperclip/