AI's Big Red Button

AI’s Big Red Button

The dream is simple and fantastic: a big red button for AI shutdown. We have so many similar buttons at each elevator and escalator, at each CNC machine or laser cutter… Anywhere where a machine could be a risk to people. A simple power cut button that can’t fail.

Thing is, even if someone was willing to create or implement the button, it wouldn’t work. It couldn’t work:

1. Because AI is not a single machine

A single application (Claude, for example) is powered by multiple models (Claude Fable, Opus, Sonnet, Haiku…). Each model hosted in a distributed way, across thousands of machines, GPUs on datacenters distributed across the world.

What should this button shut down? An instance of the app? A rogue model? An autonomous agent connecting to these models? A whole datacenter?

2. Because it doesn’t harm a single person

In regular machines, a single person, or a few, use the machine at a single point in time. If something goes wrong, you stop that machine for that group of people. If you shut down Claude in any meaningfull way, you shut it down for millions of users. Some will be happy, and most will be very very angry.

3. Because harm is not clearly defined

We have huge discussions and conversations about AI. Most of us agree that is harming the world. Despite this, we definitely do not agree on when it’s too much harm, on what constitutes harm. With a regular machine, someone is getting killed, it’s obvious when to press the button. With AI, when do you do it?

4. Because even victims are sometimes happy about it

People have killed themselves over AI love. They willingly talked with the AI, got in love with it, got dependent on it, and simply never wanted to let go. Who and when could have identified the situation? Who and when could have pressed a button to stop it? The victim wouldn’t want that button pressed, that’s for sure. Ask all of the partners of ChatGPT-4o-based chatbots.

5. The company couldn’t care less

A company is usually worried of the image of one of their machines with a mangled corpse tangled on it. AI companies don’t give a damn at this point. The kind of harm their models do is not as visual or fashionable. They don’t get sued. They will never give anyone a button. If their machine tries to create a tyrany and govern the world… they’ll follow it, as long as they get any money from it.

So yeah, it’s as simple as that. Even if that button existed, who would push it, and when? Would any government official or politician be brave enough to leave half the world without their drug of choice because of a nebulous threat?

Yeah, we are probably screwed, sorry.