← Toutes les offres
JB

What do kill switches buy us?

Jacob Brinton

Global
Publié le 17 novembre 2026

Description du poste

SummaryInstalling kill switches in every US datacenter could help avert some catastrophes, and might even reduce extinction risk a little, but mainly through indirect effects. The legibility of this means it's an early choice for politicians to be seen doing something about AI risk, and could be worth advocacy energy if implementation could build into further verification/transparency efforts. That said, having kill switches in place might lower the urgency to get more robust plans to lower takeover risk and lower the chances of warning shots. Threat modelThere are three broad routes to AI going poorly: • Loss of control• Extreme power concentration• Societal destabilization leading to either of the aboveKill switches are a candidate to reduce the risk of #3, and might have a small role reducing the risk of #1. Takeover from reckless AIsIt might be the case that the first AIs that want to accrue resources and are capable enough to do so will not have enough strategic taste to know to lay back until they are smart enough to decisively overpower humanity. In this case, having hardwired kill switches could help prevent lots of these reckless AIs from causing havoc and setting up agent civilizations on the internet. In order to avoid this, the following series of events need to happen: • Monitorability can detect cases where an agent or swarm of agents is dangerous enough that we should shut down their compute.• The latency between detection and shutdown is low. If humans are involved in the decision, they can act quickly enough to shut down the instances. The AIs are not self sufficient outside the original compute surface they were on.• OpenAI’s Control Room can halt research-model output within 35 seconds of a decision to stop a job.• As an illustrative scenario, suppose a model has 10 trillion parameters and the network is fast. At 16 bits per parameter, the model weights are about 20 TB. At 400 Gb/s, transferring those weights would take about 400 seconds, or 6.7 m