← Toutes les offres
MS

The Coziness Problem

Morgan S

Global
Publié le 20 novembre 2026

Description du poste

To spell out this vibe-comic[1]: Safety made the locally correct decision here at every step of a bad situation. But like in Caleb Biddulph's parable, this globally resulted in an awkward state of affairs. From the Public's POV (penultimate panel), they would naively see AI safety as complicit in the building of the machine. If it really is a doomsday device, why would Safety involve itself at all? (Safety in turn has a good answer: Public wasn't willing to listen back in the 2010s. Ordinary people would nod along but not lobby their politicians, and politicians would not take it seriously, because most AI experts wouldn't either.) The realest case of a doomsday device was probably Project Sundial, a bomb so big as to destroy the Earth in one go. When Edward Teller proposed it in backroom briefings, people thought it was an "advertising stunt." The prior on gadgets truly being doomsday devices is very low. Hence, when even Safety shows some signs of not taking it seriously, the Public doesn't either. In the final panel, we have Safety looking at the other two characters. In the past, they would be timid about shouting too much to Public, due to Company's implicit threat of bad relations. Company has become more powerful than they were in the 2010s. But in comparison to the now half-sleeping giant, not so much anymore. I also spent some time researching historical precedents for groups becoming socially cozy with entities they're advocating against: • Many of the Manhattan project scientists advocated for treaties, but after they'd already built the A-bomb. Some like Oppenheimer advocated against the H-bomb, and this did not go well. But before the A-bomb most outsiders considered the idea ludicrous (e.g. "That is the biggest fool thing we have ever done. The bomb will never go off, and I speak as an expert in explosives.").• Recombinant DNA technology was stopped at Asilomar in 1975, but this was internal to the researchers themselves, without having to involve Publ