Description du poste
In favor of costly signals to pace the frontierNote: I think this might be one of the most important things I've ever written.
The most insane power sibling duo ever. Image source: FortuneThere are, broadly speaking, two camps in AI safety (with respect to how people view Anthropic):
• Camp 1 believes that all the frontier labs are ~equally bad, insofar as they’re all “racing straight to self-improving superintelligence and gambling with our lives.”• In addition to Jacob Coxon, some notable players here might include folks like Holly Elmore from PauseAI and Nathan Calvin from Encode.[1]• Camp 2 doesn’t necessarily claim Anthropic is safe. Instead, it claims Anthropic is distinguishable – that its record contains costly, verifiable choices no other frontier lab has made (see a roundup from Dario Amodei here), and that Anthropic being the ‘least bad’ of the labs counts for something.• A lot of Anthropic employees – many of my friends included! – fall into Camp 2, though it’s harder for me to point to specific ‘notable players,’ especially after the recent AEGIS bill controversy (which I’ll discuss more in-depth below).• Perhaps one example: While I definitely cannot say for certain which camp Oscar Delaney falls into, he’s written a post on the merits of Claude’s Constitution, which I think captures some of the sentiment of Camp 2-ers.[2]• Worth noting: Following the events of the Department of War confrontation, Claude became the #1 downloaded app on the App Store. I’d wager that a lot of the general public falls closer to Camp 2 than Camp 1 for this reason, though the ‘camps’ distinction here mainly intends to focus on folks who are high-context on AI safety.One crux of these two camps is the degree to which you believe alignment is tractable. Another crux is the degree to which you trust the folks at Anthropic to do the right thing, and the weight you place on the actions they’ve taken (e.g. supporting SB53, implementing bio safeguards, breaking with other folks in