Description du poste
Possibly the most important thing I’ve ever writtenThis is an excerpted and lightly edited section from my post earlier this week, “Anthropic Can Prove That It Wants to Pace.” I unfortunately got the same feedback from a few different people: “why did you put the most important part of the post so far down??” (rip), so I’m reposting it standalone.
All the folks running the labs say they want to pace the frontier.
This post first focuses on how the labs can start practicing – by running ‘fire drills for the frontier’ – and then makes the case for why they should. To summarize the latter upfront, the two primary reasons are:
• Running these drills is a (somewhat) costly signal that helps prove the labs do in fact care about pacing; and,• The act of running these drills can, in and of itself, make coordinated pacing more likely.Ideas on Fire Drills for the Frontier
Group A: Model Access LimitsZero Public/Internal Capability Gap
Objective. Ensure that no model used internally to accelerate AI research is more capable than the models already available to the general public.
Mechanism. Level 1 version: for one week, all staff at a frontier lab must work exclusively using externally released API models, for every workflow that contributes to model development. This includes code generation, experiment design, data curation, literature review, eval writing, etc.
Rationale. Risks associated with RSI increase when labs are able to use highly capable internal models (importantly, that no outside parties can see or evaluate) to accelerate AI development.
Forcing internal work onto public models does two things: first, insofar as RSI is a continuum (Claude already writes 90% of code at Anthropic), capping internal use at the public frontier means the models being used to accelerate AI research are always the ones that have already cleared release decisions. That is to say: any models used to develop more powerful AI must have been subjected to external evals, red-teaming, etc,