Description du poste
This post reflects my own takes, and not that of my employer's.
This is a simple idea: before a new model can be deployed internally, you must first test it for a minimum, fixed period.
You can adjust various axes of this proposal:
• Duration of the wait: in calendar time, or until you fulfill some condition.• What it blocks: external deployment, internal deployment, or the training of future checkpoints.• When the fixed period starts: from the first checkpoint, the final checkpoint, or any checkpoint that is “sufficiently similar / representative” to the one that is ultimately deployed.One concrete target that I like is: “No checkpoint may be used internally for purposes beyond safety testing, until at least X calendar days after its hash is filed with an auditor.”
This is one specific implementation of internal deployment lags, an idea that the AI Futures Project (AIFP) briefly discusses in How to pace the US frontier as “AI R&D can only be conducted or assisted by AIs that were trained at least 9 months ago”.
What does this look like when implemented?Let’s run through a scenario to pin down the details of this proposal. The places where it could go wrong are in the open questions at the end.
• A legislative body sets a minimum waiting period of X calendar days (starting at 10 and possibly ratcheting up to 90). No checkpoint in a covered model family may be used internally, beyond safety testing, until X calendar days after its hash is filed with an auditor. Continuing to train a checkpoint’s own weights does not count as internal use.• The alternative is for the wait to also block the training of checkpoint N+1, and not just the internal use of checkpoint N. I didn’t go with this because it incentivizes developers to pull out fewer checkpoints for testing, and because delaying training results in an algorithmic overhang.• More capable models start exhibiting problems that are harder to surface in short periods (e.g. swarms, long-horizon, memory), so X should pos