← Toutes les offres
AD

CLR's 2027 SPI agenda plans

Anthony DiGiovanni

Global
Publié le 21 novembre 2026

Description du poste

These are my plans for work next year on the safe Pareto improvements (SPIs) agenda, which I lead at the Center on Long-Term Risk. (Thanks to Tristan Cook for helpful comments and suggestions.) Theory of change To recap the theory of change from our April 2026 agenda post (“more” = relevant sections of that post): • We’d like for (1) the transformative AIs that we can influence to implement SPIs, and (2) the agents they bargain with to be willing to participate in SPIs with them. • Most likely, the transformative AIs themselves will be in a better position to implement SPIs than we are, due to (e.g.) better commitment ability. But: • (a) (On (1):) Early decision-makers might make commitments (broadly construed [1] ) that lock out the option to implement SPIs later (more; more). • (b) (On (2):) Whether agents are incentivized to agree to SPIs might be path-dependent; e.g., maybe one needs to adopt a surrogate goal before any other bargaining commitments (more). • (c) Both the above risks are exacerbated if frontier AI companies aren’t paying much attention to SPIs generally (more), and if AIs aren’t differentially capable at reasoning about SPIs (more). • → We should prioritize: • evals on (a); • conceptual research on (b); and • a pitch to trusted contacts at AI companies, and research automation prep, to address (c). I mostly still endorse this picture, but with some updates (“more” = relevant sections of this post): • We’ve (re-)discovered several important challenges to the individual rationality of SPIs, and potential backfire risks of SPI implementations. (See, e.g., discussion here and here. I’ll say more on this point in a forthcoming list of open problems in the SPI agenda.) • → Upweight mitigations of (a) (more), and downweight basic deconfusion research on (b) (examples). The former seem decently robust to our uncertainties about individual incentives, while it seems less likely the latter leads to actionable insights. • → Upweight f