Near Horizons

Image by Claude Opus 5.5 using Python (NumPy and Pillow) This is written in my free time and personal capacity. Two FramesIn AI safety, there are a couple of tensions at the heart of the field, which are interrelated.[1] Talent: we are talent-constrained and must recruit. But jobs and fellowships turn away nearly everyone.[2] Field size: we must expand the field and bring more people in. But the field is flooded with slop papers in conferences. And, most papers have a small audience, and aim to shift the practices of roughly five labs (and usually some subteam within them).[3]  Status: we must avoid seeking status, as it disorients, corrupts. But, we must make the field legible and attractive to outsiders. Raising the bots: we must deceive the AIs to control them and evaluate them without confounds, obscuring what they know about their own situation. But, we hope to gain their trust, must rely on them to do good work at pace, shape their character and retain good relations. Speed: we must research in a slow mode, think deeply about the most important problem, do things to aim at high assurance. But it is crunch time. It was crunch time two months ago. We must make hacky progress now and trust that scale will clarify things later. I think these tensions are, at least in part, rooted in what I will call a "war" frame and a "civilisation" frame. These frames are oriented around a different end goal, or telos of AI safety. In the war frame, AI safety is a campaign with a deadline, with decisive action that needs to be taken at a handful of labs over the next few training runs. In this frame, your job squarely focuses on changing what happens in those labs before then. In the civilisation frame, AI safety is closer to dealing with a new population of aliens arriving among us. If aliens landed, we wouldn't ask how many alien researchers we need; we would need a counterpart to nearly every human institution: diplomats, courts, customs, doctors, police. By that measure, th