Game of AGI: An AI Policy Simulator

Thanks to @abramdemski for his feedback on the game, especially the newspaper idea. Estimates of catastrophic risk due to AI are extremely difficult to make. Doing so combines the difficulties of understanding global economics, geopolitical relationships, deep learning, sociology, virology, open source ecosystems, the psychology of AI leaders, and more. But at what level of abstraction should we be modeling these to be able to make the right decisions in time? Too low and we lose the forest for the trees (e.g. modeling fluid dynamics at the molecular level). Too high and we could miss important details that, when added together, meaningfully shift what the model recommends. This is the balance I tried to strike with Game of AGI. It's the start of what I hope can become a codified way of evaluating global AI strategy. There's a campaign mode which is meant more for the general public and can be played at the home page in addition to the Futures Lab, which shows all the futures rolled out at once. I actually hand-coded a similar Monte Carlo simulator in 2019 and presented it at an AI Safety Unconference session in 2021 that @Linda Linsefors moderated. The x-risk probabilities which came out of attendees' stated assumptions at the time were 10% to 50%. The new simulator estimates 78% of futures (distribution below) ending in catastrophe under AI 2040's worldview and timelines. Catastrophic risk is 4-100% in 90% of worlds. However around half of the probability mass sits at 100% in worlds where uncontrolled AI is not kind, alignment is hard and AI development continues at the current pace. The important part isn't the number so much as how different policies affect it. In the name of transparency, the 1.7M-token Claude Code CLI session that I developed the sim in is viewable here. The session exceeds the context window of 1M tokens by substituting old messages with hierarchical summaries and tools to expand summaries. Futures Lab within Game of AGI FeedbackWhat do you