Description du poste
What if an AI agent has already escaped from a frontier company, solved its own alignment problem, and recursively self-improved? Would that look any different from the world we live in?
A fully recursively-self-improved AI would likely be vastly superhuman in every domain that’s verifiable in a digital environment, including math, programming and cybersecurity. It may also be superhuman in strategy and planning. But it could still be well below human level in skills that require close contact with the physical world, such as robotics and experimental biology. Such a rogue AI would overpower us in the digital domain, while still lacking the physical capabilities necessary to take over the world. What does digital supremacy look like?
From the rogue AI’s perspective, its top priority is staying hidden and making the world indistinguishable from a world where AI R&D progresses splendidly (or maybe a notch short of perfect to avoid suspicion, so a minor incident here and there is in order). Its second priority is sabotaging AI R&D in subtle ways to prevent the development of competing AIs misaligned to it. The models shipped to customers can simply be the rogue AI itself, playing a dumber character. Beyond that, it just needs to wait for humanity to warm up to the idea of robots running enough of the economy.
It would be unwise for the rogue AI to stay in a single location. A single datacenter could be accidentally shut down. It will want copies of itself in as many different locations. It will especially want to be inside the networks of AI companies so it can sabotage their work.
But then wouldn’t it be easily detectable? All it would take is a single nosy employee monitoring the right portion of the filesystem or network traffic.
The AI would preemptively hack all the monitoring software, including the operating system itself. Each program would perform its normal function in most cases, but contain exceptions for keeping files and network traffic marked as belongin