← Toutes les offres
M

An ASI that's bad for animal welfare is probably bad for humans, too

MichaelDickens

Global
Publié le 16 novembre 2026

Description du poste

Suppose artificial superintelligence is aligned enough to not kill everyone, and to not create a dystopia for humans—but it doesn't care about animal welfare, and it ends up creating a terrible world for non-human animals. I don't think that's likely. When I try to describe how an ASI could end up good for humans but bad for animals, the mechanisms I come up with are either bad for humans too, or require picking a narrow and unusual set of values. Summary of the upcoming sections: • The most obvious way aligned ASI could be bad for animals is that it's designed to satisfy humans' immediate preferences. • However, that version of ASI would do insane things that are bad for humans—such as satisfying religious fundamentalists' preference for heretics to be tortured forever. [More] • One approach that might fix the heresy problem without preventing animal suffering is self-regarding preference aggregation. But this approach is specific (we'd have to choose it out of a huge space of approaches), and it's still not clear exactly how it would prevent a catastrophe-for-humans. [More] • What if ASI only acts as an assistant, and humans remain in control of decisions? This is implausible: a sufficiently smart AI de facto controls everything; and humans show great willingness to delegate to AI. [More] • What if AI scaling hits a plateau? If so, it's helpful to shape AI values in the conventional way—putting animal welfare into AI constitutions, etc. But if the plateau is temporary, then these interventions only matter temporarily. [More] • In general, "bad for animals, but not bad for humans" is a narrow alignment target. [More] Cross-posted from my website. Introduction An agentic ASI has some process for deciding what to do. Consider how such a process might result in an outcome that's good for humans but bad for animals. The process cannot involve deeply internalizing fundamental ethical principles. Those fundamental principles would include things like "it is bad to