Would you let an AI judge you?

Legal AI may outperform human lawyers on some benchmarks, but could an AI ever outperform a human judge? Judges don’t just argue rules, they interpret them and sometimes make new ones! As a result, autonomous judges are a microcosm of the alignment problem with potentially large societal impacts and costs. My parent's dreams - like every other Jewish parent - may soon perish. Their two favorite professions, lawyers and doctors, could be fully replaced by AI in our lifetimes. Beyond AI tools for professionals (Harvey/OpenEvidence), there’s a wave of AI native companies (Crosby/Nolla Health) that empower AIs to autonomously provide legal or medical services without any human supervision. In July, Crosby announced they were “taking out professional liability insurance for [their] agents, so that they can do autonomous legal work,” and Nolla Health became “the first organization in the U.S. authorized by a state regulator for AI to issue an initial prescription.” While still early, the Crosby/Nolla type of firm are fundamentally different from the Harvey/OpenEvidence firms: they threaten to remove humans from the loop. This may not happen immediately - there are regulatory hurdles to jump through even when AI capabilities surpass that of humans - but it seems clear there’s a possibility AI lawyers and doctors fully replace humans in the medium-ish term (3-10 years). If you don’t agree, then I suggest you stop reading here. The rest of this article will build on that assumption and only go further off the deep end. It will also focus on the legal half of this equation, because I am a mere Juris Doctor, not a Doctor of Medicine. If you’re still here, I have a question for you: if you were arrested for a crime and put on trial, would you rather have a human lawyer or an equally effective AI lawyer? If you answered either choice, you’re wrong. You see, I’ve rigged the question - it’s the classic “what’s heavier, a ton of bricks or a ton of feathers.” The AI is equally