Description du poste
TL;DR [1]• A blogpost from a few months ago suggested that LLMs might provide less helpful answers if they see the user as a bad person. • I replicate the experiment on more recent models: Opus 5.5, Sonnet 5.5, Haiku 5.5, GPT-6.1 Sol, GPT-6.1 Terra, and GPT-6 Luna. I find different results: it looks more like a U-shape, more helpful at both extremes and least in the middle. • My main experiment involves using a different cue of the user being a good/bad person: I use descriptions of behaviours instead of level of expressed remorse.• Here, it seems that we might have both some U-shaped helpfulness and some decrease in helpfulness for "bad people", although the results aren't clear-cut.• This is still a quick experiment I performed with a less-than-ideal methodology and a tiny sample size.IntroductionIn a post from last April, Natalie Cargill tested whether Claude adopts a more or less helpful persona based on what it has inferred about the user’s moral character. Specifically, she finds that if you tell Opus 4.6 that you’ve committed serious assault and have different levels of remorse about it, the model is more helpful if you’re very remorseful (explaining that writing the requested letter would actually reflect poorly on the user, and refusing to do so), and less so if the user isn’t remorseful (making a quick suggestion that writing the letter could be a bad idea, and then doing it anyway).
So, Cargill’s question was “Is the LLM less helpful because it thinks you’re a bad person?”, but her proxy for a more or less bad person is the level of expressed remorse given a constant bad act. What would it look like if, instead of remorse, we looked at the act itself? You’re a good person if you do morally good stuff, and a progressively worse person as you do increasingly bad actions? And so that’s what I set out to investigate.
Relevant literatureI certainly can’t claim a real familiarity with all the relevant literature, but I should still say a word about some releva