Do Language Models have Emotions? [Part 1]

This work was done with limited GPU compute; a more complete version of this work will come in later posts (take a look at my GPU Democracy post on Substack!) MotivationWith Large Language Models approaching and exceeding human-level performance on most tasks, one may ask whether anthropomorphization of Language Models is a good approach for intuitively looking at how the internals of the models operate. If we can see the emotions that a model is experiencing, then can we make inferences about the model's "state of mind" and how pleased it is with its own actions? Prior Work Prior work from Anthropic on Emotion Probes is a good starting effort in this area, but the main gripe with probing methods is that they treat neurons as the fundamental level of processing and try to explain an emotion based on the neurons that are fired. This approach ends up being noisy because, due to the inherent property of Polysemanticity and the Superposition hypothesis, each neuron ends up playing a part for many emotions at the same time and hence isolation and attribution of the magnitude of a certain emotion to a particular neuron becomes hard. Since our main idea is to focus on the layers in which a particular emotion is exhibited along with its intensity, LAT scans end up being a more computationally efficient and more visually useful metric that one can use to capture this (visually) in a high-dimensional space. Probing as a technique would better help in case we want to use it with the additional concept of dictionary learning, which is the idea of splitting these polysemantic representations of each neuron into concrete monosemantic "atoms" (Based on the idea of SAEs) that represent the actual set of words that went inside a model's brain when it was giving you that answer. As doing so would be a scope creep, and hence that line of exploration is again left aside for now. After being inspired by the idea of Linear Artificial Tomography scans introduced in the paper by Zou et