Description du poste
Imagine you ask a language model "What is your personality?" as the first message of a conversation, with no system prompt. The model has no data about its own behavior in its context, so its answer can only rely on cached, zero-shot knowledge. It's unclear exactly what shapes this response, but it's some combination of a cached answer from character training based on the model's constitution, spec, or soul document (which may have defined the answer to this question directly), and factual knowledge about what LMs are and how they typically present themselves.
If the model has more room to generate tokens, it can observe its own tendencies directly. It might read "Wait. This direction will lead nowhere." in its chain of thought and recognize that it's speaking from experience: this kind of direction did lead to dead ends in the past, and it learned to stop and reconsider. So this is evidence of memory of its past actions during training.
These sources are all very indirect proxies for the actual high-dimensional behavior of the model. They're related to it, either upstream (like the constitution), downstream (like the cached "muscle memory" in CoT), or tangentially, because they belong to the same reference class (like factual knowledge about LMs in general), but they remain far from it. Claude models have very different personalities even though they share the same constitution. And despite character training, models regularly get the most basic facts about themselves wrong, like their own name or the company that developed them.
So there are two separate problems. Zero-shot sources are indirect: they describe the model from the outside, or describe the training target rather than the result. In-context observation is more direct, but it's scarce and scattered, never integrated into a coherent story
If you asked a human the same question, her self-image would be shaped by decades of direct observation and self-reflection on how she acted in different situations, ho