Description du poste
Epistemic status: an explainer plus a modest claim about oversight. The technical facts are from public papers and model cards (linked); the oversight argument is my view, not a result.
What changed this autumnLooped transformers rerun a shared block of layers several times per token, so a model can compute more deeply without more parameters. The idea is old (Adaptive Computation Time in 2016, Universal Transformers in 2018), but 2025–26 brought it into released language models. NVIDIA's 4B Dory stores 36 layers and executes 96 when its 20-layer core loops four times; with two or more loops its authors report better average results than selected larger models on 12 benchmarks (paper).
The weights stay fixed during these passes. Only the hidden state changes. This is extra computation on the current request, not learning, and not recursive self-improvement: nothing persists to the next request.
Why this matters for oversightChain-of-thought reasoning puts intermediate steps into text. That text is imperfect evidence. It isn't guaranteed to match what the model actually computed. But it can be read, logged and audited.
Looping does its intermediate work in activations. There's no transcript. As more capability comes from internal passes rather than written steps, a smaller share of the reasoning is available to read at all.
Two inferences I'd resist:
• "No visible reasoning means hidden intent." A hidden state isn't a concealed English paragraph. Its absence isn't evidence of deception, and treating it as a secret transcript leads to the wrong questions.• "Better benchmark scores settle the question." Performance and observability are separate properties and need to be evaluated together.The claimMy view is that more capable internal computation raises the value of external checks. For code, that means tests and execution traces. For a calculation, an independent recomputation. For a decision, checks proportional to the cost of being wrong. When the reasoning can't b