Description du poste
Cross-posted from my blog.
Epistemic status exploratory, no running protocol, theoretical estimates from a single experiment and 4 closed-weights models
Two copies of the same model agree on arbitrary questions more than other models would. If the model's weights are secret, this similarity allows identical LLMs or their users to agree on a private key over a public channel with only one-way communication. Measuring agreement also allows model "fingerprinting" or self-recognition by two instances of the same model. Copies of a model could use these tools to selectively coordinate over public channels.
This is related to work on LLM evaluators recognizing their own outputs, such as Panickssery et al. 2024, wherein models are prompted with text samples and asked if they wrote them. In the work detailed in this post, "recognition" comes from a statistical test of agreement instead, so it doesn't require that kind of self-recognition.
Key generation
This method can generate a shared private key if adversaries can't query the same model.
• Generate a set of binary choices from a random word list. For example, "do you prefer a navy helmet or a marble teapot?"
• Broadcast that set of binary questions on a public shared channel.
• The models answer each binary choice question to create a string of bits.
• Two copies of the same model agree with each other more than they would with a different model, so that string of bits is a noisy shared key.
• The noisy shared key can then be refined using Maurer's secret key agreement by public discussion.
This is a key exchange that LLMs or their users can conduct:
• without access to their underlying weights,
• with only one-way communication, and
• without exchanging obvious cryptographic information (until the Maurer refinement step).
Fingerprinting
This is much simpler. Identical models agree more than any other two models, so we can assess if two respondents are the same model with a simple test of agreement.
Methods
Mode