Description du poste
Methods note: The code used for the experiments and related open-source repo were built with Claude. The experimental design and writeup is my own, with minimal editing and formatting amendments made with Claude
Introduction
This is a research report inspired by a single question: in a model engineered to answer a simple biological question, how are higher-order, auxiliary concepts represented and used? ESM-2 is a model designed to infer missing amino acids in protein sequences (Lin et al. 2023), but what does the model use and know of emergent secondary structure? These are two different questions, and the data in this post serve to distinguish between the two: information about secondary structure is decodable, but when ablated has a limited, quantifiable effect on the model's accuracy. The implications of this research are in service of the greater question: when we use an LLM for a biological task, but we (appropriately) worry about how higher order or secondary concepts are being used by that LLM, how can we monitor or manipulate the representation of those secondary concepts? I personally believe that the bigger questions require parsing the smaller ones first, and so here I characterize the representation and usage of secondary structure (DSSP-labeled helix, strand, or coil) in ESM-2 150M.
Is secondary structure linearly represented?
The experiment to answer this question is relatively simple--use a linear probe to decode the vectors which represent the DSSP-labeled secondary structures, ablate those representations, and compare model performance to ablation of a random plane. The first thing to test is that the ablation worked; how well it worked provides information about the linearity of the concept representation. At layer 29 of 30, performing a linear ablation and then refitting a new linear probe on the same layer decreased decodability (hereafter referenced in macro-F1) from 0.81 to 0.71. This suggests, simply, that while some of the concept is linea