← Toutes les offres
X

Taking apart a 32-number network

Xabilimon

Global
Publié le 16 novembre 2026

Description du poste

Cross-posted from my blog: xabier.me. First post of a series on ARC's AlgZoo networks. 1. The Leeuwenhoek Context. I’ve decided to name the project after Leeuwenhoek, a pioneer of microscopy who identified thousands of tiny creatures—“animalcules”—in a single drop of water. Our “animalcules” are neural networks. ARC (Alignment Research Center, an AI safety lab) published a series of tiny networks ranging from 8 to 1,408 parameters—which are “tiny” compared to the models we have today. These networks had been trained to perform very basic tasks, such as finding the second-largest number or the median. They think that if we can’t understand networks as small as these, how are we going to understand ones with billions of parameters? It makes sense if you stop to think about it, and they raise a very interesting challenge. When I was reviewing their progress and contributions, I noticed that many people were focusing on a family of AlgZoo models, specifically 2nd_argmax (ARC already has a pretty good grasp of its 32-parameter network, which isn’t this one), and that, from what I’ve been able to find (November 2026), argmedian hadn’t been publicly explained yet—so here we are. 2. What It Does The network receives three numbers, (x1, x2, x3), and says which of the three is the median. It’s a recurrent neural network, meaning it reads the three numbers one at a time and stores the necessary information in four neurons. It has a total of 32 parameters and gets it right 98.46% of the time. We’re going to answer two questions: how does it do this with just four neurons, and why does it fail 1.54% of the time? We’re not focusing on the 98% success rate—we want to see where it fails and why. 3. Where It Fails The first thing I noticed was that the network has no biases; that is, if we multiply all the numbers by the same value, the neurons and the output scores are multiplied by that number, but the answer does not change. Two sets of proportional numbers, such as (1, 2, 3