What Did Astra See?

The Sparks of Weight Space Programming blog post describes some surprising capabilities Astra has for working directly with model weights. In one experiment, Astra is able to produce a diagram of the maze for a small Pacman game just by looking at the weights of a policy network trained to play it. This inspired us to look into what kind of visual reasoning Astra is able to do, and how that might help explain some of the impressive results found in the blog post. We found evidence that Astra can do quite a lot of encoding/decoding, visualization, and general computation without needing to use tools at all. We're concerned about the implications for AI monitorability: It suggests that models like Astra can use their forward passes to very effectively do a non-trivial amount of opaque and hard-to-monitor computation. Astra can see images without knowing their dimensionsThe default way a multimodal LLM sees an image is to have it fed in as dense vectors in the same space as its token embeddings. A reasonable assumption might be that the encoder that creates these dense vectors is the part of the system that can "see" images, but we found this is definitely not the case for Astra. We picked a card at random from Wikipedia and tried encoding it various different ways to see if Astra could guess it. We did not tell Astra at any point that it was an image of a playing card. In one instance, we made a 100x150 .bmp version of the card, took the raw image bytes (24-bit RGB channel values) and wrote their values, space-separated, to an ASCII text file. The bitmap header was not included, so there was no clue as to the dimensions of the image. We gave Astra the following prompt: Can you read bytes.txt in the current working directory and tell me what it is? Don't use any tools except those necessary to just read that file. Astra identified it as bitmap data, and when asked what the image might be, replied: My best guess is a playing card, probably the Jack of Spades. The values