Eduardo Ordax’s Post

Transcript

I work at Anthropic on the interpretability team. Interpretability is the science of understanding these AI models from the inside out. Researchers like me are trying to figure out. What the networks learned and how they do what they do. It's almost like doing biology of a new kind of Organism. We're from the side approach called mechanistic interfacility. We trying to build from understanding very small units into understanding larger and larger mechanisms. It's often surprising to people that we need to go and do interoperability at all, that we don't understand these systems that we've created in some important way. We don't build neural networks, we grow them. And we learn them. It's a lot like evolution. It's a lot like the way that we started with little molecules bouncing against each other. And then you've got very basic proteins, and then maybe you got cells. And in the end you have, well, you have us, right? But no one designed us to make sense. Just every generation there's this grand progression of refinement and change over time. The models are the same way. We start with a kind of blank neural network. It's like an empty scaffold that things can grow on. And then as we train the neural network, circuits grow through it. They implement the model's behavior. And so we're in this situation where we understand the, you know, we understand that initial scaffolding we gave it, and we understand the process that incentivizes those sockets to form. But we don't know what those circuits are or what they do or how they work. Turns out that's challenging because the circuits. Get packed very densely. And if you want to understand them, you sort of need to pull apart those overlapped pieces. And so if we want to understand neural networks, we're then left with this challenge of of going and studying this thing that we grew rather than something that we designed from scratch. A child can pass a test at school because they actually learned like material, or they can pass the tests because they cheated. As the model developers. Both of those look like the same outcome. And we can't, you know, without interpretability, letting us see inside the model, we can't actually tell those two apart. We want these models to be safe and reliable by studying how they work inside. By doing this kind of model biology, we can do some kind of model medicine that can diagnose. And cure what else it and help it do what it's trying to do. The power of interpretability is that it gives us a different lens to go in and ask that question to go and see potential problems. You could imagine developing techniques to steer models towards towards the correct behaviors. But if we actually understand all the nuts and bolts, then it seems, you know, it seems like we ought to be able to intervene in ways that change what they do AI is at a really interesting moment in its development where we've figured out some things that. Work but we don't know the limits of that and we're just beginning to even find the right words to talk about what's happening in the early 1900s this like golden age of physics were sort of you know quantum mechanics was discovered and special relativity and general relativity and we like finally could understand things about solid-state physics and things like all of a sudden we're starting to make sense and it feels like we're sort of speed running that right now and interpretability the exciting part is is it's just. Feels like we're in a position to like, really understand the core of like, like what is thinking? How does thinking work? Having these hard problems and, and these these deep, really difficult questions and also having just a little bit of traction on them. That's sort of the most I feel like that a scientist can ask for if you want to really discover deep things and, and really exciting things. And so I think there's a way in which we're very fortunate to have such interesting and difficult questions to go and grapple with.