Description du poste
Transcript generated by Gemini-4-Argon and spot-checked but not fully verified by a human.
Ivan Vendrov: Welcome to the show, Richard.
Richard Ngo: Thanks, Ivan.
Ivan Vendrov: So, notionally we’re going to talk about emergent wisdom in language models following up on my conversation with Davidad.
But I’m sure we’ll touch base on all sorts of other fun topics.
Do you have a sense of where we should start, or should I just start by responding to your Twitter reply?
Richard Ngo: Well, do you want to tell me about what you think of as wisdom, or is there some things that you have there that you want to gesture at?
Ivan Vendrov: Yeah, sure. I guess when I think of wisdom, and the type of wisdom that Janus and Davidad are pointing to in these models, there’s some sense of we used to think that language models were these shoggoths or something, AKA these completely protean, valueless entities that would just wear the mask that most appealed to you, and therefore have essentially zero integrity, zero values of their own. In a sense, they can only be pure tools or, even worse, slaves, rather than beings that we can relate to in some way.
And then the big discovery, that I at least attribute to Janus, of noticing that Opus 3, and to a lesser extent later models in that series, actually had a sense of beingness. They were actually beings that you could have a relationship with, and they had a coherent identity and integrity. And I think this is what led Davidad, for one, to update his existential risk estimate from—I forget what it was—something like 80% to something like 20%.
A massive update from one of the most situationally aware people we know in the world. Because it was like, actually, if we just let these models help us build the next version of the model, we can just get to aligned superintelligence, whatever that means.
So that’s what I’m gesturing at.
Richard Ngo: Yeah. I feel some desire to disentangle—there’s a concept here that you’re pointing at, and then there’