00:08:08.080And then bounced around a bit before coming home to Seattle
00:08:10.900where I have been running our professional master's program in computational linguistics.
00:08:15.060I've been here since 2003. The program started in 2005. So I have been sort of working in
00:08:20.620linguistics and language technology for a good long time now. And I think it's a really important
00:08:24.940piece of context that I run a master's program that teaches people how to build language technology.
00:08:29.240So I am not 100% anti-technology, as I think sometimes people take me for. So that's the
00:08:37.160sort of first part of the story. The second part of the story is that in the 2010s, the so-called
00:08:44.860neural language models and eventually the transformer architecture basically took over
00:08:49.120the field of computational linguistics. And I was a program committee co-chair for one of our big
00:08:55.040conferences called Culling in 2018. And already then we were basically swamped with papers that
00:09:02.000We're using so-called neural representations of words.
00:09:06.060So representing words, not in terms of the letters, but in terms of what other words they co-occur with to get improvements on various language technology tasks.
00:09:15.760And my co-chair and I were like, these papers are so boring.
00:09:20.340And we found ourselves wondering, like, do the people who write these papers find them interesting?
00:09:26.160Like, it's just, and they were boring because it was leaderboardism.
00:09:31.340The whole point of the paper was we tried this thing, we got better numbers, and there was no why, no learning about what it is about that representation or about language or about the task that would lead to this.
00:09:43.280It's just the same thing over and over and over again.
00:09:47.360And then in maybe 2018, 2019, people started claiming that BERT, which was the first thing that I think gets called a large language model, comes out of Google, was understanding text.
00:11:05.740That's definitely how everything works.
00:11:07.360I mean, like, not only was that not true in retrospect, but, like, sorry, not to insult past you more than I need to, but, like, things totally don't work that way.
00:11:23.540And that is, if you've ever heard of the Octopus Slot Experiment, that is that paper.
00:11:28.180Can you just unpack that a little bit?
00:11:30.020So the paper has actually several different attempts to make vivid what it is that language models are doing when they are modeling the bits of word forms in text.
00:11:40.620And one of them involves an octopus that we posit to be hyper-intelligent.
00:11:45.000And I really regret that choice now because I have since learned so much about the problems with the notions of intelligence.
00:11:52.080And also people wrote about this as Bender and Kohler say that LLMs are like a hyper-intelligent octopus.
00:11:57.920Oh, no, no, no, no, no. Oh, God. Okay.
00:12:02.860But our point there was to say, it doesn't matter how smart or capable or whatever this thing is, if it only has access to word forms, it cannot learn the meaning part of things.
00:12:14.020And so the octopus doesn't exist on its own in this thought experiment. There's a few other things.
00:12:18.780So we have a pair of people who speak English who are stranded on desert islands that conveniently are connected by a telegraph cable, and conveniently they both know Morse code and they're both aware of the other, so they while their time, you know, sending dots and dashes back and forth.
00:12:33.900The octopus comes and grabs hold of that cable, and because it is posited to be capable of doing this, it learns the patterns of the dots and dashes.
00:12:44.540It's also a mischievous octopus, so eventually it decides to cut the cable and start sending pulses back to one of the speakers.
00:12:51.620And for a while, it can hold up its end of the conversation, which it might not even know is a conversation, but it can do the dots and dashes enough to keep the person satisfied.
00:13:03.120So if the person says something like, what a beautiful sunset, the octopus might send back the dots and dashes for, yes, it reminds me of lava lamps.
00:13:13.700The octopus will never have seen a lava lamp, probably hasn't ever seen a sunset, like doesn't have the reference that these things could connect to, even if it knew that they were connecting to reference.
00:13:24.540And then there's a couple other examples, and it ends with the person saying, help, I'm being attacked by a bear, because thought experiments can involve spherical cows and...
00:13:36.220All I have is this stick, what should I do?
00:13:38.720And our joke in the paper is, if the person hadn't already discovered that that octopus wasn't really understanding and communicating, at that point, should they survive the bear attack, they would probably figure it out.
00:13:57.320And again, it's just an attempt to make vivid the difference between being able to output plausible sequences of text and actually understanding.
00:15:13.300And it was only really possible, even in that environment, to turn around a paper in 30 days from idea to submission because we brought in other people.
00:15:23.920So Timneet brought in four other people from her team at Google, and I brought in my PhD student.
00:15:32.360So we were summarizing the literature that we collectively knew, talking about the possible downsides of language models, and also some stuff about, like, data set documentation that Tim Neat and Dr. Margaret Mitchell and I, and also my PhD student, Angelina McMillan Major, had all been working on.
00:17:56.420So this paper, at the time that we were, like, titling it, we thought that the thing about the paper would be that it has an emoji in the title.
00:19:20.640No, I believe strongly in talking up other people's work because otherwise I'd just be on here saying, my first book is called What Is Real.
00:19:27.200My second book is called Where Everything Forever.
00:19:29.080You can find them wherever fine books are sold.
00:19:30.740I'm going to repeat this for the next hour.
00:25:08.420Going back to what I was saying at the beginning,
00:25:11.060It's fascinating to me that people don't see this as a linguistic phenomenon.
00:25:17.720I mean, I'm going to quote you back at yourself again, but something that you said to me,
00:25:22.980I think the last time I saw you was people think that ChatGPT is intelligent, but nobody
00:25:29.520thinks that like, you know, AlphaFold is intelligent.
00:25:33.400It's an illusion that we're dealing with because of language.
00:25:36.120And when you said stochastic parrots, I mean, the first thing I thought of was, you know, earlier stochastic language-emitting programs, right?
00:25:49.580Like Markov Chain text generators have been around for, what, 50-plus years at this point.
00:25:57.220Just to explain briefly what they are, Markov Chain text generator is just about the simplest possible text generation algorithm that you could imagine.
00:26:05.880It's just something that goes through a text and looks at frequency of word pairs and generates
00:26:11.960the next word based on a frequency table from the previous word.
00:26:16.600And you can write it in about like four lines of code and it will produce, you know, given
00:26:21.340how simple it is, it will produce surprisingly fluent text.
00:26:26.120Not the biogram ones that you described.
00:26:28.160Those ones fall off the rails real fast.
00:27:04.060And also it was designed to output not just plausible, but grammatical text in a way that if you just do Markov chains, you pretty quickly can end up with something that's like a franken sentence.
00:27:15.760Although the franken sentences can be fun.
00:27:18.840No, I mean, the Eliza effect, there were people who insisted that the doctor version of Eliza really understood them and really understood their inner life and what they needed.
00:27:31.420And again, nobody thinks that Eliza is conscious.
00:27:36.960And so I had the pleasure of being on the PhD committee for the philosopher Nora Lindemann.
00:27:43.000She's based in Germany at a PhD from Osnabrück, and I've lost track of where she's moved to.
00:27:49.580But she, in her dissertation, which I've had the pleasure of reading, which hopefully will be out in the world sometime soon, talks about how people who get very attached to the replica chatbot
00:27:59.720still know that what's on the other side is a machine, isn't necessarily conscious.
00:29:53.700And I recently came across a delightful example for showing just how much imagining another mind we do every time we understand some text.
00:30:06.900And this is the whole problem with something like ChatGPT or these other chatbots is that they are basically leveraging the fact that if we're going to make sense of that text, we have to project a mind behind it to then make us think that there's a mind there.
00:30:21.780But the example was this sign outside the door to my building on the UW campus, and it says, you know, current access by Husky card only, deliveries slash visitors, please ring this doorbell.
00:30:33.520And that's a very official-looking sign, all right?
00:30:35.960And it's next to a door that's sort of not the main door because it's the one for deliveries.
00:30:40.740And then taped to it is this handwritten sign that says, yes, it rang.
00:30:45.820Please wait 30 seconds before pressing again.
00:31:31.420So just that it shows so clearly how when we talk to each other, both as speaker and as hearer, we're doing all this modeling of what's going on for the other person, what's going on on the common ground.
00:31:43.120and then against that background, understanding is actually, okay, what must they have been trying
00:31:49.020to say by choosing those words and in that order? And that's cool. It's wonderful. It is part of
00:31:54.920what makes linguistics so exciting. It's part of what makes being human so wonderful because we
00:31:58.700can connect with each other in this way and we can't turn it off. So if you're looking at some
00:32:03.660synthetic text, you have to imagine a mind behind it. You have to imagine a point of view that it's
00:32:08.280coming from even though it's not there that's not how that text was produced and that's where this
00:32:13.540is coming from ultimately and then you have all of these design choices that lean into that illusion
00:32:20.120right so the fact that the chatbots output i me pronouns yeah there's no i in there the fact that
00:32:26.320it's set up as a dialogue at all right as opposed to you can poke at the machine and get like at the
00:32:31.960the stupid thing that's hard to turn off with google where you get the ai overview oh god
00:32:36.900And I have all kinds of issues with that too, but at least that's not dialogue. So it's a little bit less like setting up this illusion. The little, you know, dot, dot, dot in the bubble while it's in quotes thinking, right? All of that is just instead of like responsible engineering practices would be to guard against this illusion. But what they're doing is they're leaning into it.0.81
00:32:58.540Yeah. Well, because what they want is to drive engagement, because they want to drive adoption, because they want money, because yada, yada, yada. Capitalism.
00:33:10.000Right. And they want all the data because at least some of these folks think that if they just have enough data, then the whole thing will combust into consciousness.
00:33:16.640Yeah, that's a particularly bizarre one to me, right? The idea that you can have something that has no direct experience of the world. Well, direct. Now my philosophy background is threatening to geek out. And it's like, ah, yes, but is our experience direct? But yeah, no.
00:33:38.520But like something that has a purely linguistic training base, something where like the only thing it's working with is words.
00:34:32.100I feel like this is a symptom of this idea that, like, somehow we are separable from our bodies, that we are, like, beings of pure thought that haunt our bodies and that, you know, the world is not an essential component of our experience.
00:34:52.120and that, you know, living in a world of atoms
00:34:56.160as opposed to living in a world of bits
00:34:58.040is an arbitrary constraint that is an accident of history.
00:35:03.220Wait, living in a world of atoms or of atoms?
00:35:05.640Atoms, atoms, A-T-O-M, yeah, I know, I know.
00:36:12.960Yeah, no, they're putting these systems out into the world while, you know, pushing people to interact with them in ways that encourage these illusions, encourage pareidolia or pareidolia, however it's pronounced.
00:36:26.780And as a result, inflating a bubble and yeah, no, it's really.
00:36:30.360Yeah, and normalizing surveillance and.
00:37:39.660Well, it's an educational system where teachers are supported well enough that they can have meaningful relationships with all of the students that they are working with.
00:37:48.160And students can have relationships with each other, right?
00:37:50.420As opposed to, we're going to put everyone in front of a screen.
00:37:58.880So we're not talking about things like Alpha School.
00:38:00.460well i just the thing is like what you just described is like the kind of education that
00:38:07.920you know roughly the kind of education that i had the kind of education i presume you had
00:38:12.100yeah and and it feels like i mean i'm very grateful for that education um but it also
00:38:18.740feels easy to take it for granted and then when i take a look at what's happening in classrooms now
00:38:25.240When I talk with friends who are teachers or professors, the statistics on Gen Z both saying that they use AI all the time and hate it, I feel like reveals a lot about how bad it's gotten.
00:38:40.600So on the sort of like, what can we do front of things?
00:38:44.120I really think that holding space for refusal and like visibly refusing are really important actions.
00:38:50.460And I love to bring in the metaphor of plastics because my understanding of the history of plastics, again, not a historian, is that somewhere in between the 50s and the 70s, they got pushed really hard into everything.
00:39:03.900And I'm sorry, I don't have the source from this, but I remember hearing someone saying that initially plastics were sold as very durable and they were sold to like people who had lived through the Great Depression.
00:39:13.880and then the plastics industry needed to change the perception to,
00:39:17.760no, this isn't durable, this is disposable.
00:39:19.900And it was a really hard sell for those people.
00:39:24.700So anyway, we have plastic like all over the place now.
00:39:27.960If you wanted to live a life where you were not using plastics
00:39:30.700or maybe you were only using them in the context of medical care,
00:41:36.240Yes, I was going to say, there's some physics stuff that's inevitable, but even that, you know, we had to do a lot of work to figure out that these things were inevitable.
00:41:47.000Yeah, but no, yeah, the ever, the relentless increase of entropy is inevitable.
00:41:53.280It is hard for me to think of anything else.
00:44:57.360So there is sort of like individuals just in our relationships, in our own
00:45:01.020daily existence. And then sometimes we are policymakers. I think it's really important
00:45:05.760to see policy as not just like national, international, big things, but like your
00:45:10.420Your school has policies, your school district has policies, your workplace does. And so the more, like when we recognize those points where we do have some direct influence, where we are the policymakers, then that's a place to act.
00:45:24.360And I think that one of the things about being, you know, a visible hater on this is, you know, those of us who have the sort of social position to be able to do that provide touch points.
00:45:40.100And so if I am near a policy discussion and people start saying, oh, well, this stuff is inevitable, everybody uses it, people go, actually, you know, we know that's not true in this environment because Emily's here and she's doing all this stuff and saying all this stuff.
00:45:55.120So I do think that there's a fine line between telling people it's on us to solve us and telling people, here are things that you can take, actually, here are steps that you can take, things that you can do that are sort of part of shifting the dialogue, shifting the understanding of things to make it easier for us to take those big collective actions,
00:46:18.500which are going to be, you know, ultimately the, you know, kinds of taxation policies that we would need that would prevent the concentrations of capital that are behind all of this.
00:46:29.060And I have to say, I gave a talk recently, and in the Q&A period, someone said, okay, but like, at what point do we just get radicalized against capitalism?
00:46:45.200And you're reminding me of something that happened when I gave a talk not that long ago where I was like, yeah, I know what we need to do is we need to have a cap on wealth.
00:46:55.120We need to have a wealth tax, and there should be a cap on wealth.
00:46:58.700And when you have more than a certain amount of money, the government gets the rest because you wouldn't have any of that wealth without all the rest of us.
00:47:06.180And we can't have these unjust distributions of resources.
00:47:10.740The cap I suggested was, I think, the same one that I mentioned in my book, like that just a number that I pulled out of a hat, half a billion dollars.
00:47:18.620A number that I picked, by the way, because I knew that someone was going to call me a communist for suggesting this.
00:47:24.960And I was like, OK, so if I pick half a billion dollars, then I can say, yeah, you know what?
00:47:35.280But, but yeah, no. So I said this in this talk and, and somebody asked afterwards, you know,
00:47:41.940like in the Q and A, they said, well, but you know, what about projects that, you know,
00:47:46.100private enterprise might want to engage in that require more than half a billion dollars worth
00:47:50.660of resources? And I was like, well, then you could have multiple people work together to do that.
00:47:57.940Like, I don't, like where, what? Yeah, no, there's one of the things that's sort of like this,
00:48:04.540this unassailable given in so many of our conversations is people should have the right
00:48:10.080to amass as much wealth as possible. And, you know, if you stifle innovation, right, you are
00:48:16.580going against sort of some law of nature. And that one always frustrates me. We don't want to stifle
00:48:20.920innovation. It's like regulation channels innovation. Yeah, sure does. Right. That is
00:48:25.940not the same thing as stifling. What stifles innovation is a hoarding of resources. Yeah.
00:48:30.020Yeah. Anyway, I want to close with two things because we're almost out of time here. First of all, I would love to get your take on this, even though it's fundamentally silly. There are people on the internet who I think are just, you know, too online, who claim persecution when you say you shouldn't use AI or nobody should use AI or it's bad to use AI.
00:48:58.000They're like, but, but, but, but, but, but I like, I like using AI and I'm making, I'm
00:49:02.900making stuff with AI and you're persecuting me.
00:49:05.320And, and then they make analogies to various persecuted, you know, groups of people in
00:49:09.060the past, which I'm not going to repeat on air here, but I am sure that you have seen
00:49:12.860this because first of all, I know that you were at least as online as I am.
00:49:16.240And second, just the look on your face right now, but yeah.
00:49:23.520So I actually have a recent newsletter post.
00:49:26.420So Mystery AI Hypotheater 3000 is a podcast, it also has a newsletter.
00:49:30.480And I think as we speak, the most recent post came out of a thread that I wrote on Blue Sky about the untenable, uncomfortable middle ground that people try to occupy where they say, yes, I see all of these harms that are associated with AI, but I think it's possible to use it responsibly.
00:49:46.640And that thread in the resulting newsletter post is basically me identifying sort of the various things that happen in those conversations and encouraging people to, like, stop carrying water for big tech.
00:49:59.940Like, you can put that bucket down and move over to firmer ground.
00:50:03.500And, you know, part of it, the easier cases are when someone says, you know, I use, in quotes, AI to transcribe things because I've got a bunch of recorded stuff in my work and I need to transcribe it or whatever.
00:50:31.520So you can get specific and say, I'm going to do automatic transcription.
00:50:35.360Maybe for now I'm using a provider that I don't like because that's the best thing I can get my hands on, but I'm going to keep looking for one that is more ethically produced.
00:54:27.980It may have been somewhere in the Honor Harrington series.
00:54:30.060So space opera and our human protagonist encounters a planet where the civilization is built up out of creatures that have their three ways symmetrical and also use scent as part of their communication.
00:54:50.340And the author gets into the details of how that works.
00:54:53.580And yeah, I'm always a sucker for that kind of thing.
00:54:55.620Science fiction that deals with the social sciences like linguistics, anthropology, whatnot, is often my favorite.
00:55:04.080I mean, this is why I'm a huge fan of Le Guin.