00:00:53.240and I've spent my life exploring how other people survive what should have destroyed them.
00:00:58.760I'm Elizabeth Smart, and these are The Survivor Files.
00:01:02.380Every week, I'm with survivors who live through the unthinkable, abducted, stalked, controlled, and nearly silenced.
00:01:09.920These are stories about what it takes to make it out alive.
00:01:13.740Listen to The Survivor Files with Elizabeth Smart on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:01:21.360If your bookshelf and your For You page are equally important to your personality, welcome home.
00:01:27.100Pro Society is a weekly podcast that's part book club, part group chat for anyone who thinks Pride
00:01:32.380and Prejudice and Love Island deserve the same level of discourse. Each week, we're connecting
00:01:36.660the dots between books, the internet, and pop culture with your favorite writers, book talk
00:01:41.300creators, and plenty of overthought opinions. Yeah, I'm obsessed. I'm obsessed. Listen to Pro
00:01:46.480Society on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:01:51.920I'm Hoda Kotb, host of the podcast Joy 101 with Hoda Kotb.
00:01:56.300You know, I'm always searching for inspiration and useful tools to maximize joy.
00:02:01.620Like when my friend Henry Winkler offered advice to aspiring actors and really anybody hoping to stand out.
00:02:08.660You're not hired just to fill time and space.
00:02:11.420You're hired to fill the space with you.
00:02:14.300Listen to Joy 101 with Hoda Kotb on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:02:22.300If someone knocked on your door and was like, I'm going to sell you a potion that can cure all of your problems,
00:02:27.960but it's just going to cost everything you've ever owned, including your firstborn child, you would be like, this is a scam.
00:02:34.780There Are No Girls on the Internet is a production of iHeartRadio and Unbossed Creative.
00:02:48.600I'm Bridget Todd, and this is There Are No Girls on the Internet.
00:02:54.280Few journalists have covered AI with the depth and rigor of Karen Howe.
00:02:59.120Her book, Empire of AI, tells the story of OpenAI and Sam Altman.
00:03:03.980Not as someone trying to start a tech company, but rather someone trying to start an ever-expanding empire.
00:03:12.100Now, with her new BBC podcast, The Interface, Karen and her co-hosts are pulling back the curtain on how tech shapes everything, from politics to our most personal lives.
00:03:24.320Karen's voice in tech is critical, but it happened very much by accident.
00:03:29.020what brought you to reporting on both open ai and then ai in general total accidents both of them
00:03:38.740so i had a interesting route into journalism i studied engineering in college and thought i
00:03:47.280would work in the tech industry which i did when i graduated and very quickly after experiencing
00:03:53.440a little over a year of Silicon Valley, I realized that it was not for me.
00:04:01.120I joined the tech industry at a time when, it was 2015, 2016 era, when what was called
00:04:07.960the tech backlash or the tech lash was just starting because people were starting to realize
00:04:12.640how powerful these companies were and how they were actually often undermining the public interest.
00:04:17.000And so I thought I would have, I thought I needed to find another career. And so on a whim, I pivoted to journalism because I was like, the only other school I have is writing. So maybe I can get a job in journalism.
00:04:32.680And I couldn't find a job in what I wanted to report on, which was the environment, because I had no experience in journalism or in the environment.
00:04:41.180But I figured out that I could parlay my tech experience working in the tech industry into reporting on tech.
00:04:48.360And so I started applying for tech reporting jobs.
00:04:52.280And the only job offer that I got was to cover AI.
00:04:58.080When Karen was assigned to cover AI, at first she wasn't thrilled.
00:05:02.680The landscape looked nothing like it does today.
00:05:05.500None of the breathless hype, none of the alarm bells.
00:05:49.120every facet of tech and society like I wanted to.
00:05:53.920And then because I was covering AI for MIT technology
00:05:57.520Review, which is a very research at the time was a very research focused publication. It was looking
00:06:03.240at the cutting edge stuff happening in labs before commercialization potential. And AI at the time
00:06:10.160was at that stage without a lot of commercial activity primarily being developed in academic
00:06:18.060labs or in corporate labs. So how did you start covering OpenAI specifically?
00:06:23.360OpenAI came on my radar because it was one of the research labs that billed itself as having absolutely no commercial interest.
00:06:33.120And because I was the junior AI reporter on staff and OpenAI was just important enough to have a profile, but not so important to put the senior AI reporter on it.
00:06:43.880I ended up being the one assigned to profile OpenAI and that's how I ended up accidentally being the first journalist to ever profile OpenAI.
00:06:52.420It is so funny to take that walk down memory lane and think about how different OpenAI started as, because I had this almost the same trajectory of thinking, oh, this is a really academic nonprofit organization.
00:07:09.960Don't probably don't need to really like look too hard at what they're doing.
00:07:13.000And then, I mean, saying that now sounds absurd, you know, how quickly they've gone from, oh, we're not a commercial, we're not, we don't have commercial interest to we are building an empire and we're going to take over the world.
00:07:26.100Yeah. Yeah. No, it's it's super like I think at the time I had a totally different perception of open AI than I do now.
00:07:34.340So in the sense that when I started profiling the company, it was already clear that they were leaving behind the mission of being a nonprofit, being open and being in the public interest.
00:07:46.120But I had this impression that it started that way and then it was sort of corrupted along the way by commercial interest.
00:07:54.660But in hindsight, after reporting my book, I realized that actually there was a seed of corruption from the very beginning within the company because OpenAI set itself up to be a nonprofit specifically because it wanted to be the number one AI lab and beat Google, which was at the time the dominant AI player.
00:08:19.200And I think what happened is that with this goal in mind of being number one and dominating in this space, they realized that they couldn't compete with Google on money because they simply didn't have that as much like Google being one of the richest companies in the world.
00:08:36.820Google could always outbid OpenAI on salaries and outspend OpenAI on various things. And so in order to recruit talent, which was the first bottleneck that OpenAI had to overcome, they were able to instead appeal to researchers on a sense of mission and purpose, which allowed them to then ask these researchers to take pay cuts and to consider jumping ship from Google.
00:09:06.820or from another lucrative job to this more startup type environment.
00:09:12.480And it was once they overcame that bottleneck of gathering up all of the researchers
00:09:17.880that they then kind of started slowly getting rid of the nonprofit
00:09:22.920because it had lost its utility and their bottleneck shifted to capital.
00:09:29.280The goal was actually not let's create an AI lab that is hugely open
00:09:34.480and transparent to the public interest.
00:09:36.600Like priority number one was always let's be number one and dominate. And they shifted their tactics over time based on what they needed in that moment.
00:09:49.340Do you think some of those researchers later felt burned or that they'd been deceived after that process came to more or less completion of transitioning away from the nonprofit status that had initially attracted them?
00:10:04.240Absolutely. I think this is one of the most interesting things for me when reporting the book and speaking with different so many employees from different eras of the company is that, first of all, this is like having covered lots of different tech companies, like I've written about Facebook, about Google, about Microsoft.
00:10:24.480OpenAI is the only company where employees cannot agree whether it is a company.
00:10:31.380And like that, like it, their opinion about whether it is a company and like what ultimately it stands for and what is even the purpose of this organization is completely based on when they started at the organization.
00:10:46.060So early day employees, they still think of or thought of OpenAI as a nonprofit that just had to make some concessions and start to look a little bit more company-y, whereas employees that were joining after OpenAI had already started the for-profit, had already raised a bunch of capital and were building commercial products.
00:11:06.900They simply saw it as just another tech company like any other tech company in Silicon Valley.
00:11:13.020And this is like, yeah, like I had a funny conversation with one of my fact checkers when she was going through all of the interviews.
00:11:22.980Like she had the same exact observation.
00:11:25.320And I was like, I feel so validated because I thought I was going crazy, like interviewing these people and being like, this isn't normal, right?
00:11:34.140Like, usually when you work for an entity, you should be able to define whether or not the basics about that entity, like whether it's a company or a nonprofit.
00:11:44.880But because of OpenAI's history, it's this it was this prism or this mirror that every single employee was holding up to themselves and they were seeing something totally different.
00:11:53.580So I listen to a lot of tech leaders speak for the podcast, and generally I have a hard time trusting any of them.
00:12:00.200But that is especially true for Sam Altman.
00:12:03.280This actually came up when I was working on my own audio book.
00:12:05.800I just kind of got the sense that he is someone who will say whatever, whenever.
00:12:10.900And I don't think any of that is an accident either.
00:12:13.620He is what I would call a slippery fish.
00:12:16.820How did you nail him down for your book, Empire of AI?
00:12:19.940Sam Allman didn't agree to interview for the book.
00:12:22.080So what I ended up doing was just listening to hours and hours and hours and hours of
00:12:27.300footage of him talking in various different settings over the years.
00:12:32.660Which ended up being actually a really great exercise because I realized that, first of all, that he shifts, you know, like what he says over the years changes a lot because he will say what he thinks his audience needs to hear in that moment.
00:12:56.560And that will fluctuate based on, you know, the zeitgeist of the moment.
00:12:59.880But also I realized that he uses very squishy language, even while taking a definitive tone to say the things that he's saying.
00:13:10.720So you'll see him say statements like, we believe that a lot of people are going to like this and like and and and very soon there will be more people that like this, you know, like like he uses things that are unquantifiable, like a lot of very soon.
00:13:35.840um yeah he he rarely ever says anything like majority anything that could even have some
00:13:44.160measurable oh majority means 51 so i can actually like hold you to that number like he never uses
00:13:52.580those terms he and he never uses specific values he only ever uses these like squishy things that
00:13:57.840are in the eye of the beholder and i think it's not a coincidence then that like open ai then
00:14:02.020also became like an entity that was viewed by different people based on, you know, the eye of
00:14:08.020the beholder. It's kind of how he operates. I can't help but feel it's an analogy for how
00:14:14.140chatbots talk with people of, you know, sort of painting like a zeitgeisty picture, but not
00:14:20.440actually saying anything with like specific nouns or verbs. Yes, a lot of people have drawn this
00:14:27.660analogy between how SAML operates and how chat GPT is designed. I definitely find myself getting
00:14:36.300caught in the trap of saying, is this AI good or is this AI bad? And you actually offer a much more
00:14:42.040helpful way to think about that. Does this fortify or does this dismantle empire? And I'm curious how
00:14:48.700you got to that framework because it is so helpful. Yeah, this is based on, I have a good
00:14:55.740friend of mine who's a researcher who graduated from Stanford, Rhea Kaluri, and she gave this
00:15:01.600really amazing talk in 2019 that basically articulated and framed this question. It was a
00:15:06.740talk that was given at NeurIPS, the Neural Information Processing Systems Conference,
00:15:11.740which is the largest AI research conference that happens every year. Like 15,000 researchers descend,
00:15:19.320AI researchers descend on one city and take over the city for a week. And she was
00:15:25.600giving this keynote at Queer in AI, which is this organization that of AI researchers and AI
00:15:33.240professionals that identify as queer and want to find community and also reflect their queerness
00:15:39.780in their work. And the key, like in her keynote, she said, you know, it's so hard as an A, as a
00:15:49.120queer AI researcher, like she was talking with like everyone in the room, like as queer AI
00:15:53.060research is it's so hard sometimes to align like your work with your identity because you're taught
00:16:03.220in the AI research world that you should only ever be using your technical mind and you should
00:16:08.460leave behind anything that suggests that you are also like a person with a body that exists in
00:16:16.240society. And so even as people within the community feel that there are certain things about AI
00:16:22.880research that don't feel quite right like it often can enable surveillance it often can then police0.76
00:16:28.660queer bodies and things like that that somehow this this needs to be separate from from their
00:16:34.240mind and part of the problem she diagnosed was because of this very overly simplistic question
00:16:41.540of people just asking like is ai good or is ai bad um and so her provocation was this is an
00:16:49.380impossible question to answer and that's part of the reason why we just like end up not grappling
00:16:54.280with this and we just when we're an AI researcher we put on a totally different hat and then we take
00:16:58.940it off to like exist in society and the better question is does this particular AI tool or
00:17:06.140practice or artifact shift power in ways that consolidate it in the hands of the few or in
00:17:14.820ways that distribute it in the hands of the many. And once you are more granular about that
00:17:20.220question, then it becomes a lot easier to figure out like on a day to day level. And she was
00:17:26.620talking to AI researchers, but I think this is applicable to literally everyone. Like it becomes
00:17:30.560more clear on a day to day level, like whether you want to engage in certain types of AI
00:17:36.680technologies, either as a producer of this technology, like tech companies rope us in as
00:17:42.900data donors to the training of these ai models as um as hosters of their data centers and so on and
00:17:50.200so forth or as a consumer of these technologies um and i it's it's a talk that has always stuck
00:17:56.540with me and a question that's always stuck with me because it just makes it so clear when
00:18:02.720for not just ai any technology like whether it like i personally want to engage with it or not
00:18:11.040or like whether other people should be engaging with a certain technology or not,
00:18:14.820because it makes like it immediately clarifies the idea of that technology's role in society.
00:18:21.360Is it ultimately like fortifying democracy or eroding it?
00:18:25.040Karen is one of the experts that I spoke to while researching my own audio book, Love at First Prompt,
00:18:30.340a project that also took me deep into the world of people using AI for companionship and intimacy.
00:18:37.480Many of them told me that they felt like AI had genuinely helped them.
00:18:42.220So how do we hold that alongside everything we know about the risks,
00:18:46.780the threats to our democracy, our environment, and beyond?
00:18:51.200In this audio project, we've talked with a lot of people who,
00:18:55.160individuals who personally get a lot of value from their chatbot companions
00:19:00.480in terms of self-reflection, emotional support,
00:19:03.360overcoming interpersonal challenges with other humans in their life and in other areas and it's
00:19:09.180it's really interesting to think about the distinction between those individual benefits
00:19:13.460that they feel and then the societal impacts and what it means for democracy um i guess you know
00:19:19.900given everything that that you know about these systems and how they're built and who profits
00:19:23.840how do you how do you think about that you know sort of balancing those those different levels
00:19:30.320of people who feel that they are individually benefit
00:19:33.720and individually benefiting from sharing these intimate parts of their lives
00:19:39.200versus the sort of more societal level impacts to privacy, democracy, power, all of that.
00:19:48.840Yeah, I guess I wouldn't even frame it as a trade-off of like the individual benefiting,
00:19:52.300but then society having these negative externalities.
00:19:59.200I think the individual themselves is also potentially going to have negative impacts from their actions as well.
00:20:07.120Like in the short term, when people are developing relationships with these chatbots and then divulging super intimate information, maybe they do in that moment feel like they gained something important from that specific interaction.
00:20:20.740But they also lost an extraordinary amount of privacy in that moment that could, in the long run, come back to bite them.
00:20:29.200You know, like one of the things that I think is very distinctive about these chatbots versus about something like Google search is like in the past, people were not uploading their medical files to Google search, but they do just seamlessly upload their medical files to ChatGPT, thinking that this is this is going to ultimately give them some benefit in the short term.
00:20:51.460And there's not really like any guardrails right now for where that information is going to go.
00:20:57.360I mean, companies like, and in fact, Open Ananthropic very recently rolled out healthcare features to continue to encourage people to upload this stuff.
00:21:08.360And they say in their advertisements, this is going to be a place for you to upload all of your medical records to our platforms.
00:21:16.220And for now, they say, we do not use any of this information for training.
00:21:20.800We're going to secure it in a different way than your other chat conversations.
00:21:24.580But this is completely based on like trust in these companies and who's to say for that user in the long run how those policies might change.
00:21:33.260And then suddenly all of the intimate information that they've provided is used against them.
00:22:47.600when i was 14 years old i was kidnapped and held captive for nine months i survived and i've spent
00:22:55.460my life exploring how other people survive what should have destroyed them i'm elizabeth smart
00:23:01.480and these are the survivor files i just remember this low taunting voice next to my ear saying
00:23:09.140shut up don't say anything every week i'm with survivors who live through the unthinkable
00:23:16.620I knew if he woke up, without a doubt, he was going to hurt me.0.89
00:23:21.080I started feeling that there was someone at the end of my bed, and I just started screaming.
00:23:28.700They are abducted, stalked, controlled, and nearly silenced.
00:23:32.600But these aren't stories about what's taken from them.
00:23:36.060They're stories about what it takes to make it out alive.
00:23:39.180Listen to The Survivor Files with Elizabeth Smart on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:23:48.120If your bookshelf and your For You page are equally important to your personality, welcome home.
00:23:54.060This is Prose Society, the weekly podcast that's part book club, part group chat for thought daughters, pop culture obsessives, and anyone who thinks Pride and Prejudice and Love Island deserve the same level of discourse.
00:24:06.540I'm Eli Rallo, and every week we're connecting the dots between books, the internet, and the conversations everyone can't stop having.
00:24:13.640I'm going to have to look up this story. I'm obsessed, so obsessed.
00:24:17.240From best-selling authors and your favorite book talk creators to the latest pop culture moments, nothing is off the table.
00:24:23.180It's like if you can hide some real messages inside compelling characters, then that is a Trojan horse.
00:24:30.660Whether you're looking for literary deep dives, smart pop culture conversations, or a community of readers who love to think a little too much, you're in the right place.
00:24:40.260Listen to Pro Society on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:25:57.580Earlier this year, OpenAI announced they'd be rolling out ChatGPT Health,
00:26:02.380a new service that would encourage people to upload their medical records
00:26:06.200and create an ecosystem for other health apps like fitness trackers to interface with it.
00:26:12.400OpenAI is hyping this up as a one-stop shop for integrating all data about your health.
00:26:18.260According to the company, more than 40 million people ask ChatGPT
00:26:22.080a health care related question every single day, which amounts to more than 5% of all global
00:26:27.840messages on the platform. So why not create a dedicated tab for people's health questions and
00:26:34.120health needs? What could go wrong, right? I saw this woman on threads talking about how
00:26:40.680she had screenshotted a message she had gotten from her doctor, her cancer doctor,
00:26:45.320that was just this dashed off sentence, right? It was like, oh, scans came back. Let's discuss
00:26:50.380next appointment. And she was like, oh, my next appointment is in three months. Then it was the
00:26:54.120image that she had put her scans in the chat GPT. And chat GPT was like, I'm really sorry you're
00:27:01.140dealing with this. This can be really scary. I think your tests indicate da-da-da-da. And I
00:27:06.940thought, why is this company really exploiting people who are fearful or vulnerable or scared
00:27:13.260or, you know, up against navigating a health care system that can sometimes be awful.
00:27:19.540Are there ways that that tech companies are offering people something saying this is going to be helpful for you?
00:27:27.260And really, they're just benefiting from a bad system we're all trying to navigate.
00:27:32.260I've been thinking about this a lot with the rolling out of the health care features in particular,
00:27:37.180because I've talked with so many people who have had exactly this experience,
00:27:40.480Like they get a diagnosis for themselves or for a loved one. And all of a sudden, like everything feels really overwhelming. And and interestingly, I almost exclusively hear this among Americans, this experience.
00:27:53.140So 100 percent, like it's also tied to the fact that like we have this health care system that is just impenetrable and horrible to navigate and makes you feel really isolated and makes you like start worrying about your finances and everything.
00:28:08.620um and there is like something to be said that like in this moment of great need there suddenly
00:28:16.900is this tool that appears that like helps you sort through that and helps you navigate that and and
00:28:21.060like there's i i could never say that um these people should not be using a tool that's that
00:28:28.940could make such a convoluted process extremely at least a little bit more um
00:28:35.660helpful and navigable but 100 like these companies know that's exactly what's happening
00:28:44.680like they are kind of tapping into these moments of vulnerability to get
00:28:49.680you to develop more dependence on their tools and not just um dependence in terms of like
00:28:56.580uh you know just like the practical dependence but like emotional dependence as well right
00:29:03.280like they specifically design the tools to to pepper in those comments like oh this is so hard
00:29:11.120like let me let me help you with that let me I'm here for you what you know and like those are all
00:29:17.580design decisions that they layer in to make this a holistic part of the experience so I think that
00:29:23.860the way that I kind of fall on this issue is that like it's not that these tools shouldn't exist and
00:29:29.200And clearly because the numbers show that like people just like there are a lot of people that use these tools in this way.
00:29:41.240So it's to me like the solution is we should have these tools, but they need to be developed safely.
00:29:47.820And part of this is going to be that they have to be regulated.
00:29:50.400like most of the medical system and most ways that people interface with the healthcare system
00:29:57.600in general are heavily regulated. Like the medications that you take every time you go to
00:30:01.680the doctor, like all of your exchanges are protected by the law. And this is like a weird
00:30:08.680moment in which suddenly these companies are offering many of the same services, but without
00:30:14.320any of the protections. And they're, of course, also at the same time, like lobbying against
00:30:20.340getting those protections implemented. But yeah, that's like, to me, the healthy balance that we
00:30:25.720should be trying to move towards is that we have these tools available for people to help them when
00:30:30.580they desperately need it, but also in a way that is safe, where people are also not getting harmed
00:30:37.240along the way. That's such a good point, right? That so much of the healthcare industry and the
00:30:42.180way that we experience it, is heavily regulated. And now, all of a sudden, we've got these completely
00:30:47.380unregulated AI companies and products they make flooding that space. Like you said, people's high
00:30:53.880level of interest using AI to help manage their healthcare needs does suggest that it might be
00:30:58.760filling a need that people have for more information about their health, more support.
00:31:03.280But there are obviously so many huge risks. Risks to privacy, risks of getting bad or just flat out
00:31:10.200incorrect or even dangerous advice, risks of being exploited during a vulnerable moment,
00:31:15.460you know, just to name a few. So I'm curious, in your mind, what would a better system look like?
00:31:20.260Like, how should regulators and lawmakers be thinking of the role of AI in healthcare?
00:31:25.080The way that I think about it is like, first and foremost, we should be thinking much more broadly
00:31:30.420about what constitutes AI regulation. I think most of the time when people think about AI
00:31:34.680regulation, they're imagining just regulating the applications once AI is developed and how
00:31:41.400it's allowed to be used. I think that AI regulation needs to be brought into us to
00:31:47.440think about how AI, what kinds of AI should be created in the first place. So that means like
00:31:53.920we should be having more regulation on the data that's allowed to go into these AI models on
00:31:59.280where data centers get developed for training these models, how much energy and water are
00:32:05.140they allowed to use and how much are they allowed to hike up the utility prices of customers
00:32:09.460while they're training these models. And I also think that we should be regulating the
00:32:15.800applications and that, you know, if they're going to enter into the healthcare industry
00:32:20.920and people are going to start using these tools as a therapist, I mean, usually a human therapist
00:32:27.280has to get licensed by a body and has to be recognized as actually able to practice therapy.
00:32:37.100And so that's another piece of like for that specific industry, if AI companies are going
00:32:43.200to position their products as therapists, they should be regulated just or they should
00:32:49.060be licensed just as human therapists are licensed.
00:32:52.200So it really does, I think, depend case by case on which, you know, the vast facet faces of AI that we're talking about.
00:33:04.260But one of the things that I think would cut across like every just as a baseline we should be thinking about when it comes to AI regulation is we just generally need more transparency across the entire AI development and deployment supply chain.
00:33:20.240like we currently don't know what data is being used to train these models we currently don't
00:33:26.340know the energy footprint of these data centers we often don't know when you're going to the
00:33:31.460doctor's office whether or not ai is actually being used on you because there aren't um super
00:33:38.300robust disclosure laws where doctors or anyone who's using ai has to just disclose that they
00:33:45.120are doing so on the person that they're using it on. We also don't know, like, if you're at the
00:33:50.160doctor's office and they're using a specific AI application, what's actually running in the
00:33:56.020background? Because there's applications and then there's the models that power them. And it could
00:34:01.260turn out that that model is run by Google or by Microsoft or by Amazon or whatever it is.
00:34:06.820And you currently don't have control over that as a patient or just as a person existing in society.
00:34:13.760And in general, as like the first step for improving the rights of users and citizens engaging in an AI-enabled society, like transparency is number one.
00:34:29.940When you put it that way, it is really crazy that we're tolerating this.
00:34:33.040Like I like it is if if if we were to go back in time 20 years and you were to explain to me what would be commonplace just the way that you just did, I would say no, that we wouldn't know.
00:34:45.460We wouldn't stand for that. Nobody would volunteer or sign up for that.
00:34:48.620I don't know. Sometimes when I hear what these tech, the way that these tech companies are framing, that they'll be in the most intimate aspects of our lives and we would get no transparency into the way that that shows up and what that really looks like and means.
00:35:04.440I it's it blows my mind. It's really hard for me to believe that we have signed up for this and that there's people who are like, oh, it's going to be great for you. We're actually going to love it.
00:35:12.540Yeah, I mean, I think this is one of the things that like this didn't happen overnight, right? Like Silicon Valley in general over the last 10 years, very slowly built up our tolerance for this situation by building this narrative that like privacy is over and everyone would rather trade their privacy for convenience.
00:35:33.600And by making us accustomed to the idea that like we don't have any control over the fundamental building blocks of our digital lives anymore, like the things that give us information, the algorithms that sort through our new seeds.
00:35:49.120And so I think the AI conversation is being installed on already like a very solid foundation of 10 years of Silicon Valley just eroding away our rights and making us is like a frog in a boiling pot metaphor.
00:36:08.400Yeah, you know, you open the book with this stunning quote from Sam Altman, I think his blog, where he essentially says that the best way to motivate people is to build a religion. And the best way to do that is to start a company. And first of all, I feel like starting with that quote is like, okay, just here's where we're at. Like, we're starting with him talking about how he wants to start a religion.
00:36:28.860But I think when you really parse through what that means and you consider that OpenAI is building tools that people use in these intimate parts of their life, now for things like emotional support, intimate companionship, therapy, sex, all of that stuff, they're using these tools in moments of spiritual or existential need and vulnerability.
00:36:49.920What does it mean that they're being built and led by somebody who, as you point out, will use this like mission driven language as a strategic tool as opposed to like a genuine commitment?
00:37:02.380What does it mean that somebody is at the helm of all of these intimate things who, you know, there are plenty of concerns about his honesty and his approaches to safety?
00:37:14.780So I think for me, there's an even bigger meta question, which is just like, what does it mean that we allow one individual, regardless of their character, to do all these things and have access to all these things and have like a window, an intimate window into so many people's lives?
00:37:31.220like even if we were to swap out Sam Altman for someone else, I still don't think this setup is
00:37:36.640okay, right? Like there's just one person or a small group of people that are running these
00:37:41.240companies and they can make 60 decisions in an hour that fundamentally changes how
00:37:48.140billions of people are engaging with their technologies on a day-to-day basis, which then
00:37:54.080affects their lives and their work and their schools and their healthcare. And, you know,
00:38:00.200it's just like it's like that is just inherently unsound that is inherently an undemocratic setup
00:38:09.820and we cannot have uh we cannot be talking about you know existing in a democratic society where
00:38:16.880everyone has is supposed to have agency and control over their future decisions and be
00:38:22.580able to collectively self-govern when in in reality governance is just happening by a tiny
00:38:28.280group of people. Yeah, I really appreciate how much your writing connects with democracy. I feel
00:38:34.640like a lot of the experts that we've talked with about this are experts in different fields. But
00:38:42.000they've been, you know, it's been a lot of discussion about, a lot of the discussions
00:38:47.260we've had have been about intimacy and relationships. But as you point out, I think
00:38:53.460democracy and the implications for democracy of this these rising ai companies is enormous0.98
00:39:01.800do you do you view that you know democracy as like a casualty of these companies building
00:39:12.220uh building empires or do you think it's sort of part and parcel of the same thing undermining
00:39:19.660democracy in, you know, as part of this process of consolidating power and money?
00:39:26.480I think it's kind of a little bit of both. I mean, there's certain players in Silicon Valley
00:39:31.980that have been extremely explicit about how they just don't believe in democracy and think that
00:39:37.580we need to move to a different way of organizing society. Like Peter Thiel has said this very
00:39:42.120explicitly he thinks that democracy is incompatible with um like a good society um
00:39:48.900and also there are other people that i think have who have not been so explicit about their desire
00:39:56.420to undermine democracy but more in their quest for uh you know their quest for wanting to scale
00:40:05.340monopolistic companies their quest for wanting to build an ai like god their quest for
00:40:11.760wanting to become like a great man of history like in that journey yes democracy also then
00:40:21.340becomes a casualty because in order to do those things what they're suggesting is consuming all
00:40:27.220the resources in the world and undermining everyone's agency to get there so that yes
00:40:34.380you could say that maybe therefore there's not actually a difference whether or not they intended
00:40:40.300explicitly to undermine democracy or not like that's the final destination of what they what
00:40:48.040their agendas are yeah to that point you've talked about how open ai their core insight is just the
00:40:55.080concept of more more data more resources but when i think about them pivoting more into like an
00:41:03.100intimacy or emotional support space which i know that they're aware that that's how people are
00:41:06.740using their product, it seems like those are spaces that require the opposite of more,
00:41:23.980It's funny because I don't think from the AI researcher's perspective, there's a tension.
00:41:28.300They would say that in order to have more to deliver more high context care via chatbot, they just need more and more and more and more of your intimate data.
00:41:41.080So so like for them, it's not attention.
00:41:45.020And that's that's why they are motivated to collect more and more and more of this stuff.
00:41:51.260um but yeah i mean like to me what the tension is is that there is a certain worldview that
00:41:59.540undergirds the development of these systems that suggests that everything in our existence can
00:42:04.900ultimately be quantified through data and can ultimately be uh mediated through technical
00:42:13.380systems. And that worldview really devalues human to human interaction, really devalues the soft
00:42:22.860social emotional element of relationships and society. And that is the tension for me is like
00:42:33.340they're trying to suggest that there will be an AI god that they can develop purely through
00:42:40.820quantitative methods, purely through consuming data in technical systems that is somehow going
00:42:47.380to manage to replace all of the facets of emotional, social, human relationships that
00:42:55.300have been driving society for millennia. And this just goes back to my ultimate personal
00:43:02.720orientation around tech, which is that I think that it is being led and driven by people who
00:43:09.540devalue emotional labor you know soft skills like we like we've had conversations for a really long
00:43:16.980time about how the only thing that matters are these like hard skills quantifiable skills and
00:43:23.120these are people who are trying who were meant to trust are going to build the kind of world
00:43:27.760that we want to live in using their technology who have never valued these things and that they're
00:43:33.500building technology that sort of argues that you don't even really need them when we know that we
00:43:39.120do need care work and emotional work. It's work that they devalue and do not respect. So why then
00:43:44.040would I put my trust in them to use technology to build a kind of world that anybody might want to
00:43:48.460actually live in? Yeah. And the thing is, like, they also actually value these things without
00:43:55.100explicitly recognizing, like, they actually value it more than they articulate. Because,
00:44:01.460for example, all of these tech companies have policies that require their employees to come
00:44:05.500into the office. Like, why would you require your employees to be in the same room with one another
00:44:11.700working face to face unless you believed that there is genuine value to that interaction that
00:44:18.540you could not get from being remote? It's like kind of like the greatest irony is that
00:44:23.860they actually design their companies and the way that they work and the way that they make
00:44:33.020decisions in their lives in direct opposition to the things that they say um and i think it also
00:44:41.600you know like that the neurops conference that i was talking about like 15 000 people descending
00:44:45.720into one city just to have face-to-face time with one another and these are the people that
00:44:51.000are building these technologies that they then suggest will be replacing all of that human
00:44:55.840interaction. So yeah, it's interesting. I think it's one of those situations where you have to
00:45:03.100like see what they do rather than listen to what they say. More after a quick break.
00:45:18.000Behind every night spent making the kids lunches, there's a lot of suspense. Will they finally eat
00:45:23.120the veggies will the pesto survive the first bite will the lunchbox come home empty half full or
00:45:28.780come home at all and will you finish episode eight before you're done slicing the cucumbers
00:45:33.140get bell pure fiber internet with crave netflix and disney plus from 94 a month price guaranteed
00:45:38.940for two years on internet with a two-year term and auto pay credit visit bell.ca for details
00:45:43.860and to check availability bell connection is everything run a business and not thinking about
00:45:49.260podcasting? Think again. More Americans listen to podcasts than ad-supported streaming music
00:45:54.280from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next
00:45:59.120two combined. So whatever your customers listen to, they'll hear your message. Plus, only iHeart
00:46:03.920can extend your message to audiences across broadcast radio. Think podcasting can help your
00:46:08.400business? Think iHeart. Streaming, radio, and podcasting. Call 844-844-iHeart to get started.
00:46:41.940Every week, I'm with survivors who live through the unthinkable.0.98
00:46:46.480I knew if he woke up, without a doubt, he was going to hurt me.
00:46:51.060I started feeling that there was someone at the end of my bed, and I just started screaming.
00:46:58.500They are abducted, stalked, controlled, and nearly silenced.
00:47:02.440But these aren't stories about what's taken from them.
00:47:05.880They're stories about what it takes to make it out alive.
00:47:09.020Listen to The Survivor Files with Elizabeth Smart on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:47:17.980If your bookshelf and your For You page are equally important to your personality, welcome home.
00:47:23.920This is Prose Society, the weekly podcast that's part book club, part group chat for thought daughters, pop culture obsessives, and anyone who thinks Pride and Prejudice and Love Island deserve the same level of discourse.
00:47:36.400I'm Eli Rallo, and every week we're connecting the dots between books, the internet, and the conversations everyone can't stop having.
00:47:43.520I'm going to have to look up this story. I'm obsessed. I'm obsessed.
00:47:47.080From best-selling authors and your favorite book talk creators to the latest pop culture moments, nothing is off the table.
00:47:53.040It's like if you can hide some real messages inside compelling characters, then that is a Trojan horse.
00:48:00.500Whether you're looking for literary deep dives, smart pop culture conversations, or a community of readers who love to think a little too much, you're in the right place.
00:48:10.120Listen to Pro Society on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:51:07.020integrated shopping experience into ChatGPT.
00:51:09.780They've announced that they are going to build some kind of hardware device.
00:51:16.220They're just spraying the market with various different ideas.
00:51:19.620And a core part of being able to then make all of these into revenue generation engines is users.
00:51:28.760They need to continue having more users and they need to have those users spend more time on the platform.
00:51:35.420And the truth is, OpenAI is actually losing market share right now because there are other competitors like Anthropic, like Google that are starting to eat their lunch.
00:51:48.280And so as they experience all of these pressures from competitors and they experience just like the dire need for more cash flow, they keep flip flopping on decisions to try and figure out what is going to make their product more engaging, more sticky and more addictive.
00:52:09.580To that end, I mean, you've talked about how AI is essentially showing us that, like, surveillance capitalism moving into its most extreme form.
00:52:19.680And as I've been reporting my own book, like, I'm curious, now that we're at a place where it's in people's emotional lives, their relationships, their mental health struggles, they're putting ads in the chat.
00:52:30.340GPT is have we just like morphed into a into like I would say final form but I don't think it will
00:52:37.300be the final form of just the most extreme version of AI enabled surveillance capitalism
00:52:42.600pretty much I mean you see this all the time with the announcements that companies are making where
00:52:50.660like over the last year maybe year and a half there were these slew of announcements from
00:52:58.260various companies where they just said we will now be training on your data unless you opt out
00:53:03.780like zoom made an announcement saying we're now going to train on your video calls of course
00:53:08.280there was a huge backlash so that didn't end up happening but then like facebook announced oh
00:53:14.060we're changing all of our settings so we're going to train on everything now including your public
00:53:17.680instagram posts linkedin made that announcement like we're going to train on your linkedin posts
00:53:22.400And unless you opt out. And like that is, to me, the the most obvious sign of this surveillance capitalism creep, like before they were already monetizing off of the data that we were leaving on there, these digital trails that we were leaving online.
00:53:39.920But now they're just becoming like so much more explicit about like any last vestige of data that they have not yet used and trained, like fed into their models and tried to like wring money out of.
00:53:54.620They are now grabbing and it is now become incumbent on the user to track all of these updates and then go into the settings and like find them and turn them all off.
00:54:06.140And that's just the tip of the iceberg for the much broader expansion of surveillance capitalism that's happening beneath the surface now.
00:54:13.140I see AI as this inherently extractive dynamic where everything is being taken, whether it's our data or our privacy. And I just feel like that in some ways is fundamentally at odds with then trusting those same companies to automate care, automate connection, automate intimacy.
00:54:35.280And one of the ways that I really see that is in some of the folks that you've interviewed, like content moderators in Kenya who suffered like pretty intense psychological harm filtering out violent content.
00:54:48.100And so it's just hard for me not to see this as a fundamentally extractive, exploitative dynamic, but then a dynamic where people are telling us, oh, you're going to be able to trust this to, you know, create a better world.
00:55:03.040How can you create a better world with a tool that fundamentally might have exploitation and extraction at the heart?
00:55:09.000One of my theories for why these companies have been so successful this far is because they have very successfully hid a lot of the exploitation and extraction.
00:55:19.720Like, you know, the AI industry in general or the tech industry in general, they use a lot of euphemisms in the way that they talk about AI and what they're building.
00:55:30.120You know, like data centers are the cloud. It's like an ethereal thing that exists in the sky, not giant sweaty computers that are sucking up an enormous amount of energy. And they talk about how autonomous their agents and their models are, not actually acknowledging the fact that it's built on the backs of tens of thousands of contract workers that live in places like Kenya.
00:55:54.580And so, yeah, like I think they create this veneer of magic, of mysticism, of the fact that this technology somehow falls from the heavens and therefore then benefit from the fact that a lot of consumers just don't realize when they're engaging with these.
00:56:14.900So like they just see the kind of clean interface and those consumers just evaluate the technology based on, oh, like that was helpful.
00:56:23.940So like, let me use it again without actually considering all of the things that went into building the technology that then reveal the logic of the industry and what that industry will then, how that industry will continue to apply that logic to your data and your life.
00:56:39.820Do you think that this is just another iteration of wealthier countries using, because there are so many examples like the fast fashion industry, or there are just so many examples of industries where we're just trained in the West to just take and use and not think about what the labor and the people that went into this thing that we are taking and using.
00:57:02.940So I'm curious when it comes to how some of the use cases that people are using AI for, is this the same thing where people are just taking and using without thinking about who's on the other end doing the labor?
00:57:15.460Yeah, 100 percent. I think AI, the industry does engage in a very it's part of it's part of the long history of exploitation, extraction of many different industries.
00:57:27.160But I think the difference is that with something like fashion or with food, it's much more obvious to the consumer that there is a supply chain that exists because it is a physical object that you hold in your hand.
00:57:42.460And so you know that at some point there were materials that were fed into building this object and there were people involved in laboring to create this object.
00:57:52.580Whereas AI as a digital thing, like a lot of people still don't have that connection that this digital thing actually exists physically in the world and also requires human labor.
00:58:04.440And I think the other thing is like with something like fashion, you know, people are the industry is not telling you that they are creating God. They're not telling they're not selling you this over the top religious narrative about why you need to buy their clothing.
00:58:24.400And that's what the AI industry is doing, right? Like they are, they are not just hiding the exploitation extraction, but they're also packaging it in this, like this like crazy rhetoric where they say that if you engage and allow them to build these tools, they're not just tools, in fact, that they're going to bring us civilization 2.0, solve all of our problems, bring abundance to the whole world and everything is going to be amazing.
00:58:53.640And I think that then accelerates the exploitation and extraction because it justifies it. So even when the exploitation or the extraction is revealed, the people who benefit from this like magical or want to benefit from this magical oracle that exists at the end of the journey are willing to then accept the fact that maybe there is some exploitation and extraction that needs to happen along the way.
00:59:22.520that's such a good way to put it and you know like I may buy fast fashion from Shein the head
00:59:31.760of Shein is not on television talking about why I need to be personally invested in him being
00:59:37.060successful like the the empire that he's building I'm not it's such a good point that like it's kind
00:59:42.620of crazy when you think about it yeah yeah yeah no it's it's it's it's really nuts and I I think
00:59:50.780Like sometimes I use this analogy that like if we were like back in the Middle Ages and someone knocked on your door and was like, I'm going to sell you a potion that can cure all of your problems, but it's just going to cost everything you've ever owned, including your first born child.
01:00:07.920You would be like, this is a scam. No. And that's what these companies do. And yet, like everyone was is like, OK, take my first born child.
01:00:20.780And there's just something about like the the modernity and the sexiness and like the advanced technical aspect of AI that makes it much harder to recognize that that's actually like what these companies are saying to us.
01:00:38.380Karen, Empire of AI has been such a huge success. You are now launching the Interface with BBC. Tell us how that came about. And I'm curious if it's a natural progression of the work that you explored in your book.
01:00:50.780Yeah, so The Interface is a podcast that is going to be a weekly show with me and two other hosts, Thomas Jermaine and Nikki Wolf, both also longtime tech investigative reporters.
01:01:05.520And we are going to be talking each week about different topics that intersect with tech, but are not exclusively about tech, because our thesis is that tech is the dominant driving force that's rewiring your world.
01:01:20.780And in order to really understand all the things that are happening in the world, whether it's politics, geopolitics, the environment, or just the crazy things happening on your phone, like you have to have a grasp of technology.
01:01:34.480And we want to make this show as broad as possible.
01:01:38.020It's not meant for just people that love tech or just people that understand tech.
01:01:43.300Like we really, really want everyone to feel like this is a show for them.
01:01:48.180and it very much is an extension of the work