00:02:23.940According to a study from Deep Trace Labs, 96% of deep fakes are non-consensual sexual depictions of women.
00:02:31.640So we're calling these images deep fakes, which is not wrong,
00:02:34.620but one technical thing that is important to understand about them,
00:02:38.160is that we used to use the term deepfakes to describe content where someone's head or was put on another person's body
00:02:45.560to give the impression that that person is nude or doing something sexually explicit.
00:02:50.200These Taylor Swift deepfakes are different in that they were created out of whole cloth using commercial AI tools.
00:02:57.860The images originated on the alternative messaging platform Telegram, then made their way to 4chan.
00:03:03.440This Telegram group was actually kind of a marketplace for celebrity deepfakes,
00:03:07.660where users were requesting and trading deepfake images of specific celebrities since last year.
00:03:13.780Now, most commercial AI image generators do have some safeguards in place that are meant to prevent
00:03:19.800abuse. So if you ask AI to generate an image that depicts a sexualized image of a child,
00:03:25.580for instance, it is not supposed to do that. But people are able to circumvent the moderation to
00:03:30.940generate non-consensual deepfake images, like the ones we saw of Taylor Swift, pretty easily.
00:03:35.780I have to give a major shout out to 404 Media right here because they did a great job of digging and found that those images of Taylor Swift were likely made on Microsoft's AI creation tool called Designer.
00:03:49.180So Designer does have some safeguards in place to prevent the platform from being used to create this kind of content of celebrities.
00:03:56.140However, 404 Media ran a couple of tests and found that designer will not generate an image if you just put Jennifer Aniston.
00:04:03.000But they were able to generate suggestive images of Jennifer Aniston if you use the phrase Jennifer Actor Aniston.
00:04:09.720So just put actor in between Jennifer and Aniston and boom, non-consensual deepfakes.
00:04:15.200So prior to these deepfake images of Taylor Swift going viral on Twitter, a user in that Telegram group recommended that members who were trying to make sexually explicit images of Taylor Swift
00:04:25.300You use the phrase Taylor Singer Swift instead of just Taylor Swift to generate those images.
00:04:31.360Basically, anyone can get around these guardrails pretty easily.
00:04:36.800And to get deep fakes of celebrities engaging in what looks like sex acts, all people needed to do was describe activities that kind of look like sex acts without using sexually explicit terms.
00:04:47.480Now, none of this is terribly surprising.
00:04:50.260It's the kind of thing that people have been warning is really easy to do with AI for a long time.
00:04:55.340I cannot stress how much 404 Media stayed on Microsoft's neck on this story.
00:05:01.420At first, Microsoft was kind of trying to weasel out of this and deny that their platform was being used to create these images and that their platform is where these Taylor Swift images originated.
00:05:11.760They initially said, we are investigating these reports and are taking appropriate action to address them.
00:05:16.360Our code of conduct prohibits the use of our tools for the creation of adult or non-consensual intimate content, and any repeated attempts to produce content that goes against our policies may result in a loss of access to the service.
00:05:28.140We have large teams working on the development of guardrails and other safety systems in line with our responsible AI principles, including content filtering, operational monitoring, and abuse detection to mitigate misuse of systems and help create a safer environment for users.
00:05:41.200So after that, Microsoft said they did an investigation, but they were unable to confirm the images were made using their tool.
00:05:48.100But after 404's reporting, Microsoft did close loopholes that would have allowed for these images to be created.
00:05:55.580404 Media was able to verify that those loopholes did work to create these kinds of images on Thursday before Microsoft made its changes.
00:06:03.620And that those loopholes stopped working after 404 Media published an article on it.
00:06:08.140So it just really does seem like Microsoft was the culprit.
00:06:11.900And it's a really important reminder of the importance of good journalism to keep people accountable for the harms that their technology causes.
00:06:18.300Because it would have been really easy for Microsoft to just keep denying that they were responsible for these images and just quietly close that loophole that allowed for images like the ones of Taylor Swift to be made on their platform.
00:06:30.280After this change, users on Telegram were complaining that it seemed like the loophole had been fixed that allowed for them to make these celebrity deepfakes.
00:06:38.900But unfortunately, that Telegram group where people were requesting and trading non-consensual deepfake images of celebrities is still active because Telegram does not really remove people who use their platform to do this kind of thing.
00:06:52.040And people on that platform are already talking about what AI tools they're going to use to continue creating non-consensual deepfake porn of celebrities in the future, since Microsoft closed the loophole that allowed them to use their platform.
00:07:06.340So they're just going to keep doing this, and it is shockingly easy for them to keep doing.
00:07:10.980So these Taylor Swift images had been on Telegram and 4chan for a while, but one specific user who subscribes to Twitter Blue, so has a verified checkmark and likely has their content algorithmically boosted on Twitter, brought them over to Twitter.
00:07:24.820It took Twitter 17 hours to suspend that user and delete that user's posts with the Taylor Swift images.
00:07:32.500However, because of the way the internet works, those images were still all over Twitter.
00:07:38.820Like just suspending that one user does not mean all those images are going to go away too.
00:07:43.500So the way that Twitter handled this whole thing, I think is just abysmal and also really telling.
00:07:48.860First, I guess because those images were really all over Twitter,
00:07:52.620they just tried to fix it by blocking people from being able to search Taylor Swift temporarily.
00:07:58.360Joe Benarach, head of business operations at Twitter, said,
00:08:01.540this is a temporary action and done with an abundance of caution as we prioritize safety on this issue.
00:08:06.640But because it's Twitter, this was actually super easily circumvented.
00:08:11.620Attempts to search for Taylor Swift's name without quotation marks on Twitter on Monday
00:08:42.940have to have their name blocked on social media search?
00:08:46.060Like, what if this happened to somebody who is not famous?
00:08:49.240You know, I'm thinking of content creators and journalists, people who need to be able to show up on those platforms for their careers.
00:08:56.220It's basically like saying that when you are targeted for this kind of thing, the only way to keep you safe in these online spaces is to obscure your presence there.
00:09:04.800Which I take issue with because participation in online platforms is increasingly linked to how people just show up in 2024, socially, politically, and civically.
00:09:13.700So if the response to this is to make targeted people, most of whom are women, show up less in those spaces, it means that they will be even more marginalized online, a space where women are already marginalized.
00:09:26.100It just feels like the fix is to further obscure and silence women's voices in a space where we are already obscured and silenced.
00:11:44.780Elon Musk said that Twitter was going to hire 100 full-time staffers to do content moderation to combat sexually abusive material on Twitter.
00:11:53.000He said that he would be building a trust and safety center in Austin, Texas, and hire in-house agents who will be tasked with enforcing the site's content and safety rules.
00:12:02.580So I don't even have time to get into how annoying this is to me, because basically the very same teams that Elon fired right after he took over at Twitter in 2022, he's being like, oh, whoops, actually, we needed those teams.
00:12:16.120Those teams actually did something important.
00:12:17.740So Twitter released this whole thing this week about how they were taking really seriously sexually exploitative content, particularly content that involved children.
00:12:25.140And you might remember that for whatever reason a while back, Elon Musk made combating child sexual abuse material like his thing, like he started talking about it and grandstanding about how much work he did to fight it.
00:12:35.880However, Wired reported that back in 2022, when he took over at Twitter, even though he said that removing child exploitation was his, quote, number one priority, after all the firings and resignations, just one sole staff member remained on a key team dedicated to removing child sexual abuse content from Twitter.
00:12:55.140And remember, he also reinstated the account of a right-wing provocateur who posted child exploitation material on Twitter, and they explained that decision to reinstate his account by saying, well, if abusive sexual child content is posted on Twitter to express outrage, then it's okay.
00:13:13.220Basically what I'm saying is that what Elon Musk says he cares about doing and what he actually does in practice are two different things.
00:13:20.220You know, it was his choice to gut the trust and safety team and content moderation teams when he took over at Twitter.
00:13:25.900So now in classic Elon form, he's backtracking on that great big mistake.
00:13:31.460This is actually a pattern for Elon Musk.
00:13:33.080It's like he comes in, gets rid of everything that people know that you need to make a platform work, realizes, oh, crap, turns out there was a reason for having that and we actually need it, reinstates it, and then I guess just like stands back and calls that innovation.
00:13:46.420When he first took over at Twitter, he immediately let people who had been previously spended for abusive content or hate speech back on the platform.
00:13:55.040People like Kanye West, who Elon Musk then had to resuspend after Kanye West tweeted out a swastika.
00:14:04.180I just can't believe he would announce creating content moderation teams and trust and safety teams like it's some new innovation that he has come up with, as opposed to something that was already in place at Twitter that he got rid of and now is reinstating.
00:14:20.000And it's funny because I was talking to producer Mike about this, and he made a very good point, which is that all of this assumes that Elon will actually follow his word and do the thing that he says he's going to do.
00:14:29.800So will he actually create this content moderation team? Will he actually create a trust and safety team? Who knows? Because Elon has such a poor track record of actually doing the things he promises to do.
00:14:42.800So there's been this huge reaction about these Taylor Swift deepfakes, and rightly so, but they are not the first time this has happened.
00:14:49.920We've covered how this kind of thing is happening at schools across the country in places like New Jersey and Colorado,
00:14:55.120and how they've been dealing with groups of male students essentially creating AI deepfake extortion rings of the girls in their school,
00:15:02.760where the boys in the school are invited to pay to see deepfakes nude images of their classmates,
00:15:08.080And the girls are basically forced into being extorted to pay money to avoid their images being circulated.
00:15:15.420In New Jersey, high school student Francesca Mani, a teenage victim of non-consensual sexually explicit deepfakes, joined Representative Jo Morel to tell her story and advocate for legislation that would make this kind of thing a crime.
00:15:27.860And teen Marvel star Social Gomez was targeted with non-consensual deepfakes earlier this year.
00:15:37.240It's really heartbreaking because she says that Twitter pretty much did nothing to take these images down.
00:15:43.660They did not respond to requests for comment about it.
00:15:46.180And that she said in a podcast that she basically had to make personal peace with the fact that these deep faked images of her are out there.
00:15:53.400Like, imagine what we are asking young people, children, minors to make individual peace with because the powers that be are not taking it seriously or doing anything to prevent this kind of thing from happening.
00:16:05.780or dealing with it when it does happen.
00:16:08.440And I do want to say something else about Sochel Gomez.
00:16:10.800We've talked about this a bit on the show before,
00:16:12.720but there does seem to be some kind of a tiered system of response
00:16:16.420depending on how famous the target is.
00:16:19.460Because certainly there was not this kind of outcry for Sochel Gomez,
00:16:23.560who was targeted for AI deep-thick images before Taylor Swift.
00:16:26.580We kind of saw something similar under the earlier iteration of Twitter
00:16:30.340where marginalized people were speaking up about
00:16:32.680getting torrents of online harassment and abuse,
00:16:35.060Like, Black women activists were really open about what they were experiencing.
00:16:39.160However, it took Saturday Night Live actor Leslie Jones to speak up on Twitter when she was facing a torrent of harassment after her Ghostbusters remake came out that Twitter took any meaningful steps to suspend the accounts of people who were doing it.
00:16:52.840Then Twitter CEO Jack Dorsey personally intervened to suspend those accounts and, like, replied to Leslie Jones saying that he was sorry that it happened.
00:17:00.780But our teens who are not famous are already being targeted for non-consensual deepfake images.
00:17:06.860You know, you shouldn't have to be Taylor Swift's level of famous to get people with power to take this seriously when we are all at risk.
00:17:14.460Now, the Taylor Swift deepfake images on Twitter have triggered a response from the Biden administration.
00:17:19.760At a White House press briefing, a reporter asked if Biden supported legislation to ban AI-generated porn images.
00:17:25.660And White House Press Secretary Karine Jean-Pierre said,
00:17:28.680we are alarmed by the reports of the circulation of false images.
00:17:32.200We are going to do what we can to deal with the issue.
00:17:34.360The Screen Actors Guild, the union that represents performers in Hollywood,
00:17:38.480has spoken up too, saying the development and dissemination of fake images,
00:17:42.360especially those of lewd nature, without someone's consent, must be made illegal.
00:17:46.680As a society, we have it in our power to control these technologies,
00:17:49.820but we must act now before it is too late.
00:17:52.020The way performers are depicted in AI without their consent
00:17:54.660has been a major point of contention with actors' unions like SAG.
00:17:58.200The union also supports broader legislation, like the No AI Frauds Act, that would prohibit the use of AI to distribute faked replicas of real people without their consent, regardless of whether those images are sexually explicit or not.
00:18:12.280We support Taylor and women everywhere who are the victims of this kind of theft of their privacy and right to autonomy.
00:18:18.660So I do have to add that there is something weird happening around Taylor Swift right now.
00:18:24.380Like, it does not surprise me that these deep fakes were first circulated on Telegram and 4chan, where far-right extremists proliferate.
00:18:32.780So just yesterday, failed Republican presidential candidate, Vivi Akramaswamy, spewed some pretty wacky conspiracy theories that Taylor Swift and her boyfriend, NFL football player Travis Kelce, were being, like, artificially, culturally propped up by the media for some kind of a Democratic, I don't know, psyop campaign.
00:18:52.500He tweeted, I wonder who's going to win the Super Bowl next month, and I wonder if there's a major presidential endorsement coming from an artificially, culturally propped up couple this fall.
00:19:02.860Taylor Swift's boyfriend plays for the Chiefs, who just won the playoff game and are going to the Super Bowl.
00:19:08.240And on Fox News this week, they called her boyfriend Mr. Pfizer because he promoted vaccination to curb COVID.
00:19:15.840So something I've always found weird about this is that Taylor Swift is not even super publicly or explicitly that political.
00:23:48.100If you want to help us grow, rate and review us on Apple Podcasts.
00:23:51.160For more podcasts from iHeartRadio, check out the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:24:21.160Every week, I'm with survivors who live through the unthinkable, abducted, stalked, controlled, and nearly silenced. These are stories about what it takes to make it out alive. Listen to The Survivor Files with Elizabeth Smart on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
00:24:41.240If your bookshelf and your For You page are equally important to your personality,
00:24:45.760welcome home. Pro Society is a weekly podcast that's part book club, part group chat for anyone
00:24:51.540who thinks Pride and Prejudice and Love Island deserve the same level of discourse. Each week,
00:24:56.120we're connecting the dots between books, the internet, and pop culture with your favorite
00:25:00.080writers, book talk creators, and plenty of overthought opinions. Yeah, I'm obsessed. I'm
00:25:05.080obsessed. Listen to Pro Society on the iHeartRadio app, Apple Podcasts, or wherever you get your