Maintenance Phase - July 20, 2021


School Lunches, P-Hacking and the Original "Pizzagate"

Topics
The Keto Diet The Body Mass Index

Episode Stats


Length

1 hour and 17 minutes

Words per minute

193.55

Word count

15,001

Sentence count

785

Harmful content

Misogyny

4

sentences flagged

Toxicity

101

sentences flagged

Hate speech

38

sentences flagged


Transcript

Transcript generated with Whisper (turbo).
Misogyny classifications generated with MilaNLProc/bert-base-uncased-ear-misogyny .
Toxicity classifications generated with s-nlp/roberta_toxicity_classifier .
Hate speech classifications generated with facebook/roberta-hate-speech-dynabench-r4-target .
Topics generated with Qwen2.5-3B-Instruct.
00:00:00.000 hi everybody and welcome to maintenance phase the podcast that will gladly trade you two
00:00:15.900 regular milk tickets for one chocolate milk ticket that's good i knew we were talking about
00:00:22.560 school lunches and i that was my one very vivid memory the podcast that serves you turkey tetrazzini
00:00:30.000 I'm Aubrey Gordon.
00:00:30.920 I am Michael Hobbs.
00:00:31.680 If you'd like to support the show, we are on Patreon at patreon.com slash maintenance phase.
00:00:35.920 We're now releasing bonus episodes so you can enjoy some bonus content.
00:00:39.960 Support the show if you want to and don't if you don't.
00:00:42.580 Keep listening and never give us a dime.
00:00:44.140 It's chill.
00:00:44.740 That's right.
00:00:45.440 And today we are talking about school lunches, I think.
00:00:49.700 Aubrey, I'm so excited.
00:00:51.360 Oh, I can't wait.
00:00:52.020 This is our first, I think, straightforward clickbait episode.
00:00:56.300 I am going to title this episode, School Lunches, P-Hacking, and the Original Pizzagate.
00:01:04.420 The Original Pizzagate?
00:01:06.240 That's like how we're drawing people in.
00:01:08.300 But the actual story that we're going to talk about today is basically the rise and fall
00:01:12.280 of a single food and nutrition researcher who was one of the most prominent people in this
00:01:18.180 field for more than a decade.
00:01:19.880 His name is Brian Wansink.
00:01:22.140 And I think it's a really good story.
00:01:23.560 it's like by far our most methodology queenie episode. But also, if we called it Brian Wandsink,
00:01:30.580 nobody would listen to it. So we've gotten you here with a catchy title. And now we're gonna
00:01:35.360 pump statistics into you. I can't wait. Also, like my knowledge of this topic runs like an
00:01:41.320 eighth of an inch deep. Excellent. The only reason the name rings a bell is a listener
00:01:46.400 sent in an email being like, I think you should do an episode about this guy.
00:01:50.980 And I told you about it when you were like, I don't know what I'm going to do for my next
00:01:54.020 episode.
00:01:54.760 And then my eyes got as big as the dinner plates in the research we're about to cover.
00:01:58.600 Well, the great and hilarious thing is that I told you about it.
00:02:01.520 I was like, apparently there's this whole like nutrition research scandal.
00:02:04.380 And you were like, oh, Brian Lansing?
00:02:05.860 Yeah.
00:02:07.340 Peak Michael Hobbs response.
00:02:08.920 So I have actually, I've been following this for a while because, full disclosure, I was
00:02:14.040 like one of the people who totally fell for this guy.
00:02:17.500 I'm not going to pretend to be above any of the biases that we're going to talk about
00:02:21.900 or any of really, I think, the structural problems in media and in academic research
00:02:27.140 that this episode is like an entry point into.
00:02:30.020 You know, he spent more than a decade being like one of the most prominent sort of brand
00:02:34.580 name researchers in this field.
00:02:36.080 He wrote two bestselling books.
00:02:37.620 He was on the sort of the TED Talk circuit.
00:02:39.800 When this entire downfall happened, the New York Times writes an article about it, this
00:02:42.980 kind of perfunctory article.
00:02:44.000 And at the end, they note that he had been quoted in 60 New York Times articles over the course of almost 20 years.
00:02:52.220 Good Lord.
00:02:53.620 So one of the frustrating things about this, honestly, is that for our main protagonist, Brian Wansink, there's actually very little information available about his early life and sort of how he got into the field of food research.
00:03:06.520 What we do know about Brian is that he starts as a marketing professor, which is that we're already in the foreshadowing section.
00:03:14.000 I was going to say, this does not bode well.
00:03:16.300 This is an excerpt from his book Mindless Eating, which comes out in 2007.
00:03:20.300 I'm never sure what to say when someone asks how I first became interested in food,
00:03:24.340 psychology, and marketing.
00:03:25.680 I usually say I really liked Vance Packard's 1957 book The Hidden Persuaders because he
00:03:31.040 tried to show how advertising unconsciously affects us.
00:03:34.200 I think this also happens when we eat, except the hidden persuaders are the way we set up
00:03:38.680 our tables, our kitchens, and our routines.
00:03:41.280 I'm going to go out on a limb and also assume that Brian Wonsink is not a fat dude.
00:03:45.800 Oh, absolutely not. 0.87
00:03:46.600 He's a skinny white guy. 1.00
00:03:47.740 He's blonde.
00:03:48.520 He looks around like 6'1", something like that.
00:03:50.780 I'm looking up a picture of him just to, yeah, there you go.
00:03:54.440 He's got kind of the Ed Begley Jr. kind of look about him.
00:03:57.840 Yeah, like Suburban Dad.
00:03:59.800 Like, do you guys want some nachos?
00:04:01.100 Like calling in from the kitchen.
00:04:02.320 Yes.
00:04:02.580 Guy who owns a recumbent bike.
00:04:04.300 Absolutely.
00:04:05.700 Is the vibe with this guy.
00:04:07.220 Yes.
00:04:07.440 I mean, the only thing that I think that we can sort of pull out of these origin stories is really that he's fascinated by the idea that people, especially consumers, make choices without really knowing why they're doing it.
00:04:21.160 Right.
00:04:21.540 So after he gets his PhD from Stanford, he's basically a kind of normal marketing professor at various business schools.
00:04:27.840 He works at Dartmouth.
00:04:29.840 He goes to the Wharton Graduate School of Business at the University of Pennsylvania.
00:04:33.540 And eventually he sort of lands at the University of Cornell in 2005.
00:04:37.440 Because there's kind of no biographical details in his books, I kind of had to piece together his career basically from his like Google Scholar citations.
00:04:46.580 So I just like organized all of his research in chronological order and just started looking at the kinds of studies that he was publishing.
00:04:53.340 Over the course of his career, he publishes over 480 academic studies.
00:04:58.800 Man, oh man.
00:04:59.600 So where does that productivity come from?
00:05:01.860 I mean, we all got there. 1.00
00:05:04.100 Fuck. 1.00
00:05:04.740 Okay, shit. 1.00
00:05:05.420 Huge. 1.00
00:05:05.620 I can't tell you without a huge spoiler.
00:05:07.060 Okay.
00:05:07.440 For the first 10, 15 years of his career, the research that he's producing is, like, very straightforward marketing research.
00:05:15.300 So one thing that he's really obsessed with is this idea of unit size that, you know, if people buy a large bag of chips, they'll eat the whole bag.
00:05:23.580 And if they buy a small bag of chips, they'll eat the whole bag.
00:05:25.880 But he's doing all of this research to give advice to companies on, like, how big their unit sizes should be.
00:05:31.140 Like, it's very clear that what he's doing is, like, he's helping companies sell more products.
00:05:35.740 Like, that's the way that all of his work is framed throughout the 1990s.
00:05:39.820 Yeah, this is the Halo Top approach, right?
00:05:41.940 Yes.
00:05:42.340 Don't stop till you hit the bottom.
00:05:44.080 Like, but, but, but, but, but, like, let's just assume that you're going to eat the whole
00:05:46.760 package of whatever you buy.
00:05:48.460 Exactly.
00:05:49.180 So his early work shows that, like, what you name food is actually really important for
00:05:53.620 whether or not people buy it.
00:05:55.440 And it even affects their taste.
00:05:57.240 So people will actually rate, like, wine that is from California or that they think is from
00:06:02.240 california as more tasty than wine that they think is from north dakota people rated freedom fries
00:06:08.280 as tasting superior to french fries this is one of those places where it's like it's very tempting
00:06:14.120 to believe that we're all much more sophisticated than we are and our decision making is really
00:06:17.980 different than it is but we're all kind of on autopilot and we're all way more predictable
00:06:22.340 than we would like to think that we are completely we all are profoundly affected by marketing and we
00:06:27.620 all think that we're not yeah like i bought a casper mattress man oh did you oh yeah not
00:06:32.120 It's not because I, like, did a literature deep dive.
00:06:34.380 It's like, no, no, they mentioned it on, like, five different podcasts.
00:06:36.680 Truly two days ago, I bought a Helix mattress for exactly the same reason.
00:06:41.820 A fun fact, Aubrey, when your Helix mattress comes, you're more likely to eat more of it
00:06:46.020 if it comes in a large package than a small package.
00:06:49.500 So, I mean, a lot of his early work is sort of around these same kinds of ideas.
00:06:53.360 It's basically trying to figure out what makes people purchase products.
00:06:55.700 He finds that you eat more fat if you put olive oil on your bread than butter,
00:07:00.140 but you also eat less bread. His studies get a little bit of play in the media, but he's not
00:07:05.540 really a name in nutrition research. That all changes in 2005. He publishes two studies that
00:07:13.020 are explosive in the press. You could not avoid these stories at this time. The first, I'm sure
00:07:19.120 you've heard of this one. Do you know the bottomless bowl study? I don't know the bottomless.
00:07:24.120 Is this like an all you can eat kind of thing? Yeah. So they did this thing. He has this lab now
00:07:29.280 that has like hidden cameras and two-way mirrors and it's all these ways of like surveilling the
00:07:33.520 way that people eat and why they eat differently and the study is exactly what it sounds like they
00:07:38.020 build a bowl that has like a little tube underneath it where it's actually feeding more soup into the
00:07:43.080 bowl as you're eating it and so he finds that people eat i think it's 53 percent more of this
00:07:48.640 bowl that is refilling itself it's this idea that like you just eat until the bowl is empty like
00:07:53.720 none of us are eating based on any satiety cues we're just like uh there's more left in the bowl
00:07:58.300 So I better keep eating.
00:07:59.940 This lab of his was just the Olive Garden?
00:08:03.380 Yeah.
00:08:04.540 There actually is like a test restaurant.
00:08:08.000 I don't know why they don't call it the test-a-rant at this university where like people like diners can come and they know that they're participating in studies.
00:08:15.480 It's like marketing studies.
00:08:17.060 And it'll be like the menu will be like, you know, have French names one night and then English names the other night.
00:08:21.860 And they'll test like does this affect your purchasing decisions?
00:08:24.320 So like he's also doing these lab studies in this test-a-rant constantly.
00:08:28.300 That's fascinating. And also, like, I feel as you're just talking about the experimental designs, I feel myself going like, oh, right? Like, it feels like an interesting, like, dinner party conversation sort of topic. So I'm like, there's a lot of curb appeal to these studies.
00:08:43.220 I mean, this is like catnip for journalists.
00:08:45.700 Sure.
00:08:46.080 The other big study that comes out in 2005 is the, do you know this one, the popcorn study?
00:08:51.320 I don't know the popcorn study, at least not by name.
00:08:53.680 This is one where people are going to a movie and sort of at the door, he tells them like, you've been entered into a drawing something something.
00:09:01.360 We're giving everybody free popcorn tonight.
00:09:03.500 And so they do this in like a bunch of different conditions, right?
00:09:05.780 So in one night, they give everybody a medium-sized bucket of popcorn.
00:09:09.540 And then the next night, they give them large buckets of popcorn.
00:09:12.180 Sure. 0.95
00:09:12.440 So the sort of the twist of this study is that half of the people in all of these conditions get like shitty popcorn. 0.75
00:09:18.520 They said it was like squeak when you eat it. 0.97
00:09:20.420 Like it's old, it's stale, it's gross.
00:09:22.340 and two people apparently asked for their money back
00:09:25.520 and they're like, the popcorn's free.
00:09:27.280 Like, just don't eat it.
00:09:29.520 And so this, of course, makes a big splash
00:09:31.120 because it turns out, you know, first of all,
00:09:32.980 people eat way more when they are given
00:09:35.040 this large bucket of popcorn. 0.98
00:09:36.400 And secondly, they eat more of even like shitty food. 0.93
00:09:40.800 So this is what he says in Mindless Eating about this study. 0.88
00:09:43.780 Did people eat because they liked the popcorn?
00:09:45.980 No.
00:09:46.520 Did they eat because they were hungry?
00:09:47.940 No.
00:09:48.440 They ate because of all the cues around them.
00:09:50.740 Not only the size of the popcorn bucket, but also the distracting movie, the sound of people eating popcorn around them, and the eating scripts we take to movie theaters with us.
00:09:59.160 All of these were cues that signaled it was okay to keep on eating and eating.
00:10:02.980 So, like, I understand that people are eating this sort of mindlessly and without really any connection to the quality of the food or how fresh it was or anything like that.
00:10:12.120 But, like, how does he get from there to, it's all these other keys.
00:10:15.840 Like, I've figured out what it is.
00:10:17.080 Like, I know it's not this thing.
00:10:18.260 Therefore, it's this other thing.
00:10:19.740 Exactly.
00:10:20.180 Like, how does he get from point A to point E? 0.91
00:10:23.060 These questions, Aubrey, are exactly the questions that nobody fucking asked at the time.
00:10:28.020 These two studies, like, they're written up in The Atlantic. 0.93
00:10:30.900 They show up in The New York Times.
00:10:32.020 I mean, there's just a huge frenzy of media activity around these two stories.
00:10:35.960 And thus begins Brian Wansink's, like, long career of just, like, publishing blockbuster
00:10:40.820 study after blockbuster study.
00:10:42.700 This is an excerpt from a Vox article.
00:10:44.620 His experiments have found, for example, that women who put cereal on their kitchen counters
00:10:48.860 way more than those who don't and that people will pour more wine if they're holding the glass 0.97
00:10:54.520 than if it's sitting on the table i hate this shit so much mike i know dude i remember years 0.98
00:10:59.960 ago do you remember that show the doctors it was like a daytime tv show where they had like a few 0.98
00:11:05.060 doctors on at the end of every show they would read out shit like this that is like completely
00:11:10.400 decontextualized completely like nonsense right yeah and i absolutely remember being at a nail 0.96
00:11:15.880 place one time getting my nails done the doctors was on in the background and they were like here's
00:11:20.540 an interesting finding women who have fresh cut flowers at home report being happier than those
00:11:26.580 that don't so if you want to make your wife happy bring her some flowers and i was like 0.99
00:11:31.080 what the fuck is this you're not going to talk about like who has 20 bucks to blow every week on 0.98
00:11:37.080 fresh flowers like flowers are expensive yeah it's like a very broad statement but because 0.99
00:11:42.080 it comes from a sort of science-y source, it feels more legit when you're like, it's 0.99
00:11:47.520 still weird bullshit garbage. 0.98
00:11:49.940 It's perfectly structured to be in that form of like, hey, did you know women who do this 1.00
00:11:54.580 also have this? 1.00
00:11:55.600 As if it's some kind of rule.
00:11:57.280 It's essentially like a Snapple cap fact.
00:11:59.800 Yes, exactly.
00:12:00.580 Where you're like, no, no, no, no.
00:12:01.780 Listen, the great thing about the Snapple cap fact is that I find out that fish take
00:12:05.580 naps.
00:12:06.280 Yeah, yeah, yeah.
00:12:06.660 It doesn't make me be a fish who takes a nap.
00:12:09.020 I mean, the best example of that is one of his studies that gets, like, an amazing amount of play in the mainstream media is one that shows men eat 93% more pizza and 86% more salad in the presence of women.
00:12:23.260 What?
00:12:23.660 It is one of those things where, like, people try to translate this into, like, a weight loss rule. 1.00
00:12:27.260 They're like, don't eat with women.
00:12:28.560 It's like, I think something else is going on. 1.00
00:12:32.080 There's also an infamous study where Brian goes through old editions of The Joy of Cooking.
00:12:37.920 You know, the joy of cooking has been around since like 1936.
00:12:41.040 He publishes a study where he says only, you know, there's only about 18 recipes that have like endured from 1936 through all the additions to I think the most recent at the time was the 2006 edition.
00:12:53.360 And it turns out the calories in those recipes have increased significantly over time.
00:12:59.780 dun dun dun. So one of his sort of explanations for the obesity epidemic is this idea that like
00:13:05.740 portions, unit sizes, the food environment has completely changed over time. And this is like
00:13:10.940 a perfect little encapsulation of that, that like a sort of a normal dinnertime recipe
00:13:15.360 is just like 30% larger now than it was in 1936. That feels observably true, right? Like in my
00:13:22.120 lifetime alone, right? Like the largest drink size that you can get has like doubled, maybe triple,
00:13:28.280 another thing that he mentions in the study is that muffin tins if you look at old muffin tins
00:13:34.000 the muffins were like half as big as they are now yeah there you go another infamous one
00:13:38.500 is he measures the eye angle of cartoon characters on cereal boxes what and he finds that brands aimed
00:13:48.460 at children the cartoon characters are looking downward at an angle of 9.67 degrees so that it
00:13:56.300 looks to children in the grocery store as if they are looking at them this is another of his like
00:14:01.780 very famous studies that if you google it you can still find like 50 references to it man when you
00:14:06.020 started talking about cartoon eye angles i was like are we gonna get into real racist territory
00:14:11.260 no anyway back to this dr seuss book from the 40s like oh no for once that isn't where that
00:14:17.460 was going thank god first time on the show so by 2007 he's basically translating all of these
00:14:25.580 findings into weight loss advice so his book mindless eating comes out in 2007 becomes a
00:14:31.580 massive bestseller it's you know reviewed in the new york times and the new republic it's like this
00:14:35.180 huge deal what his research implies is that like there are all of these sort of small forces on
00:14:42.260 our behavior and if you can change those small forces you can actually lose weight sort of
00:14:47.060 without really knowing it right so the you know one of the main pieces of advice that goes around
00:14:51.720 from his book is like you can just switch to smaller plates in your house because people tend
00:14:55.780 to eat less from smaller plates this explains why so many co-workers around that time were like
00:15:02.020 get a smaller plate right this is absolutely what people want to hear you can become thin
00:15:06.980 without having to really think about it exactly and without anything else in society changing
00:15:11.640 either so we are going to watch a clip oh this is a presentation of his work from a michael
00:15:17.820 Pollan documentary in the early 2000s. Also, at some point, we're going to do some talking
00:15:23.560 about Michael Pollan, man. I know, dude. That's, I know. So grab a plate up there. The pasta's
00:15:30.640 right on the stove. Serve yourself up. Brian Wansink is an expert on eating behavior. He's
00:15:36.780 discovered we're often not aware of why we eat as much as we do. Sometimes it's because of
00:15:42.340 something we don't give the slightest thought to, like the size of our plate. We'll bring people in,
00:15:47.500 and we'll give them a large plate to serve themselves.
00:15:49.840 But what they don't realize is that the pasta is cold.
00:15:53.540 Wansing concocts an excuse,
00:15:55.500 so that everyone has to get a different plate,
00:15:57.640 which is slightly smaller.
00:15:59.180 These things weren't the right temperature,
00:16:00.920 because I'd like you to come back
00:16:02.040 and just grab another plate out of the cupboard there.
00:16:07.260 One of the things we find
00:16:08.680 is that they'll serve themselves a second time.
00:16:11.560 They won't believe they serve an amount any different
00:16:14.620 than they did the first time.
00:16:16.500 Did you guys notice anything different between the first time you served yourself and the second time you served yourself?
00:16:21.500 It feels a lot smaller. It looks smaller.
00:16:23.660 Oh.
00:16:24.460 So here's one thing we found.
00:16:25.560 The size of a plate tremendously biases us in terms of how much we serve.
00:16:30.580 The smaller the plate, the less food people take.
00:16:33.300 You serve four ounces on a 90s plate, you go, holy cow, I'll never be able to eat that.
00:16:38.160 So let's take a look at what happened to you guys.
00:16:39.420 Now I have some big plate, 207 calories.
00:16:42.940 Smaller plate, it dropped down to 162 calories.
00:16:45.740 whoa that's about 40 calories if this happened three times a day over the course of a year
00:16:52.220 but using a smaller plate you didn't weigh nine pounds less than you would if you had a bigger 0.99
00:16:57.440 plate just really really small things make this really huge difference i hate this shit i know 0.98
00:17:04.560 i hate it so hard i knew you would it really feels like it plays into this sort of life hack 0.98
00:17:09.780 kind of approach to there are scientific reasons that are beyond your control and if you just 0.99
00:17:15.720 fix those scientific reasons you will become thin and again like any fucking fat person can tell you 0.98
00:17:22.880 having less pasta on your plate does not make you nine pounds lighter at the end of the year 0.98
00:17:28.820 right like it's just it's such weird facile logic that i think because it's coming from a researcher
00:17:35.560 and because it's coming from a researcher at an ivy league university that it feels fancier it
00:17:41.320 feels more legit i also love the fact that it's like this wildly artificial scenario right it's
00:17:48.200 like it's only what appears to be college undergrads they are all white they are in this
00:17:54.460 weird situation where they serve themselves food and then he says no no no we have to serve you
00:17:58.900 again out of this like microwaved bowl and new plates and then they take the plates but it's not
00:18:03.920 clear like if they're going to eat the first thing they serve themselves so it kind of makes sense
00:18:08.020 that like the second time you would take less because you're like, well, do I have to eat all
00:18:11.380 of this? Yeah, the whole scenario is just so fucking weird and artificial that it's like, 0.62
00:18:15.760 it's not clear to me that you can actually extrapolate from this. Also, I'll tell you what
00:18:19.860 that pasta looked overcooked before it went back in the microwave. I know it looked bad. It looked
00:18:24.140 mushy by the time it came out. So that part's also like I would take less of the like, it was
00:18:29.820 already overcooked. And now it's like pasta mush. I mean, do you remember this context? Do you
00:18:34.400 remember the book nudge yes also this was around the time that like malcolm gladwell sort of burst
00:18:39.880 onto the scene and we all started talking about 10 000 hours of whatever for mastery and it was
00:18:44.980 a whole wave of again this sort of like life hacking kind of stuff totally that was like you
00:18:50.380 just need to know these little like they seem small but they're really important scientific
00:18:55.160 findings that will impact every other thing about your life yeah i mean this was like one of the most
00:18:59.840 important, prominent ideas at the time was this idea of behavioral economics that sort of the
00:19:07.040 way that people behave is one of the books that came out at the time was called Predictably
00:19:11.400 Irrational. So the canonical example that like showed up in every single article about this was
00:19:16.000 organ donations. Some countries have like 17% of people volunteer to be organ donors. And then if
00:19:22.920 you look at, I believe it was the Netherlands, it's like 60% of people. And you know, it seems
00:19:26.880 like, oh my God, there's so much more virtuous. Like what's going on in the Netherlands? They're
00:19:29.960 all so much nicer about their organs. And it turns out that it's just the default on the form.
00:19:35.260 In America, you have to take a box to say, yes, I will donate my organs. And in the Netherlands,
00:19:39.000 you have to take a box that says, nope, I don't really want to donate my organs.
00:19:41.980 It's the same thing with automatic voter registration. Like when people have to opt
00:19:45.580 out of being a registered voter, more people are registered and more people turn out to vote.
00:19:49.940 Yes. And Brian Wansink was actually a huge part of that. So the guys that wrote the Nudge book,
00:19:56.500 like the book that basically began this entire trend in 2008, they wrote a really long review
00:20:01.780 of Brian Wansink's book in The New Republic. Like this idea of sort of our eating behavior
00:20:06.540 being a metaphor for all of these other behaviors, like whether we pay our taxes,
00:20:10.840 which schools we send our kids to. This feels like an extremely
00:20:13.840 dude way of approaching the world, which is like, we just need to listen to the data and
00:20:18.160 do what the data says. That's it. It just, it was this very sort of quote unquote rationalist
00:20:23.420 approach of that, you know, life can be broken down into these sort of inputs and outputs,
00:20:28.380 like these little flow charts, right? And we know that people will do X if we give them Y,
00:20:33.080 right? Like we can predict the ways that people are going to behave. And all we need to do is
00:20:37.700 follow the science and we'll be able to solve all of these social problems.
00:20:41.140 Yeah.
00:20:41.380 So before we get to the downfall, I just want to talk a little bit more about the kind of work
00:20:44.980 that he was doing. He did a lot of workplace wellness consulting.
00:20:49.260 Oh, no.
00:20:50.220 I know. This is the thing. I don't want to get into it because we need to do a whole
00:20:53.160 episode on like the unbelievable trash fire that is this field but i just wanted you to read like
00:21:00.120 one very brief excerpt like this was the kind of advice that was going around to workplaces at the
00:21:06.480 time this is an excerpt from his book slim by design and he's talking about how he's doing a
00:21:11.680 consultation with google to prevent what's called the google 15 fuck off that when people start
00:21:18.120 working at google because there's like these canteens everywhere and the food is really good 0.89
00:21:21.340 and it's free. Everybody gains weight when they start working at Google. This is one of the ideas
00:21:26.000 that he comes up with. I'm going to send this to you because I cannot get through this without
00:21:29.460 tittering. Okay. To tackle the I gained weight before I knew it problem, Bob Evans, one of their
00:21:35.920 software engineers, had an idea. Have you ever seen those iPhone or Android apps that let you
00:21:41.000 upload a photo of yourself and it shows you what you would look like if you were 20 or 40 pounds
00:21:45.860 skinnier or fatter? Oh, Mike, I hate this already. I know. It only gets worse.
00:21:50.480 John figured out there might be a way to have a quote unquote food scanner set up that could scan someone's tray and a camera screen in front of them would take their photo and instantly display what they would look like in a year if they ate this much food every day for lunch.
00:22:07.940 Way cool is the end of that quote. 0.99
00:22:11.620 It's one of those demented fucking things I've ever heard. 0.99
00:22:14.280 i hate every so they have no concept of people with eating disorders or body dysmorphia they 0.99
00:22:21.540 have no concept of like fat people and increasing bias they have no concept of like a lot of the 0.83
00:22:28.820 things that i care the most about imagine being a fat person at google and somebody gets their 0.98
00:22:34.740 tray out and they hold it under this fucking miserable like minority report scanner and it 0.98
00:22:40.280 shows them a body that looks like you like what are we fucking doing here brian it really feels 0.97
00:22:47.100 like they're a hop skip and a jump away from just adding like an oinking sound effect or something 0.98
00:22:52.500 you know i mean this is like uh this is like school bully shit oh yeah he also at one point 0.96
00:22:59.020 in the same chapter suggests that employees should have to sign health declarations fuck off where 0.97
00:23:05.300 they promise their employers that they're gonna like exercise two days a week and that like if 0.99
00:23:10.560 they if their bmi goes above 30 they have to be like mandatory attendance at weight watchers fuck 0.98
00:23:17.200 off i know truly madly deeply fuck off so again we're gonna save like most of our fire emojis 0.98
00:23:25.700 for our eventual workplace wellness episode but like that is a main thread of his work 0.97
00:23:30.120 The other main thread of his work is this Smarter Lunch Rooms program.
00:23:35.360 Have you heard of this?
00:23:36.460 I remember this because, again, like I have a mom who's an early childhood ed person who never stops yelling about this study, which is that kids were offered, I think it was like fruit and candy.
00:23:48.020 Yeah, it was apples and cookies.
00:23:49.420 Apples and cookies.
00:23:50.720 And they studied essentially like if you just offer kids an apple or a cookie, which one do they pick?
00:23:56.860 Unsurprisingly, a lot of kids picked cookies.
00:23:59.300 and then they put stickers of elmo yes it was elmo is it elmo on apples and then they were like more
00:24:05.840 kids chose the apples but then and this is the thing that my mom never stops yelling about she's 0.97
00:24:11.460 like look at the ages of the kids that they're putting fucking elmo stickers on apples for 0.97
00:24:15.700 they're like 10 8 to 11 yeah uh you know who doesn't care about elmo is like a fourth grader 0.98
00:24:22.500 man you are in danger grave danger of spoiling this episode oh really
00:24:29.360 We will come back to this.
00:24:30.740 Really, really?
00:24:31.500 Yes.
00:24:32.320 I mean, this is like one of the central studies that becomes the basis of this entire program.
00:24:38.680 So this is by Nick Brown, a researcher who looks into this later.
00:24:41.580 Here's how the study worked.
00:24:42.780 Researchers recruited 208 students at seven elementary schools.
00:24:46.580 As part of their regular lunch menu, these students were already allowed to take an apple, a cookie, or both in addition to their main dish.
00:24:53.380 Before the study period, about 20% of the children chose an apple and 80% chose the
00:24:58.220 cookie.
00:24:58.840 But when researchers put an Elmo sticker on the apple, more than a third chose it.
00:25:03.360 So it's like perfect Brian bait, right?
00:25:05.600 It's cheap.
00:25:06.500 It's easy.
00:25:07.240 It doesn't require like taking away the cookie.
00:25:09.760 It's just like this little tiny thing and you got more kids eating fruits and veg.
00:25:13.220 It does feel very odd to be like, what if fruits and vegetables were branded?
00:25:18.600 Yeah.
00:25:18.800 What if these were Star Wars grapes or something where you're like, well, sort of.
00:25:23.380 that's fine. I mean, a lot of this actually is sort of trying to use traditional marketing
00:25:27.960 techniques for, you know, relatively unsexy fruits and vegetables. So another one of the
00:25:32.480 canonical studies that's part of this program is renaming vegetables. What? Yeah. So they try to
00:25:38.400 sort of brand vegetables in cafeterias to like make them cool. So there's like x-ray vision
00:25:44.000 carrots is one of them because like, you know, carrots have beta carotene and that like helps
00:25:47.700 your eyeballs. So some of the other ones, this is a list of the brands that they use for fruits
00:25:52.220 and vegetables in lunchrooms, orange squeezers, monkey phones, that's bananas, snappy apples,
00:25:59.640 cool as a cucumber slices, sweetie pie, sweet potatoes, and they renamed healthy bean burritos
00:26:05.480 as big bad bean burritos. And so according to the studies, this actually increases consumption as
00:26:11.780 much as 30%. So this basically becomes like a massive, like a massive sort of long running
00:26:17.420 program. And in 2007, he's appointed to the USDA and he starts helping them design this
00:26:24.840 Smarter Lunchrooms program, which is like exactly what you would expect from his kind of work.
00:26:29.140 There's a checklist of 15 different changes. And they're all sort of along these lines.
00:26:35.260 So it's, you know, you add a salad bar, but like you move the salad bar sort of in the middle of
00:26:40.020 the cafeteria. So kids kind of have to walk around it, right? It's not like in a corner where they
00:26:43.620 and ignore it. He suggests things like, you know, you put fruit in a bowl next to the cash register
00:26:49.100 rather than like this special place where kids have to go search for it. He moves the chocolate
00:26:54.480 milk to the back of sort of the rack so you have to like reach a little farther for it. It's all
00:26:59.940 of these like little tweaks. It's the school lunchroom equivalent of putting tabloids next
00:27:05.320 to the cash register. 100%. Yes, that's a very good metaphor. So I did not know this when I
00:27:10.300 started researching this but like this was used in 30,000 schools whoa so these are like the three
00:27:15.840 main threads of his work there's the weight loss stuff there's the workplace wellness stuff and
00:27:20.300 there's the school lunchroom stuff you know he's giving talks on every continent and he's quoted
00:27:27.320 in the newspaper a billion times and he's this massively famous researcher like one of the few
00:27:32.160 sort of brand name researchers in this field and on december 25th 2016 the whole thing comes
00:27:38.900 crashing down is this where we get to the scandal part super duper scandal part yes excellent give
00:27:44.080 me some scandal i fucking love this i this is like one of my favorite downfalls really i feel bad
00:27:50.800 about celebrating this but like this is just one of the most delicious downfalls i've ever seen 0.95
00:27:54.860 okay so the entire thing the dominoes start to fall with a blog post wait brian wansing
00:28:01.680 writes a blog post yeah like he has a blog at the time that's quite well known and you know he talks
00:28:06.880 about like his research and sort of their findings and just you know in the ways that like academics
00:28:10.840 have blogs he'll just have sort of musings on various things sure and so in late 2016 he writes
00:28:17.320 a blog post that begins with three paragraphs that i am going to make you read oh is it going 0.81
00:28:24.600 to be better or worse than the fucking google scanner oh way better okay good by the standards 0.99
00:28:29.560 of our show like this is this is weak shit this is fine okay like as far as the trauma meter that 0.99
00:28:34.700 is always like bouncing in the red at the bottom of our show at all times. Like this is green to 0.99
00:28:39.260 yellow. Okay, good to know. Okay. A PhD student from a Turkish university called to interview to
00:28:45.680 be a visiting scholar for six months. When she arrived, I gave her a data set of a self-funded
00:28:51.060 failed study, which had null results. It was a one month study in an all you can eat Italian
00:28:56.560 restaurant buffet, where we had charged some people half as much as others. I said, this cost
00:29:02.640 us a lot of time and our own money to collect, there's got to be something here we can salvage
00:29:07.560 because it's a cool, rich, and unique data set. I had three ideas for potential plan B, C, and D
00:29:13.980 directions since plan A had failed. Every day she came back with puzzling new results, and every day
00:29:19.620 we would scratch our heads and ask why and come up with another way to reanalyze the data with yet
00:29:24.160 another set of plausible hypotheses. Eventually we started discovering solutions that held up
00:29:30.220 regardless of how we pressure tested them. I outlined the first paper and she wrote it up.
00:29:35.720 This happened with a second paper and then a third paper, which was one that was based on
00:29:40.020 her own discovery while digging through the data. What do you think?
00:29:42.840 So basically, he talks about sort of bringing in this PhD student to help out at his lab.
00:29:48.940 He's essentially asking her to keep reinterpreting the data basically until she finds something. And
00:29:55.500 every day she goes and reinterprets the data and brings it back to him. And he goes, that's not
00:29:59.480 quite it. Reinterpret it again. That's not quite it. Write this paper and write it differently.
00:30:03.720 It seems real fucking wild to go back to the same data set again and again and again and go, 0.75
00:30:08.420 what about this? What about this? What about this? What about this? Of course, interpretation is 0.94
00:30:12.800 always, always, always part of the deal when you're doing research, right? Like everything
00:30:16.440 gets interpreted by humans. There is nothing that is like fully, fully, fully objective as we want
00:30:20.720 to think there is. And there's a point at which either there are conclusions to draw or there
00:30:26.160 aren't. Yes. And when you start to force it, you start to change the shape of the data itself,
00:30:32.260 right? Yes. It feels almost like photoshopping. Oh, yeah. There's a point at which you're
00:30:36.920 changing the contrast and the brightness, and then there's a point at which you're actually
00:30:40.080 just manipulating what's in the photo. Yes. Do you want to hear the titles of the papers that
00:30:45.400 came out of all of this data digging? Oh, God. Low prices and high regret. How pricing influences
00:30:52.280 regret at an all-you-can-eat buffet. Lower buffet prices lead to less taste satisfaction.
00:30:57.880 How traumatic violence permanently changes shopping behavior. And also remember the study
00:31:03.160 we mentioned earlier that men eat more in the company of women? That's one of the five studies
00:31:09.640 that they get from just scraping this data basically to death to find any associations in it.
00:31:16.020 Right. So they didn't set out to be like, let's take a pool of people who've experienced traumatic
00:31:20.320 violence and see what happens to them in a grocery store they were like we are doing a study in a
00:31:25.340 grocery store what happens when we look at just the people who've had experiences with traumatic
00:31:29.300 violence basically right yes that's a big oops yeah it's bad and it's more than an oops right
00:31:34.460 and so the reason why i find this so delicious this is you know the cute opening anecdote
00:31:40.100 to a blog post that like isn't about this the blog post is basically this like rise and grind
00:31:47.440 bullshit where he's comparing this hardworking, unpaid Turkish researcher to a paid grad student 0.85
00:31:55.080 in his lab who refused to do this for him. The whole thing is like this fucking subtweet of this 1.00
00:32:00.360 poor woman who left his lab and didn't want to do this wildly unethical research. So he says, 0.99
00:32:06.340 six months after arriving, the Turkish woman had five papers accepted or submitted.
00:32:11.060 In comparison, the postdoc left after a year and also left academia with one quarter as much
00:32:16.860 published as the Turkish woman. I think the person was also resentful of the Turkish woman.
00:32:21.320 God damn it. It's also, I will say in this blog post, it feels extremely wild to watch someone 0.99
00:32:27.280 commit career suicide without knowing that that's what they're doing.
00:32:30.140 Right? And having no idea?
00:32:31.840 It's really something.
00:32:33.820 But so are you familiar with this term p-hacking?
00:32:36.500 It's one that I've heard. I don't totally, like my understanding is that it is sort of
00:32:40.820 this general practice of like you interpret the data so much that you start to manipulate it.
00:32:47.640 Basically, yeah. I actually think that a better term for this is harking, which stands for
00:32:53.700 hypothesis after results are known. Oh, that's a great acronym.
00:32:58.980 It's good. And it's a good word too. You can say like harking at the moon and stuff.
00:33:02.680 This is basically what Brian is describing here, where it's like you've gathered all of this data,
00:33:07.680 the central question that you're trying to answer, you didn't get the result that you wanted or it's
00:33:12.720 inconclusive or whatever. And so basically you just start like systematically going through your data
00:33:17.920 and being like, well, what about, you know, men eating with women? What about like older people
00:33:22.360 and pizza? What about salad? You just start going through it and being like, well, is there anything 0.96
00:33:27.920 else here? This is a problem in science generally, but it's especially a problem in nutrition
00:33:34.100 research. I think that's something that people don't really know or like haven't really internalized
00:33:38.640 is that there's essentially no way to research nutrition. Because you can't really induce diet
00:33:45.000 changes in people in any sort of scientifically robust way. Like this is why essentially every
00:33:50.220 study that compares like the Atkins diet to the Ornish diet, none of them actually find
00:33:55.360 interesting results because nobody can stay on these diets very long. Right. Almost all of those
00:34:00.760 studies are like there's like a little section where they just sort of mention briefly like 70
00:34:05.820 percent of the people on this diet dropped off anyway the results are blah blah blah and you're
00:34:09.840 like well exactly the only thing that leaves you is to survey what people are already doing right
00:34:15.080 so this is how you get a billion of these studies where they'll take you know 10,000 people or these
00:34:19.740 like giant cohorts and they'll ask them a bunch of questions do you eat blueberries do you eat
00:34:24.640 apples do you have cancer are you tall are you short are you redheaded and then you can publish
00:34:29.060 the associations that you find people who eat breakfast every day like way less than people 0.67
00:34:33.660 who don't eat breakfast every day like every day you can find a study coming out that is like along
00:34:38.200 these lines right people who ate full fat dairy as a kid are more likely to be thin in adulthood
00:34:43.900 exactly that is like blows people's brains out of their heads and is one of these sorts of
00:34:49.820 associations where you're like okay but what else does that mean exactly there's a really good
00:34:53.760 series of articles by Christy Eschwandan at BuzzFeed, who sort of does a deep dive into like
00:34:58.760 the way that these large scale studies are done. And the sort of the main thing to know about these
00:35:03.600 huge survey studies where they're asking people, you know, about their health conditions and about 0.82
00:35:07.400 their weight, is that like the data is total trash. Because there's really only two ways that
00:35:14.480 you can get information from people about what they're eating. The first way is you do these
00:35:19.120 like 24 hour recall studies. You keep a diary for a day and then you sort of write down like today
00:35:24.920 I had a sandwich for lunch and then like I went to McDonald's for dinner or whatever. But of course
00:35:28.420 the problem with that is that first of all the minute you start keeping a food diary you start
00:35:32.620 eating differently. Yeah that's right. Like if somebody tells you to write down everything that
00:35:35.700 you're eating you're probably going to eat better that day. Yeah. And even that Christy in her
00:35:40.180 article talks about like she goes to an Indian place and eats like a curry for dinner and she's
00:35:44.620 like well how many calories was that? How many grams was that? What were the ingredients in that 1.00
00:35:48.640 curry, none of us know like the weight of what we're eating or whatever. Totally. And even when
00:35:54.280 you do have sort of like straight ahead kinds of foods that you're eating, even if you're like
00:35:59.080 eating a stalk of celery, the difference between a small stalk of celery and a large stalk of
00:36:03.420 celery is subjective, right? So, you know, 24-hour food diaries are problematic in their own way.
00:36:09.660 And so a lot of studies will do food questionnaires. They're called food frequency questionnaires,
00:36:14.260 which is like a pretty standard methodology for these kinds of studies that include all kinds of,
00:36:18.440 you know, a huge battery of questions about, like, how often, in general, you eat various foods.
00:36:24.520 Uh-oh.
00:36:25.140 You know, as you are probably guessing, this is also super problematic.
00:36:29.540 Yeah, yeah, yeah.
00:36:30.240 So this is an excerpt from Christy's article in BuzzFeed where she actually took one of these questionnaires.
00:36:35.560 Huh.
00:36:36.100 Some questions, how often do you drink coffee, were straightforward.
00:36:39.240 Others confounded us.
00:36:40.520 Take tomatoes.
00:36:41.540 How often do I eat those in a six-month period?
00:36:44.200 In September, when my garden is overflowing with them, I eat cherry tomatoes like a child
00:36:47.720 devours candy.
00:36:48.660 But I can go November until July without eating a single fresh tomato.
00:36:52.620 So how do I answer the question?
00:36:54.340 Questions about serving sizes perplexed us all.
00:36:57.020 In some cases, the survey provided weird but helpful guides.
00:37:00.040 For example, it depicted what a half cup, one cup, or two cups of yogurt looked like
00:37:03.760 with photographs of bowls filled with various amounts of wood chips.
00:37:06.940 I don't know why they didn't just use bowls filled with yogurt, but whatever.
00:37:09.620 It seems really odd to choose wood chips.
00:37:11.720 It seems like an easy one.
00:37:12.400 You know what it is?
00:37:13.060 it's the um commercials for tampons and pads that are like it's just mystery blue liquid yeah you
00:37:18.920 know that time of month when you like just there's a bunch of windex that shows up
00:37:22.860 this is that other questions seemed absurd who on this planet knows what a cup of salmon or two
00:37:30.300 cups of ribs looks like i noticed that when i was offered three choices of serving sizes
00:37:35.400 my inclination was to pick the middle one regardless of what my actual portion might be
00:37:40.100 there's quite a few studies of like how bad these are like in one of these large cohort studies
00:37:46.180 they found that people were underestimating their calorie counts every day by as much as 800 calories
00:37:52.420 whoa and so it basically makes like all of these comparisons are completely invalid
00:37:57.060 because you can't say that you know blueberries prevent glaucoma or something if like you don't
00:38:02.880 actually know how much blueberries people are eating this feels like an inroad to the studies
00:38:07.680 that we're sort of constantly getting on foods that are sort of like controversial nutritionally,
00:38:12.980 right? Like cranberry juice is really good for you. No, it's really bad for you. Dark chocolate,
00:38:17.780 have it every day. Never have it. Oh my God. Red wine, drink it for your heart health,
00:38:22.420 but not after age 70 or whatever the things are, right? That we're sort of constantly getting
00:38:27.880 conflicting information about a handful of things like eggs were this way for a long time. 1.00
00:38:33.060 fucking eggs, man. Again, it makes it feel like it is sort of impossible to know, you know, 0.99
00:38:39.380 as a consumer, what you should and shouldn't be eating. And actually, like the answer here is to
00:38:43.820 be more transparent about like, it's really hard to find conclusive findings. Yeah, we just don't
00:38:48.740 know. We're not good at that. We don't have media systems that are especially good at sort of
00:38:53.000 display things as take it with a grain of salt, right? They're just like a bunch of systemic and
00:38:57.180 individual problems. Also, I would say like, we're asking people to do this thing, which is estimate
00:39:01.800 the size and amount and weight and caloric value of things without really having any training in
00:39:07.540 how people should do that. Generally speaking, we're all pretty bad at that. And we shouldn't
00:39:13.800 actually get good at it. Because when we get good at it, a lot of us develop eating disorders.
00:39:17.460 Yeah, yeah, no kidding. She has a really interesting section in her article,
00:39:20.080 where she talks about, you know, because this questionnaire includes 54 questions,
00:39:23.880 this is where p hacking comes in, right? Because you're getting 54 variables.
00:39:28.160 And she says,
00:39:29.720 The food frequency questionnaire we used produced 1,066 variables,
00:39:34.200 and the additional questions we asked sorted survey takers according to 26 possible characteristics.
00:39:40.200 This vast data set allowed us to do 27,716 regressions. 0.99
00:39:45.440 Holy shit. 0.94
00:39:46.300 So according to the data that they collected, 0.99
00:39:49.380 people who eat cabbage are more likely to have any belly buttons.
00:39:53.440 People who eat shellfish are more likely to be right-handed.
00:39:57.420 People who eat more fried fish are more likely to be Democrats.
00:40:01.020 And people who eat bananas have higher scores on the SAT verbal.
00:40:05.120 Yeah, I ate a lot of bananas.
00:40:07.540 These are all statistically significant results, by the way.
00:40:10.000 Like, these are all technically publishable.
00:40:11.660 I also think, like, part of the backdrop of findings like these and interpreting findings like these and people just sort of running with them is this desire to believe that scientifically we have arrived.
00:40:22.460 Yes.
00:40:22.840 Right?
00:40:23.120 That we sort of, like, know everything that is knowable.
00:40:25.380 we have reached the pinnacle and we are there now and now we're looking out onto all of the world
00:40:30.320 as it is right rather than going wait a minute sometimes people are not as meticulous as we want 0.97
00:40:36.660 them to be or sometimes we put wishful thinking into our science or sometimes there's shit we
00:40:40.800 don't know and techniques that are developing now that will help us in the future instead what we
00:40:45.360 get is a conversation about science that is like the science says this therefore it's true therefore
00:40:50.520 you gotta get smaller plates or something where you're like well that's not the whole picture
00:40:55.020 I also think that one of the fundamental misunderstandings here, and this comes up so much, is that when you hear that something is a significant result, that makes you think that it's big.
00:41:04.380 Like if I say watching Legally Blonde had a significant effect on my life, you'd be like, okay, it's a big effect.
00:41:10.340 Yeah.
00:41:10.700 But the term statistically significant, all that means is that it's unlikely to be due to chance.
00:41:16.460 Yeah.
00:41:16.800 This is from a Nature article about this.
00:41:19.700 Critics bemoan the way that p-values can encourage muddled thinking.
00:41:22.780 Last year, for example, a study of more than 19,000 people showed that those who meet their spouses online are less likely to divorce than those who meet offline.
00:41:31.760 That might have sounded impressive, but the effects were actually tiny.
00:41:35.360 Meeting online nudged the divorce rate from 7.7% down to 6%.
00:41:40.060 So those are statistically significant results.
00:41:43.160 But people who met online have a 1.5% lower divorce rate is not that interesting.
00:41:49.900 And also, if you're using a site like, say, eHarmony, where they're like, it's science, we're matching you based on science, that that also, like, increases your buy-in to the method of meeting and makes you feel like it's somehow more legit.
00:42:03.420 Yeah.
00:42:03.640 Again, there are so many variables here that could account for these differences.
00:42:08.360 And when you just say people who met online have, you know, are less likely to get divorced, people are like, oh, I should be looking online, right?
00:42:15.540 Rather than going, well, wait, what does that mean?
00:42:17.920 Yeah, I mean, this also comes up a lot in sort of anything involving mortality, that you always hear these things like eating nuts reduces your risk of prostate cancer by 40%.
00:42:29.020 Yeah.
00:42:29.560 And then you look at the actual numbers, and it's like, you know, you have a three and 100,000 chance, and that goes down to like a two and 100,000 chance.
00:42:37.120 It's not clear to me that I need to change my dietary habits to make this extremely rare thing like slightly rarer.
00:42:43.440 Totally. And I think in our brains, right, because most of us are sort of accepting this
00:42:48.000 news pretty passively and again, pretty uncritically. We hear that as eat nuts three
00:42:53.340 times a week and you definitely won't get prostate cancer.
00:42:57.060 Part of this sort of breakdown happens in the research itself. Part of it happens in how the
00:43:02.280 research is presented in the paper. Part of it happens in how the research is interpreted and
00:43:07.640 reported in media. And part of it happens sort of at the point of consumption, which is the point
00:43:12.360 which we hear it and sort of translate it into what we're going to do in our daily lives.
00:43:16.280 There are breakdowns at every step along the way in this process.
00:43:20.060 I also think, you know, the fundamental point about all of this research, too,
00:43:24.360 especially these large, you know, surveys, is that they don't show causation, right?
00:43:29.680 That they're very limited in what they can show.
00:43:31.280 All they can show is associations, right?
00:43:33.680 You know, there's ways that you can control for poverty, you can control for education.
00:43:37.200 I'm a little skeptical of that sort of how much statistical controlling you really can do.
00:43:43.200 But the fundamental fact is that all you can find is associations.
00:43:45.980 And oftentimes those are measuring a third thing.
00:43:49.440 There's probably something independent that is affecting how many bananas you eat and your score on the SAT verbal.
00:43:55.240 Right. Bananas as a snack are also sort of like speaking to a really specific sort of like racial class and cultural background, right?
00:44:03.940 That like, if you're in an immigrant family, your after school snacks might be a different
00:44:08.280 thing that doesn't make you less likely to do well on your SAT verbals, right?
00:44:13.340 Like, there are other sort of factors at play here.
00:44:16.620 Yeah.
00:44:17.580 It feels really challenging.
00:44:19.120 I find myself getting really angry as we're talking about all this.
00:44:22.120 Dude.
00:44:22.580 Because it's like a huge, weird, like, not intentionally, well, I don't even know, maybe
00:44:28.820 intentionally, like it's a grift economy.
00:44:30.800 I mean, this is what's so hard about this is because I think for people of good faith, if you're actually trying to find out sort of which nutritional habits are the best for promoting health, there's no perfect way to gather that information.
00:44:44.840 There's no good way to answer that question.
00:44:46.700 Most of the people in this field are trying to kind of triangulate, zigzag their way to real answers.
00:44:54.920 But the problem is that these methodologies, the gaping holes in these methodologies, leave them vulnerable to grifters and also vulnerable to the incentives of science.
00:45:06.500 So one of the things that is really important in Brian Wansink's story is this idea that you have to publish, right?
00:45:12.100 If you want to get tenure, if you want to get noticed in your field, you have to publish as much as possible. 0.96
00:45:17.940 And for a lot of people, if you've gone to all of this work to gather this data, you've spent months watching what people are doing at a pizza buffet, it's like, well, fuck, I can't just get rid of this. 0.97
00:45:28.320 I spent a ton of money. 0.95
00:45:29.160 There's grant money on the line.
00:45:30.520 Yeah, I mean, I also feel like, because of the ways in which sort of capital S science, right, has been used as a political football in recent years, particularly around climate change, particularly around, I mean, COVID is a great example. Do you believe the science or do you not believe the science? So there has become this sort of reaction on the left to be like, we believe in science, which means that we sort of like accept a lot of this stuff uncritically, right?
00:45:57.280 Right. That we sort of slip slide into this mode of just like whatever science says is the truth
00:46:02.560 without really a recognition of, you know, what most researchers and most scientists will tell
00:46:08.100 you, which is that science is like a series of very active participatory conversations.
00:46:13.640 Right.
00:46:13.860 Like the point of science is to figure out things we don't know.
00:46:17.520 And it's a process.
00:46:18.680 And it's a process.
00:46:19.720 And so this actually brings us to the original pizza gate.
00:46:23.700 This is this is not comet pizza.
00:46:26.320 No.
00:46:26.640 This is not QAnon. This is a different pizza-related scandal.
00:46:30.560 This is just the clickbait title that I'm giving this episode. Let's be clear.
00:46:34.840 So after this blog post comes out, the comment section is an absolute red wedding.
00:46:39.660 Everybody in the comment is like, this is why I left science. What you're describing is like
00:46:44.420 exactly the problem with science. So there are four grad students, kind of random people. They're
00:46:50.500 not like official investigators. They're basically just people who read this blog post and they're
00:46:54.980 like, this dude sucks. Their names are Tim Van Zee, James Heathers, Nick Brown, and Jordan Anaya. 0.98
00:47:00.900 And these four dudes dive deep into Brian Wansink's work. And the first thing that they
00:47:07.700 dismantle are these pizza buffet studies that he was talking about that he sent to this Turkish
00:47:13.620 researcher. So one of the first things that they find is in these four studies that are all based
00:47:18.620 on the same data, they have a total of 95 references to Brian Wansink's other work.
00:47:24.860 You know, in the literature review, they'll say like, blah, blah, blah, we know larger
00:47:27.740 plates, blah, blah, blah, right?
00:47:29.320 And yet, none of them have any links to each other.
00:47:32.820 And none of them even mention that there's been any other publications with this same
00:47:37.000 data.
00:47:38.200 So that's already like, kind of a statement that like, we know what we're doing here.
00:47:42.700 This is like an active choice to kind of bury the lead.
00:47:46.100 Yes.
00:47:46.440 So the other main thing that they find in these pizza studies is weird statistical irregularities.
00:47:54.020 Uh-oh.
00:47:54.620 We have to do some math to understand this.
00:47:57.100 Okay.
00:47:57.620 I'm going to try to make this as simple as possible,
00:47:59.500 partly because I'm not sure that I understand all of this,
00:48:01.760 but I'm going to try to present it as well as I can.
00:48:04.000 So basically, imagine if you had like a sample of 100 people,
00:48:08.600 and you're trying to figure out like do people like broccoli, right?
00:48:12.480 Yes or no.
00:48:13.120 And you're surveying them, and the only two options are yes or no.
00:48:16.120 They can't say, I don't know, right?
00:48:17.400 They have to say yes or no.
00:48:18.900 If I did that study and I came to you and I said 32.5% of people like broccoli, that's
00:48:26.860 an impossible number, right?
00:48:28.280 If I'm surveying exactly 100 people, there's a finite number of sort of results that I
00:48:32.340 can get, right?
00:48:33.240 Right.
00:48:33.560 So if I say 32.5% of people, that would mean that 0.5% of a person is in that data.
00:48:40.100 Yeah, that's right.
00:48:40.880 Does that make sense?
00:48:41.620 Right.
00:48:41.820 If you're doing a study of three people, which you wouldn't because that's too small, your options are 33.333%, 66.666%, or 100%.
00:48:55.060 Exactly.
00:48:55.520 If it's two people, your options are 50% or 100%.
00:48:58.440 Yes.
00:48:58.880 That's better than my broccoli thing.
00:49:00.400 Exactly.
00:49:01.640 And also, even if you have like 672 participants, there's going to be a finite number of options that you can have.
00:49:08.240 Even though it's going to be a much larger number of options, it is also going to be finite.
00:49:11.820 So one of the things that they find when they start going through these pizza studies is that a lot of the numbers are impossible.
00:49:19.100 For the sort of the regret data, people are rating their regret on a 1 to 7 scale, right?
00:49:24.520 I don't regret it or I regret it super duper much.
00:49:27.380 And one of the samples, like I think it's people who ate more than three pizza slices, there's only 10 people in that tranche.
00:49:33.300 So if all 10 people say 7, I regret it the maximum amount, right?
00:49:38.360 To get the average, you divide it by 10, right?
00:49:41.080 Because that's the number of people.
00:49:42.760 A total of 70, a score of a total of 70, divide that by 10, you'd get seven.
00:49:46.900 If everybody says seven, but one person says six, you'd get 69.
00:49:50.720 And then you divide that by 10, and that'd be 6.9.
00:49:53.320 And you can keep going all the way down, 6.8, 6.7, right?
00:49:56.140 It's basically like you'll get to a one-tenth of one percentage point.
00:50:00.940 Exactly.
00:50:01.300 No matter what.
00:50:02.080 Yeah.
00:50:02.280 You can't get pi out of that, right?
00:50:03.920 Yeah, that's right.
00:50:04.760 That's right.
00:50:05.020 So what they notice in the tables of, like, the published tables of these pizza studies is that one of the averages is 2.25.
00:50:12.980 What?
00:50:13.480 And another one is 3.92.
00:50:15.900 Are they then just fully making up numbers?
00:50:18.540 This is the thing.
00:50:19.540 To this day, we don't know exactly what happened.
00:50:23.840 The only thing that makes any sense is that, like, they're doing this from, like, different sample sizes.
00:50:28.060 Like, maybe they were using 11 people for these calculations and forgot to replace them. 0.99
00:50:31.560 But what this indicates is that it's, like, p-hacked to fucking death. 0.95
00:50:36.220 It is fascinating to me that all of this past peer review with, like, very simple arithmetic issues. 0.99
00:50:42.560 I know.
00:50:43.380 Right?
00:50:43.640 Like, this isn't even, like, how do you interpret data, blah, blah, blah, blah.
00:50:46.880 This is just, like, do all the numbers in the first four columns add up to the number in the fifth column?
00:50:53.340 No?
00:50:53.920 They eventually find 150 mathematical impossibilities.
00:50:58.400 another really weird thing is that the the sample sizes don't add up so you know if you have like
00:51:05.040 100 people in your study and then it's like people who ate no pizza people who ate one slice
00:51:08.660 people who ate two or more slices that's your whole study that should add up to 100 right but
00:51:13.380 it doesn't add up to 100 right that happens when like i'm thinking about this sort of like the
00:51:17.380 data analysis that they're doing being a little bit like hansel and gretel style that like yes
00:51:22.340 They didn't quite leave the trail of breadcrumbs to get them back out.
00:51:26.200 Yes.
00:51:27.140 So basically after Pizzagate, after like they look at these four papers, everybody starts looking at this guy.
00:51:34.500 Yeah, for sure.
00:51:35.860 So James Heather is one of the guys that's actually looking into this.
00:51:39.440 Designs a, it's called Sprite.
00:51:41.820 I forget what it stands for.
00:51:42.900 But it's this statistical tool that can actually reconstruct results.
00:51:47.380 Whoa.
00:51:47.800 And so they start going back through his papers.
00:51:51.780 Remember the popcorn study?
00:51:53.680 Yeah, yeah, yeah, yeah.
00:51:54.320 The bad tasting popcorn.
00:51:56.100 That one falls apart.
00:51:57.560 There's like numbers in there that should be impossible.
00:52:00.280 To be fair, that one is like,
00:52:01.740 they say that that's not as bad as some of the other ones.
00:52:05.100 There's one where it's one of the workplace wellness ones
00:52:07.800 where if you're sitting near a candy dish,
00:52:10.260 you eat more than if it's like placed far away from you.
00:52:12.700 That one falls apart.
00:52:14.820 Another one that gets debunked is the bottomless bowls study.
00:52:18.200 It just has a lot of data in it that like literally is impossible.
00:52:23.020 Like there's no way to get those averages and standard deviations from real data.
00:52:28.780 It's so tricky because none of this means that any of this is definitely true or definitely
00:52:33.780 untrue.
00:52:34.960 Yes.
00:52:35.400 There's a bunch of stuff we thought we knew and we don't actually know.
00:52:38.480 And it's gotten so far out into sort of the public, you know, imagination and into our
00:52:43.280 sort of collective bloodstream that you kind of can't unring that bell, right?
00:52:47.240 By the time people sort of just sort of start to intuitively believe and understand that men eat more pizza in front of women, you can't be like, well, actually, mathematically, those findings were both, you know what I mean?
00:52:59.460 Like, that doesn't mean anything to anybody.
00:53:01.040 Yeah.
00:53:01.300 And also, I would not be remotely surprised if it turned out that if a candy dish is near you, you end up eating more candy during the day.
00:53:07.360 Like, I think that that's extremely plausible.
00:53:09.640 I also think that, like, this is the whole point of science is to actually confirm things that seem plausible.
00:53:15.860 like it seems super plausible that the sun rotates around the earth but then you look into it you're
00:53:21.700 like whoops it seems like this very common sense plausible thing turns out not to be true this is
00:53:26.060 the episode when mike comes out as a flat earther i know listen there is a glass dome there are flat
00:53:32.520 edges why is there a horizon i'm just asking questions did you not know that i was a ptolemy
00:53:38.160 stan i like on little loop-de-loops and then it's like from this mathematical stuff it's just like
00:53:44.500 a non-stop avalanche of like ethical weird stuff like there's a huge amount of self-plagiarism
00:53:51.580 like entire articles that he's basically repurposing for book chapters and vice versa
00:53:55.820 like tons of that stuff there's my favorite one is apparently he wrote some weird article
00:54:01.240 about world war ii veterans and sort of like how trauma affects their eating habits something
00:54:07.740 something and in his sample this is like right there in black and white in his sample 20 percent
00:54:13.660 of the participants are women the whole thing is based on like combat veterans and like trauma
00:54:19.040 among combat veterans and it's like well women women didn't fight in world war ii we're doing
00:54:24.920 a study on the tuskegee airmen and in this sample we're going to talk through the 50 of them that
00:54:29.680 were white women and you're like what no and so we also get in this like wave of downfall stuff 0.93
00:54:38.540 we also get the complete collapse of all of this school lunches shit what happens with the school 0.99
00:54:44.640 oh my fucking god is it still is it still in play like what i mean the school lunch stuff the the 0.98
00:54:49.820 program is now defunct i mean i mean i think that like whatever there's probably schools that have 0.99
00:54:53.760 like bowls of fruit next to the cash register like i think that some of the principles are
00:54:57.040 probably still in play sure but as a sort of usda program it's it's gone like their website you can
00:55:02.420 only look at it on archive.org like the thing is toast there's a researcher named eric robinson who
00:55:07.740 starts looking into this stuff. And the first thing that he finds, it's actually like pretty
00:55:13.020 bad and pretty bad that nobody noticed this before. But they started implementing this
00:55:17.520 program in 2014. The first randomized control trials of these principles didn't start until
00:55:24.040 2014. So basically when they implemented this program, they had no data. One of the things
00:55:30.560 that Eric Robinson also notes in his paper is that, you know, there's these 15 strategies that
00:55:34.940 schools were supposed to be implementing. It appears to this day, only six of them have ever
00:55:39.680 actually been studied. Yeah, that just seems wild. And also, I will say, like, there is already so
00:55:45.780 much that we expect out of schools, right? Like, we're expecting teachers and school administrators
00:55:52.380 and school staffers to carry the weight of so many of our social anxieties. Oh, my God. And
00:55:57.660 this also becomes another one that they're now holding and carrying, right? But like, all of our
00:56:02.960 weird shit around health and weight and whatever that kids generally don't have in the same way. 0.95
00:56:10.180 This is like part of how we ensure that kids pick that stuff up, right? And that we sort of 0.98
00:56:15.100 project it onto them. And also, I mean, this was the thing that I was biting my tongue to keep
00:56:20.200 from saying earlier when you were talking about the Elmo stuff. So the Elmo study, choose an
00:56:25.420 apple with an Elmo sticker or a cookie, right? Yeah. It turns out the study wasn't on eight to
00:56:31.400 11 year olds it was on three to five year olds well that makes more sense right you sniffed it
00:56:38.200 out millions of researchers did not sniff this out aubrey you knew what i mean truly like anyone
00:56:45.760 who has spent any amount of time around a nine or ten year old can tell you that even if they still 0.91
00:56:52.900 like elmo they sort of know that they aren't expected to so like this is a fucking wild error 0.57
00:56:59.180 to make like i thought it was among eight to eleven year olds it was actually three to five
00:57:03.000 year olds in daycares so this has no application to elementary schools well and also like again
00:57:08.840 children ages three to five are in a completely different state of brain development exactly
00:57:15.780 and impulse control and like it's not quite like you should be studying another species but it is
00:57:21.600 like so far off the mark well yeah i mean the whole concept of choosing food is like very different 0.90
00:57:28.380 for a four-year-old versus a 10-year-old yes there's also another fucked up thing this actually
00:57:33.260 shows up in a lot of lunchroom studies i've been reading studies all week that in this sort of this 0.79
00:57:37.820 study where they made the chocolate milk harder for the kids to grab what happened is they took
00:57:42.700 more milk so it's like yay they're taking milk it's not flavored it's not sugared
00:57:47.100 but then they're not actually drinking it no getting kids to take vegetables is a totally
00:57:53.500 different thing from getting them to eat vegetables. Like it is very easy to get my
00:57:57.780 nephew to put vegetables on his plate. Yes. That's fine. He will do that without objection.
00:58:02.820 When you're like, hey, you got to have a couple more bites of vegetables. He'll be like,
00:58:06.280 how big of a bite? Let's negotiate. There's also, this is one of the fucking pettiest
00:58:11.180 things I've ever seen in an academic paper. You know, there are like randomized control 0.97
00:58:15.360 trials that show that, you know, smarter lunchrooms do actually improve, you know,
00:58:18.900 the number of fruits and vegetables that kids are eating. And, you know, this is sort of
00:58:22.120 evidence that brian wansink was using for years and he would talk about it in his ted talks and
00:58:25.840 like we're doing this and it really improves things for kids and then when eric robinson goes
00:58:30.880 back to the data the studies show that you know after all of these interventions in schools like
00:58:35.700 the biggest sort of study that has been done on this shows that kids are eating 0.1 unit of fruit
00:58:41.920 more than kids who aren't getting this intervention and so he has a whole page of his paper dedicated
00:58:49.160 to photos of this so he goes figure one a small apple figure two 10 of an apple and it shows like
00:58:57.080 this lonely little slice of an apple on a plate and he like clearly took the photos at his office 0.98
00:59:02.100 like they're in the shitty break room i love everything about this that's fucking brutal dude 0.97
00:59:06.840 you could have just said it there is a real special place in my heart for like the extreme 0.97
00:59:12.620 pettiness that is reserved for very high-minded fields love it so basically it appears that like
00:59:19.380 all of these interventions increased the amount of fruit that kids ate by one-tenth of an apple
00:59:25.400 we're not talking about like a revolution in children's consumption here we're talking about
00:59:30.000 extremely modest effects yeah and that was never how he was describing them publicly it's so hard
00:59:35.440 because this feels like a real like emperor's new clothes kind of an episode yeah i mean this is
00:59:41.580 honestly like my biggest revelation from this is that you know there's the statistical stuff
00:59:46.140 there's the p hacking stuff but the worst thing that he did was like out in public yeah like one
00:59:52.080 of the things eric robinson mentions is that you know he would have these studies where you know
00:59:56.700 we find kids are eating 0.1 unit of apple more and then in the conclusion of the article he'd be 0.99
01:00:03.720 like this is an effective intervention for childhood obesity god fucking damn it and you're 0.97
01:00:08.040 like, well, it's very evidently not. You have to line up so many dominoes before you can sort of 0.99
01:00:13.240 flick one and have them all fall the way that you think they should, right? And this guy essentially
01:00:17.720 set up two dominoes and they didn't make it past sort of the finish line, right? And he was like,
01:00:21.980 we did it. There's another example in here is that there's articles where in the abstract,
01:00:27.260 he says that, you know, giving kids pre-sliced fruit increased fruit consumption by 71%.
01:00:32.560 And then you read through the paper and it actually increased it by 4%.
01:00:36.680 if you and i as people who are not uh we don't have math degrees like if this stuff jumps up
01:00:45.460 out to you and i just at face value right like that that feels like oh it's not good it's not
01:00:51.740 great it's not great so this is actually sort of like the next stage of the downfall there's sort
01:00:57.060 of like this year-long period where like the pizzagate stuff is happening and there's statistical
01:01:02.040 analyses and there's a lot of the sort of behind the scenes are like intra-academic debates about
01:01:06.980 this brian wansink guy but it hasn't really bubbled up to the surface like sort of normies
01:01:13.100 were not really noticing this and the next stage this is so weird the next stage of this downfall
01:01:20.400 basically happens with a tweet thread by the joy of cooking twitter account
01:01:26.400 oh god i really love the idea of a nutrition researcher getting owned by the joy of cooking
01:01:37.340 fucking owned by my grandma's cookbook so do you remember the joy of cooking like study that he did
01:01:45.700 sure they had of course seen his study they'd seen it like it's been cited more than 30 times 0.97
01:01:50.360 it shows up in media reports it's something that like everyone just sort of mentions in
01:01:53.640 like cute little parentheses whenever joy of cooking comes up they're like lol the portion
01:01:57.640 sizes are so much bigger there's this great new yorker article by friend of the show helen rosner
01:02:02.840 who interviewed people at joy of cooking about like what it felt like to be the target of this
01:02:09.020 study and sort of trying to debunk it for years and like the world wasn't ready finally after sort
01:02:15.200 of things start to domino out of place for brian wansink they put out this thread saying that like
01:02:21.580 First of all, the entire idea of his original study was that, you know, there were all these
01:02:26.400 different recipes throughout time. He could only find 18 that appeared in every single Joy of
01:02:31.280 Cooking to compare to each other. The Joy of Cooking is like, uh, we found 275 recipes that
01:02:38.180 were in every single edition. And then they do this, like this whole thing about, it's kind of
01:02:44.480 absurd to say that portion sizes have increased when most recipes in the book aren't meant to be
01:02:51.240 eaten all at once like one of the recipes that he sort of says like has gotten bigger over the years
01:02:56.980 is gumbo it's like a big ass pot of gumbo yeah i mean it feels a little bit like look my reference
01:03:03.840 point for this whole section is just gonna be the new york times cooking section and the comments
01:03:10.740 on those recipes have you ever looked at the comments in the new york times cooking section
01:03:14.260 oh aren't they like i replaced the chicken with beef and the rice with styrofoam chips like it's
01:03:20.140 I modified it. And it was not very good. One star. You're like, well, come on, dude. And that feels
01:03:27.180 like the part of the challenge here, right? It's like, you can't and don't know. Like also,
01:03:32.840 hey, Brian Wonsink, you love your plate sizes so much. How big are the plates that people are
01:03:37.100 eating the gumbo off of? Yeah. Good Lord. That study was basically trash. Like this is sort of
01:03:42.540 the overall frustration. There were dozens of articles written about this. But like joy of
01:03:46.920 cooking portions are getting bigger. Like what? It's just not a story that means anything. And
01:03:51.460 it never was. So the final chapter of this downfall, and I think probably the most important
01:03:57.320 one, is a reporter named Stephanie Lee at BuzzFeed does a public records request. You know, a lot of
01:04:05.040 the people that Brian Wansink has been corresponding with are employees of universities. And universities
01:04:10.420 are public institutions and you can FOIA them. And so she gets this huge trove of emails from New
01:04:15.900 mexico state university where one of his collaborators works and she basically finds
01:04:20.800 that all of this was totally deliberate what he was basically running this lab as like a
01:04:25.900 publication factory in a very explicit way there's all these emails about sort of the same way that
01:04:31.900 it was with the pizza study where it's like they they've gathered all this data and then whatever
01:04:35.920 grad student gathered it has moved on and then he'll assign like another graduate student to
01:04:41.680 sort of mess with the data until they can get something publishable out of it and so he says
01:04:45.660 in one of the emails, a lot of these papers are laying around on our desktops and they're like
01:04:50.000 inventory that isn't working for us. We've got so much huge momentum going, this could make our
01:04:55.160 productivity legendary. In one case, there's this data that's like how people shop in grocery
01:05:02.220 stores where they don't speak the language. And it's just like, isn't all that interesting.
01:05:06.120 And they didn't really find like nothing jumped out of the data. And he's like, oh, we'll keep
01:05:09.480 looking at lower and lower tier journals until you can get it published. This is such a weird
01:05:15.100 capitalism forward approach to like scientific findings it's real it's really odd that feels
01:05:21.220 really odd to me and also media forward yeah because in a lot of these emails he's also
01:05:25.340 you know people will come to him with ideas and be like oh that'll definitely go viral
01:05:28.520 like this is a very explicit goal of his he's even at one point training his grad students
01:05:33.720 how to pitch their findings to the media like they'll have little sessions where they'll
01:05:37.760 practice their like elevator speeches to media sources which isn't in and of itself nefarious
01:05:43.320 right like you practice elevator speeches for all kinds of stuff but if again if that is the focus
01:05:48.320 you're not going to have findings that are as useful or as solid as you would want them to be
01:05:52.700 and eventually the cornell sun the newspaper of the university interviews his former grad students
01:05:58.020 and they say that like they felt really uncomfortable with the way that they were
01:06:01.260 pressured to manipulate data to frame things for the media to try to get these things to go viral
01:06:07.060 it seems like this was like a very well-known problem among people who worked with him yeah
01:06:11.600 Yeah. And there's like this intense power dynamic there, right?
01:06:15.180 Yes.
01:06:15.540 That he is like a nationally renowned researcher. You are a grad student or an employee who's
01:06:21.320 working for him. This is the person you would go to for a reference.
01:06:25.420 Exactly.
01:06:25.880 There is not a neutral power relationship here.
01:06:28.400 I had a bunch of other studies that I was going to debunk, but like, I think you get the idea.
01:06:32.120 Yeah, yeah, yeah.
01:06:33.040 Most of this stuff, most of this stuff doesn't hold up. And I feel like there's the sort of,
01:06:38.460 there's the debunkable articles but then i think the much bigger problem with his work is this
01:06:44.660 thing of him deliberately framing articles to get media play and a lot of these articles 0.90
01:06:50.480 a lot of them are just fucking dumb yeah there's a really interesting article by this child 0.97
01:06:55.800 marketing researcher guy this is way before this sort of the scandal happened this is in 2014 1.00
01:07:00.640 who looks into brian wants things remember the study on you know cartoon eyes looking downward
01:07:06.200 at children uh-huh this like actual marketing researcher writes this long essay who's like 0.92
01:07:10.540 this is dumb like this is not a real study you can't measure the angle of eyeballs in cartoon 0.77
01:07:16.780 characters he includes photos of like the tricks rabbit the tricks rabbit is looking upwards but 0.99
01:07:22.680 his like eyes are tilted down and he's like well does this count as down or straight on you can see
01:07:28.700 him sort of sputtering in the text he's like this is dumb like why was this published and of course
01:07:34.980 the reason why it was published was because you get media out of it yeah and i could totally see 0.88
01:07:39.740 as a marketing person that you'd be like we're not doing that we're doing other things but we're
01:07:45.840 not doing that what we're manipulating children in other ways you're making us look evil in the
01:07:51.960 wrong ways so in 2018 he's pushed out of cornell in september of 2018 the journal of the american
01:08:02.360 Medical Association retract six of his papers. These grad students, there's like a running
01:08:07.280 tally of all of the articles with, you know, plagiarism and conclusions that aren't supported
01:08:11.800 by the data, etc. And it's up to 52 publications. Some of them are self-plagiarism, which like I
01:08:17.700 honestly don't put in the same category as like faking your data kind of stuff. But also they
01:08:23.740 don't include the papers where like they just shouldn't exist, like the cartoon study. There's
01:08:30.020 been 13 articles have been officially retracted and 15 of his articles have been officially
01:08:34.820 corrected out of how many articles altogether so that he actually says this on his website he has
01:08:39.900 like a brianwansink.com he says seven percent of my research articles were retracted oh brother
01:08:46.880 so like that's his little way of like downplaying like how severe this is but it's also it's not
01:08:51.980 clear how many articles people have looked into right the the denominator is not his total body
01:08:58.540 of work, the denominator is how many did people investigate? Because I noticed that none of these
01:09:04.660 debunkings related to his work on workplace wellness. So I was like, okay, Brian Wansink,
01:09:10.020 workplace wellness, just Googled it, found a random paper. And then like the sample sizes
01:09:14.580 didn't add up in the first paper that I looked at. And it also had conclusions that were not
01:09:21.220 remotely supported by the data at all. And again, the first one that I looked at. So I think that
01:09:26.700 like i think people just got sick of debunking this guy's papers after a certain point yeah but
01:09:31.000 i think that like a much larger number of them also wouldn't hold up to scrutiny if you took a
01:09:35.220 look yeah i mean i think that makes sense also i would not be surprised if he was like seven percent
01:09:39.920 of my papers have been you know questioned or discredited or whatever and then you like look
01:09:45.160 at the data set and you're like it can't be seven percent that's mathematically impossible brian
01:09:51.620 have you learned nothing i think it's it is worth looking at him and his work very closely
01:09:58.900 and i also think in the same way that his work encourages us to look at folks sort of on an
01:10:04.300 individual level and there's an impulse to resist there i think there's an impulse to resist here
01:10:09.700 which is only looking at him and not also looking at like hey all of this stuff passed muster for
01:10:15.940 all the systems that we have oh yeah this went through the entire peer review process this got
01:10:20.880 published in the Journal of the American Medical Association, the New England Journal of Medicine,
01:10:24.900 like whatever the biggest journals are, right? So that also feels like a really challenging layer
01:10:31.060 to add on to all of this is that this isn't an exception. This isn't someone who skirted the
01:10:35.740 system. This is someone who went through the system and this is what came out.
01:10:39.540 Exactly. I mean, I think there's a huge media story here too, in that the New York Times should
01:10:44.860 not be quoting anybody 60 times. That just to me is just a huge red flag when you're going back to
01:10:50.200 the same source over and over again. Because then you have kind of like this mutualistic
01:10:54.300 relationship between the journalist and the researcher. I listened to a really interesting
01:10:58.280 podcast series on the Dr. John Berardi show about sort of the, you know, scientific debunking as an
01:11:04.440 institution. They said something really interesting. They said that, you know, in fields of science
01:11:08.740 where not very much is known, there's just a lot of mysteries still to solve. It's oftentimes the
01:11:13.620 most confident people, not necessarily the most knowledgeable people or the most careful people
01:11:18.600 who get the most attention. And I think that Brian Wansing really took advantage of that,
01:11:23.200 that he's like a good public speaker. He's handsome. He's straight and white and cis.
01:11:28.040 He's just kind of out in public giving good TED Talks, you know, writing very pop,
01:11:33.340 easy to read books. And I still cannot get over the fact that, you know, all this stuff about
01:11:38.440 like weight loss, you know, use smaller plates, eat one fewer candy bar a day, you'll lose 27
01:11:43.520 pounds. He never, it appears, even attempted to test any of this. And that's really incredible
01:11:50.180 to me because his entire thesis, his entire career was dedicated to this idea that we can make
01:11:56.000 inadvertent small changes to our food environments and lose a bunch of weight. Well, why not take
01:12:02.420 100 families and swap out 50 of them with smaller plates and see what happens? People can't stay on
01:12:09.120 the Atkins diet for six months. People can eat off of smaller plates for six months. That's
01:12:12.960 actually a pretty easy test to do. Yeah. And yet he never even tried. You know, we talk in sort of
01:12:18.880 food world and nutrition world about health halos, right? Things that seem to take on more value and
01:12:27.020 almost more moral value because they appear to be healthy. There's like a little bit of a health
01:12:32.060 halo effect with nutrition research. Oh, yeah. We are all in this sort of constant state of
01:12:37.700 desperation for more concrete answers than the various and sundry diet marketing that we're
01:12:43.900 exposed to. And I think when someone comes along who is from the academy, who has trained in the
01:12:50.740 scientific method, and, you know, does something that seems official and more concrete, we sort of
01:12:56.600 put folks who do that research up on a pedestal and put that research up on a pedestal in a very
01:13:01.440 uncritical way. Yes. I mean, this is the story of how far you can go if you are telling people
01:13:06.560 things they want to hear. I should also mention, I can't believe you just brought up the term
01:13:11.160 health halo. Do you know who coined the term health halo? Brian Wansink. Nah, fuck. Come on.
01:13:16.820 Really? Yeah, dude. Michael. This brings us to our happy epilogue. Do you want to hear the happy
01:13:21.520 epilogue? Let's hear it. I was totally surprised by this, actually. So in the research for this
01:13:27.740 episode, I did a lot of reading on school lunches generally. Did you know that school lunches have
01:13:33.240 gotten like way better really yes so in 2010 the obama administration passed the healthy hunger
01:13:40.580 free kids act and like there's all these studies showing that like kids are eating more fruits and
01:13:45.660 vegetables kids are eating more whole grains they took a lot of schools took like chocolate milk
01:13:50.660 and strawberry milk out of schools a lot of them took soda out there's all this data now showing
01:13:56.260 that like the average school lunch is significantly more healthy than the average like bagged lunch
01:14:01.880 that's i'm so i'm so glad to hear that there it sounds like more nutrient dense like yeah foods
01:14:07.100 there are more foods with like fiber in them there are fewer foods that are just high way high sugar
01:14:13.780 yeah a lot a lot more food is made from scratch now i mean there's still you know it's obviously
01:14:18.960 not perfect it's not as good as other countries there's still huge inequalities like it's not
01:14:22.820 perfect but i i do think that it's worth noting sort of these kinds of improvements and i also 0.70
01:14:28.000 think that it's worth noting like what improves kids health and it's it's like laws and fucking 0.71
01:14:35.560 money like one of the main things that went along with this act was way more money for school yeah 0.92
01:14:41.360 i think it's like three dollars and 38 cents per meal and it used to be like a dollar and 30 cents
01:14:46.900 per meal so it's like yeah when you give schools more money to feed kids they feed kids better
01:14:52.780 Yeah, that's right.
01:14:53.780 You know, there's all the hidden stuff with Brian Wansink's work.
01:14:57.440 But then the more visible stuff is like, this is something he was not interested in at all.
01:15:02.780 He goes out of his way throughout both of his books to be like, oh, we shouldn't take chocolate
01:15:06.740 milk away from kids because then they'll avoid school lunches altogether.
01:15:10.180 Like, we don't want to change anything.
01:15:11.680 That's too paternalistic.
01:15:13.140 And it's like, no, Brian, we should take the chocolate milk away.
01:15:16.740 Like, I feel fine about there not being chocolate milk in schools, dude.
01:15:20.840 Also, we live in a world of forced choices, right? It's happening all around us all the time. That is part of the central conceit of his work. It seems really weird to say we're going to stop short of forcing choices, even though all of his research is forcing choices.
01:15:34.580 Exactly. The implication, the obvious implication of his work is that we should force choices,
01:15:40.300 right? That these are designed environments. There's no such thing as a non-socially engineered
01:15:45.420 food environment. Like we are surrounded by social engineering at all times. So we might as well
01:15:50.620 engineer good environments, especially for kids. But it's like, as soon as it came to any actual
01:15:56.020 mandatory thing of like forcing kids to not have those choices, he would like completely freak out
01:16:02.560 and be like no no the corporations have to be our partners like it's very clear what his actual
01:16:06.360 project was the entire time ultimately these are incredibly complex issues and to focus just on
01:16:14.660 these sort of like individual choice slash life hack stuff feels like very short-sighted to me
01:16:21.280 so uh in conclusion don't life hack write to your senator and uh make something from the 1936 joy of
01:16:30.340 cooking tonight. Also, if there are any
01:16:32.420 researchers or science reporters
01:16:34.220 listening, put Elmo's stickers
01:16:36.300 on the good research.
01:16:37.260 Ha ha ha!
01:17:00.340 Thank you.