Ritchie is Wrong About Social Media and Mental Health
The evidence is only weak if we have a very shaky conception of evidence
I’m a big fan of philosophy of religion. Those of us who read dense tomes about philosophy of religion and wade through Aquinas’ talk about potentiality becoming actuality, often make fun of those accursed scientismists—those who think that all questions can be settled by simple applications of the scientific method; involving doing experiments and so on. When people claim that there are no experimental results that prove God, this, while true, shows nothing, because the existence of God is not a scientific hypothesis—it’s a philosophical one. When one wants to go about arguing against the existence of God—as I’ve done at length—they must raise philosophical objections, not scientific ones. One can, of course, point out that it would be very surprising that in a world where everything depends on God, none of our best science would need to reference God, but this would be a philosophical argument that references facts about science, rather than a straightforward application of scientific principles.
Some questions, especially philosophical ones, cannot be resolved by the use of a test tube or simple studies. When doing history, for example, one will try to explain substantial events without having the ability to run randomized control trials. It would be absurd to claim that the evidence for a historical claim was weak because no randomized control trials were done; we can be confident in the numbers of people killed in the holocaust even though we cannot do randomized control trials.
If one reads a lot of Substacks—as all the most virtuous individuals throughout human history have—it seems that they can’t go a day without finding someone writing about social media being the cause of rising rates of mental health issues in adolescents. Jonathan Haidt and his team have an entire substack dedicated to advancing this thesis, with a considerable amount of evidence. Richard Hanania, my favorite conservative writer, has written the same thing. The basic case is pretty straightforward—around 2012, the mental health of adolescents in lots of different countries started to decline, and there’s a strong correlation between social media and misery. The demographics hardest hit by social media are also the ones in whom depression, anxiety, and self-harm increased the most. This hypothesis is pretty good because it’s both intrinsically probable and backed by lots of evidence.
Stuart Ritchie is one of my favorite science writers. Always delightful clear, methodological, precise, and devastating at dispelling various myths backed up by shaky science. One of the most brutal takedowns I’ve ever read is this delightfully vicious Ritchie article; just a cursory glance at the introduction makes it clear that the author of the book review is in for a thorough drubbing. But in a rather surprising turn of events, I’ll be disagreeing with Ritchie, because I think he was completely wrong in his article criticizing the hypothesis that social media has been the main cause of rising rates of misery.
Ritchie agrees that “there are certainly lots of suggestive studies. And the overall phenomenon of increasing mental health problems, especially among girls, does call out for an explanation.” The “suggestive studies” at least suggest that, when people use social media, their mental health gets worse. However, he claims “digging into the details of the studies that are often used to stir up the social media panic reveals that the research is far more ambiguous than we’ve been led to believe.”
Of course, I would have no desire to try to argue with Ritchie about the finer points of statistical interpretation—I’m violently mediocre at stats (each time I tried to learn it, I kept getting distracted by social media, leaving me depressed, anxious and suicidal. I’m an adolescent, by the way). Instead, I think that Ritchie is relying on an incorrect methodology; one that might be useful in some limited cases like figuring out if a drug works, but one that is not worth using when trying to assign odds to a social phenomenon. Ritchie’s error lies not in statistical interpretation, but rather in errors at drawing inferences from various statistical techniques, and mirrors the mistakes of the accursed scientismists (autocorrect suggests changing the spelling of scientismist to scientist, which suggests that they’re in on the conspiracy).
Ritchie starts his article with three things that one should not do—two of which I disagree with. Here, I’ll explain why I disagree with his introductory throat-clearing.
1: Drawing vertical lines on graphs. You can’t read about this debate for long without seeing a graph showing children’s or teen’s mental health over time, with a big vertical line (or sometimes a shaded area) showing “the introduction of iPhones” or “the smartphone era”. We’re supposed to look at the graphs and see that obviously something went wrong at the time smartphones were introduced – after all, the rate of mental health problems went up just after it.
There’s no careful statistical inference being made here. Everyone who understands basic statistics knows that, for “correlation-isn’t-causation” reasons, you’re not allowed to imply that one thing causes another just because they happen at the same time. As it is, these graphs are little more than innuendo.
Worse, proponents can’t even agree where one should draw that line. In a famous 2017 article, the psychologist Jean Twenge drew the line at 2007: “iPhone released”. But Burn-Murdoch puts it at 2010 – apparently the start of the “smartphone era”. This shows how arbitrary the reasoning is – and that’s not even mentioning all any other major events that happened around the same time that could easily have affected people’s wellbeing (global financial crisis, anyone?).
…
3: Claiming causality from longitudinal studies. When researchers run a longitudinal observational study – where they give people questionnaires, say, over a period of several months or years – it isn’t the same as running an experiment. That is, you can’t look at the results and conclude that any one thing they measured caused anything else, because it’s still just a series of observations.
Haidt seems to disagree:
“If there is, on average, a change in happiness the week after people quit or reduce their social media time, then we can infer that the change in mood was caused by the change in behaviour the prior week.”
But that would only be true if those people had been made to quit or reduce social media time, and if you’re comparing their mood change to that from people who kept on using it.
If those people decided of their own accord to quit or reduce social media, you can’t make an inference about causes. Why did they reduce their social media time? Maybe it was because their mood changed – meaning you’d have it completely backwards if you assumed social media caused changes in their mood.
Suppose people were arguing about whether the election of the leader Shmoazod was responsible for the mental health crisis. Shmoazod controlled a small number of institutions before 2012, but after 2012, the number of institutions controlled by Shmoazod increased dramatically, and Shmoazod’s supporters were elected to an increasingly large number of positions.
We know that mental health got much worse after Shmoazod was elected—no matter which way we measure mental health, this result is confirmed. Furthermore, Shmoazod got elected in dozens of countries across the U.S. and Europe—and things turned to shit after Shmoazod got elected. Furthermore, there are no other theories that can explain why things turned to shit around that time—the other theories just don’t fit the data.
Additionally, sometimes Shmoazod leadership was rolled out in part of a country. Thus, we can compare similar precincts, where some were controlled by Shmoazod leadership and others were not in the same country. When we do this—when we carry out quasi-experiments and randomized control trials, we get even the staunchest critics of the Shmoazod-caused crisis theory admitting that “there are certainly lots of suggestive studies. And the overall phenomenon of increasing problems, especially among girls, does call out for an explanation.”
We also know that Shmoazod implemented Shmoazod centers—one of his biggest changes. These centers were insanely addictive—they got lots of people to get sucked in, while not being mandatory. Additionally, we know that many people who used to spend lots of time playing with their friends now spend it anti-socially in Shmoazod centers. Shmoazod undeniably had a transformative effect on the childhood of nearly everyone who spent time at Shmoazod centers—as a result of this, now half of people are at Shmoazod centers almost constantly.
We know that girls are more affected by Shmoazod centers than boys. We also know that mental health has gotten more worse among girls than boys. Young people spend much more time at Shmoazod centers—and their mental health has gotten consistently much worse, much faster than older generations.
We also know that people who spend more time at Shmoazod centers are much more miserable than those who don’t. The correlation between hours spent at Shmoazod centers and depression looks like this in the UK.
We also know that after people stop going to Shmoazod centers, their mental health improves. 37% of young people think that mental health is getting worse among young people because of Shmoazod centers. They consistently thought that their own mental health was made worse by attending Shmoazod centers.
There are lots of plausible mechanistic accounts by which Shmoazod centers make people worse off.
The things that make people happiest like spending time with friends and family aren’t done at Shmoazod centers. Thus, when people are spending hours a day at Shmoazod centers, it’s plausibly making them depressed.
Since the rise of Shmoazod centers, there has been much less socialization, as people spend time at Shmoazod centers rather than hanging out with friends.
Shmoazod centers regularly bring in models who are very attractive, leading to many people feeling self-conscious about their bodies.
Shmoazod centers have strict norms that people are often afraid of violating.
They’re very addictive—thus, they cause people to be constantly thinking about them even when they’re not at Shmoazod centers.
People often spend time at Shmoazod centers resulting in worse sleep.
Bullying is common at Shmoazod centers.
People often feel they get sucked in and waste hours of time doing pointless, addictive, meaningless things. The time spend at Shmoazod centers is a whirlwind of dopamine hits, leading to feelings of meaninglessness—like one is wasting hours of their time.
Additionally, people with stronger opinions about how to run Shmoazod centers were more likely to be depressed.
In this case, the Shmoazod causes mental illness hypothesis is overwhelmingly plausible—it best explains dozens of disparate facts.
As those of you with maximally sensitive antennae might have been able to ascertain, this was a metaphor for social media. Ritchie rightly points out that correlation is not causation, which, while true, does not mean that we should just ignore robust correlative data. Correlation will often be strong evidence for causation even though it does not suffice to show causation.
When we’re comparing explanations in the social sciences, we’ll try to conduct an inference to the best explanation—to figure out what best explains some large-scale phenomena. The social media hypothesis is, by far, the best explanation—just like the Shmoazod explanation is in the thought experiment I gave. None of the evidence may, by itself, suffice to show the truth of the hypothesis, but collectively they make a strong case. Let’s compare the theoretical commitments of those who believe the social media hypothesis vs those who don’t.
Commitments of those who believe social media caused decline in mental health
Mental health began to get worse because social media became popular. This is for all the well-understood and previously described reasons that social media makes people miserable.
This is a very minimal commitment. The mechanistic explanation is very simple, it requires invoking only things that we already know exist, and is able to explain all of the data. This is the gold standard in social science hypotheses.
In contrast, here are the things one has to believe if they are to maintain the hypothesis that something other than social media caused the increase in misery.
Some mysterious phenomena began to destroy the mental health of people starting around 2012, coincidentally the same time that social media was rolled out. Whatever this was, it was both sufficiently significant to destroy the mental health of a generation, and so hard to detect that no one has been able to figure out what it is. Remember, there are no other theories on offer that explain the data.
This occurred across dozens of countries in the US and Europe. Thus, whatever is tanking their mental health has to be international.
The studies that Ritchie admits are “suggestive” turn out not to pan out, and to be inadequate to show the conclusion.
This mysterious force that is destroying the mental health of young people also is hurting the mental health of old people, but less than young people. Coincidentally, young people use social media more.
It’s also hurting girls more than boys—something which we’d once again expect on the social media hypothesis, while we’d have no reason to expect on alternative hypotheses.
Despite almost half of young people spending about half of their time on social media, which is insanely addictive, has dramatically reduced socialization, causes lots of bullying, makes them self-conscious, and harms mental health in other ways, somehow this has failed to significantly hurt their mental health.
The significant correlation between social media use and declining mental health is just correlation—thus, people who are depressed spend more time on social media.
Suppose we were having a conversation in 2010 about whether social media would cause rising rates of misery, assuming that it become very popular and radically changed children’s childhoods. It seems reasonable to give that maybe 30% odds to that hypothesis—and this is a conservative estimate. Thus, let’s use Bayes theorem starting with an odds ratio of 3:7 (which means that for every 7 possibilities where it doesn’t damage mental health, there are three where it does). This is just the same as saying that we should give this 30% odds, but will make the math easier. This is very plausible—something that engulfs huge portions of people’s free time, that has various well-understood mechanisms for harming mental health, and that will radically transform childhoods is very likely to have a deleterious effect on mental health.
One anecdote supporting a high probability of social media tanking mental health: before the first draft of this post, I went to a dining hall to eat food at my college. I spent the meal reading articles on my phone—and when I looked around, I noticed that nearly everyone else was on their phone too—even some of the people with groups of friends. The idea that phones which are so addictive that people are on them even when hanging out with friends would have no deleterious impact on mental health is a relatively implausible one. I’d imagine many were on social media, though I couldn’t check.
So now let’s see the facts that we update on. The two hypotheses are social media causes the mental health crisis and it doesn’t.
Fact 1: There was a mental health crisis after 2012. This is something we’d have no reason to expect on the hypothesis that social media would not cause a mental health crisis. Thus, this is very plausibly something like 5 times as strongly predicted on the hypothesis of social media caused mental health crisis as the alternative one.
So from this fact alone, applying bayes theorem, we’re at 15:7 odds.
Fact 2: It occurred across dozens of countries throughout the US and Europe. Being conservative, let’s say that this is twice as strongly predicted on the social media caused depression hypothesis as on the other hypothesis. So from this, we’re at 30:7 odds.
Fact 3: There are lots of studies that provide at least some evidence for the claim that social media hurts mental health. Let’s say this is twice as strongly predicted on the hypothesis also. So we’re at 60:7 odds.
Fact 4: It has hurt young people’s mental health much more than old people. Let’s assume this is twice as strongly predicted also. Now we’re at 120:7 odds.
Fact 5: It’s hurting girls more than boys. This is perhaps 1.5 times as strongly predicted. So now we’re at 180:7 odds.
Fact 6: There’s a significant correlation between social media use and worsened mental health. This is maybe twice as strongly predicted on the hypothesis. So we end at 360:7 odds—which is 360/367 odds, or just above 98% odds of the hypothesis that it is social media that is destroying mental health.
There is not any strong alternative evidence. Thus, even if we take overwhelmingly conservative estimates, we get the result that there is a very high probability of social media being the cause of the decline in mental health of young people. Unless the studies end up serving as decisive counterevidence, we should think that this hypothesis is almost certainly correct. You don’t actually need robust RCT’s to have about 95% confidence in a hypothesis, just as we can be very confident that the roman empire existed, despite having no randomized control trials showing this.
In a criminal trial, we don’t have any randomized control trials. But if we had a criminal trial where there was only one person in the room where the crime was committed, a plausible account of how he could have killed the victim, and decisive evidence against alternative ways the crime could have been committed, combined with some decently strong forensic evidence, we’d vote to convict. The social media causes mental health decline hypothesis is like that—it is a very plausible account mechanistically, such that before assessing the specific evidence, we should think it pretty likely, and it effortlessly explains all of the data. It’s all very well and good to say that correlation is not causation, but when alternative proposals have to jump through half a dozen hoops, we have good reason to abandon them.
Ritchie points out that it is hard to figure out where to draw the line of when things really started getting worse—2007, 2010, and 2012 are all perfectly good dates. But if we look at when social media really took off, it was somewhat gradual, but around 2012.
Here’s the chart of the number of instagram users—it’s plausible that instagram, snapchat, and tiktok are especially bad for attention spans and especially addictive.
And here’s the chart for snapchat.
Vaynerchuk notes “By August 2014, 40% of 18 year olds in the US were using Snapchat on a daily basis.” Instagram came out in 2010—snapchat in 2012. They began to grow rapidly shortly after release. Thus, we should expect on the social media causes misery hypothesis for things to have grown worse around 2012. Reality accorded with the prediction.
Thus, I think that Ritchie’s complaint that people can’t agree on the date when social media started to take off is a bit confused. There wasn’t a precise date—it grew more and more prominent as mental health got worse and worse.
Ritchie’s next claim, that the 2008 financial crisis can explain the decline in mental health, is either false or doesn’t respond to Haidt’s view. Remember, Haidt’s claim is that it was social media that tanked mental health, rather than phones more broadly. Thus, around 2007—which Twenge cited as the date that phones took off—is not relevant to his analysis. Given this, the alternative explanation of the 2008 financial crisis doesn’t explain why mental health did not get worse until about 4 years after the crisis.
Ultimately, I think the problem with Ritchie’s view is that it’s plagued by a naïve scientism. Because there are not super conclusive studies, he concludes that we can’t conclude anything. Now, whether the studies are conclusive is something that I don’t feel fit to weigh in on, but Haid’s team disagrees. But even in the absence of great, high-quality studies, just from inferences about the world, we can think that it’s very likely that social media caused the decline in mental health of adolescents. We can have high confidence in various social phenomena even if we do not have great studies on them, as long as they are the best explanation of various otherwise puzzling trends.
If one is trying to figure out if some drug is efficacious, Ritchie’s methodology works great. After all, the prior probability that some drug will work is very low—so we need overwhelming evidence from randomized control trials. But if we go in with a sufficiently high prior probability, we shouldn’t need super robust evidence from randomized control trials. Such evidence would be nice, but it is far from required.
Of course, the absence of high-quality studies gives us good reason to conduct more high-quality studies. But it doesn’t give us reason to be agnostic. The evidence is relatively clear—social media is almost certainly to blame for the decline in mental health. Numerous converging lines of evidence point to it, such that unless it is soundly debunked by randomized control trials, we should think that it is almost certainly true. And no one—certainly not Ritchie—seems to think that the studies have actually debunked it, they just think they have failed to confirm it. But this assumes that we can’t have good reason, outside of robust studies, to believe the hypothesis. As I’ve argued here, this assumption is false.
I should get a medal for correctly using the phrase more worse.