Momentum isn’t magic – vindicating the hot hand with the mathematics of streaks
It’s NCAA basketball tournament season, known for its magical moments and the “March Madness” it can produce. Many fans remember Stephen Curry’s superhuman 2008 performance where he led underdog Davidson College to victory while nearly outscoring the entire determined Gonzaga team by himself in the second half. Was Curry’s magic merely a product of his skill, the match-ups and random luck, or was there something special within him that day?
Nearly every basketball player, coach or fan believes that some shooters have an uncanny tendency to experience the hot hand – also referred to as being “on fire,” “in the zone,” “in rhythm” or “unconscious.” The idea is that on occasion these players enter into a special state in which their ability to make shots is noticeably better than usual. When people see a streak, like Craig Hodges hitting 19 3-pointers in a row, or other exceptional performances, they typically attribute it to the hot hand.
The hot hand makes intuitive sense. For instance, you can probably recall a situation, in sports or otherwise, in which you felt like you had momentum on your side – your body was in sync, your mind was focused and you were in a confident mood. In these moments of flow success feels inevitable, and effortless.
However, if you go to the NCAA’s website, you’ll read that this intuition is incorrect – the hot hand does not exist. Belief in the hot hand is just a delusion that occurs because we as humans have a predisposition to see patterns in randomness; we see streakiness even though shooting data are essentially random. Indeed, this view has been held for the past 30 years among scientists who study judgment and decision-making. Even Nobel Prize winner Daniel Kahneman affirmed this consensus: “The hot hand is a massive and widespread cognitive illusion.”
Nevertheless, recent work has uncovered critical flaws in the research which underlies this consensus. In fact, these flaws are sufficient to not only invalidate the most compelling evidence against the hot hand, but even to vindicate the belief in streakiness.
Research made it the ‘hot hand fallacy’
In the landmark 1985 paper “The hot hand in basketball: On the misperception of random sequences,” psychologists Thomas Gilovich, Robert Vallone and Amos Tversky (GVT, for short) found that when studying basketball shooting data, the sequences of makes and misses are indistinguishable from the sequences of heads and tails one would expect to see from flipping a coin repeatedly.
Just as a gambler will get an occasional streak when flipping a coin, a basketball player will produce an occasional streak when shooting the ball. GVT concluded that the hot hand is a “cognitive illusion”; people’s tendency to detect patterns in randomness, to see perfectly typical streaks as atypical, led them to believe in an illusory hot hand.
GVT’s conclusion that the hot hand doesn’t exist was initially dismissed out of hand by practitioners; legendary Boston Celtics coach Red Auerbach famously said: “Who is this guy? So he makes a study. I couldn’t care less.” The academic response was no less critical, but Tversky and Gilovich successfully defended their work, while uncovering critical flaws in the studies that challenged it. While there remained some isolated skepticism, GVT’s result was accepted as the scientific consensus, and the “hot hand fallacy” was born.
Importantly, GVT found that professional practitioners (players and coaches) not only were victims of the fallacy, but that their belief in the hot hand was stubbornly fixed. The power of GVT’s result had a profound influence on how psychologists and economists think about decision-making in domains where information arrives over time. As GVT’s result was extrapolated into areas outside of basketball, the hot hand fallacy became a cultural meme. From financial investing to video gaming, the notion that momentum could exist in human performance came to be viewed as incorrect by default.
The pedantic “No, actually” commentators were given a license to throw cold water on the hot hand believers.
Taking another look at the probabilities
In what turns out to be an ironic twist, we’ve recently found this consensus view rests on a subtle – but crucial – misconception regarding the behavior of random sequences. In GVT’s critical test of hot hand shooting conducted on the Cornell University basketball team, they examined whether players shot better when on a streak of hits than when on a streak of misses. In this intuitive test, players’ field goal percentages were not markedly greater after streaks of makes than after streaks of misses.
GVT made the implicit assumption that the pattern they observed from the Cornell shooters is what you would expect to see if each player’s sequence of 100 shot outcomes were determined by coin flips. That is, the percentage of heads should be similar for the flips that follow streaks of heads, and the flips that follow streaks of misses.
Our surprising finding is that this appealing intuition is incorrect. For example, imagine flipping a coin 100 times and then collecting all the flips in which the preceding three flips are heads. While one would intuitively expect that the percentage of heads on these flips would be 50 percent, instead, it’s less.
Suppose a researcher looks at the data from a sequence of 100 coin flips, collects all the flips for which the previous three flips are heads and inspects one of these flips. To visualize this, imagine the researcher taking these collected flips, putting them in a bucket and choosing one at random. The chance the chosen flip is a heads – equal to the percentage of heads in the bucket – we claim is less than 50 percent.
To see this, let’s say the researcher happens to choose flip 42 from the bucket. Now it’s true that if the researcher were to inspect flip 42 before examining the sequence, then the chance of it being heads would be exactly 50/50, as we intuitively expect. But the researcher looked at the sequence first, and collected flip 42 because it was one of the flips for which the previous three flips were heads. Why does this make it more likely that flip 42 would be tails rather than a heads?