Wharton’s Barbara Mellers and Michael Platt discuss their research on superforecasters.
While nobody can get forecasts right 100% of the time, research shows that there are certain kinds of people who are better at forecasting outcomes than others. Wharton marketing professors Barbara Mellers and Michael Platt, who are also Penn Integrates Knowledge (PIK) professors at the University of Pennsylvania, examine the intersection of marketing, psychology and neuroscience to understand the traits that “superforecasters” share and that can lead to better decision making. They recently talked about their research and its implications on the [email protected] show on Wharton Business Radio on SiriusXM channel 111. (Listen to the podcast at the top of this page.)
An edited transcript of the conversation follows.
[email protected]: You’re doing a lot of research right now into the mindset of people and what goes into the decision-making process.
Michael Platt: Right. In fact, we are going beyond the mindset. We are going into the mind and into the brain. The whole purpose of our research program is to try to understand the process by which people make decisions. If we can understand how that process unfolds, all the myriad factors that go into it, we might be able to shape that decision process and help people make better decisions.
[email protected]: But there’s so much forecasting done on a variety of different things. The forecasting that surrounded the presidential election a few months ago went one way, but the result went a different way.
Barbara Mellers: It sure did. I think people look at the forecasts and say, “How did they get it so wrong?” There’s only two ways to get a forecast wrong. That is if you say zero or one. There’s a whole range of possibilities that don’t necessarily mean you’re wrong in between that. Nate Silver gave some of the most accurate forecasts about whether Donald Trump would win. His estimates were around a 67% chance that Hillary Clinton would win, 33% for Trump.
Now, Trump wins. Is that wrong? No. He’s on the wrong side of maybe. But 33% of the time, if you ran these counterfactual trials in history, Trump would win, according to Nate Silver. So, it’s very tough to say somebody’s making the wrong prediction unless they go way out on the extremes.
[email protected]: But these are people who have done this for a while, who are smart and probably more right than wrong. The job that they’re doing is a very important one, and they’re doing it properly for the most part, correct?
Mellers: I think they are. Let’s put it like this: The world is an incredibly difficult place to predict. We’ve got to give them that. I don’t think any of us realized how close the Trump election would be and how close Brexit would be. If it was easy, it would have been done before.
[email protected]: What goes through the brain when these things are going on?
“The world is an incredibly difficult place to predict.” –Barbara Mellers
Platt: I’d like to return to this question overall. When we are looking at Trump vs. [Clinton], it was clear that there should be some very predictable outcome there. But if you looked at what you might call base rates, and the fact that this country is so evenly divided, in the end people really came home and voted according to their parties. I think that that probably explains a lot of it.
When we think about these kinds of collective decisions, those are the most complicated ones that we make. Because we take into account not only what you might think of as the economic and rational impacts on ourselves, but also there are all these social factors that come into play. Emotional factors. I think that was really key during this election, where you’re not even aware of it…. Maybe you privately think you’re going to vote for Trump. Maybe you don’t even know until you go into the voting booth.
Mellers: The shy Trump voter is the hypothesis.
Platt: That is, I think, a big part of it. But we’re speculating based on behavior and on what people say they do, or what they intend to do or how they feel about it. What we can do with neuroscience is maybe uncover the processes that are actually going into that decision. Many things have to come together. But in the end, you can only do one thing or another. Pull this lever or that one.
[email protected]: The emotional part is maybe the key component, especially with what we saw a few months ago. Now, we have people who are emotional about the candidate who won, whether he is doing something good or bad. So, anger was a very powerful emotion in this, was it not?
Platt: Absolutely. It’s hard to distinguish many of these emotions. I think people are certainly very worked up. They are very keen to believe in their own side. I think that’s another thing. It’s very difficult take the other side, to view things through the eyes of another individual. I think that’s something that Barb has worked on as well.
Mellers: When we look at the best forecasters, whom we call superforecasters in the research that we’ve done, they tend to be much more analytical, much more rational. They score higher on measures of actively open-minded thinking. These folks, who also were on the wrong side of maybe in our research when it came to Hillary and Trump, do step back and take an analytical look at it and try to keep emotions out of it. Maybe not out of it, but at least not getting in the way of it.
[email protected]: Is it a hard thing to keep emotions out of some of these decisions?
Platt: Frankly, that goes against biology. Emotions are evolved for a very important reason. It’s a simple and intuitive notion to think of our emotional self and our rational self as being completely separated. In fact, our brains integrate those processes every time you make a decision. Emotions are important. They’re an important part of the forecasting process.
Essentially, you should think about your brain as not just making predictions about the election, but about everything that you do — every single event that might happen in the world. That is, are they more rewarding, more pleasant, more aversive than you might have expected? Social emotions, jealousy, fear, anger, etc. are all going to shape that process of making a prediction or responding to the outcome of a prediction. Emotions help us to learn from those outcomes and, hopefully, make better decisions in the future. That’s sort of my evolutionary psychology/neuroscience view on it. But then again, to the degree that you can potentially be aware of those emotions, you might be able to be a little more rational.
Mellers: They’re signals that we ought to be paying attention to something. That’s essential. We’re learning a lot right now about how to make better forecasts. That’s going to influence all aspects of our life because we’re constantly making predictions about who we want to spend time with, how we want to spend our money, whom we want