GreenWood Investors’s conversation with Phil Tetlock discussing superforecasting in the midst of uncertainty.
Phil Tetlock: Superforecasting in the midst of uncertainty
Q2 hedge fund letters, conference, scoops etc
Transcript
Hi. Over the next week, we're going to be launching a trial borrowing on Phil Tetlock research and clinical experience with unusually great forecasting, which he calls superforecasting. He was at our investor day a few weeks ago, whereby we announced that our 200 investors and close peers are going to be trying to actually out predict the fundamentals in the market multiples of our portfolio. This is going to be a first of its kind study in equity markets. And as we prepare for this trial to launch, we thought we'd share a few bits of our conversation with Phil, this one focusing specifically on Superforecasting in the midst of uncertainty.
If superforecasting is right, and and super teams have an ability to outperform experts, and why is the stock market not always right?
Of course, superforecasting, forecasters or superforecasting teams are often wrong, or on the wrong side of me.
Yeah.
That would be the first observation. The second would be either stock market were efficient, and were analysing all the available information optimally constantly. It still could be wrong, because we live in a stochastic world. And it could be the perfect admission process or could still have invert accuracy, just picking the structure of the world. If you believe in science critics, but God does play dice with the cosmos.
Really interesting, the slide you showed on 400 days versus 150 days, because in the market, really the ability to very early predict the accurate outcome is what drives I would say two thirds three quarters of your performance. When you when you study your super forecasters did they all have that ability to forecast early or were certain subsets better at early stage forecasting?
There's definitely quite a bit of variance among super forecasts and the data that I was showing there were based on aggregate data. So that was that was an aggregate of the most recent forecasts of the best within the time to time constraints, read the up to date forecast at the best forecasters aggregated as a function of cognitive diversity, which means giving more weight than people who would normally disagree. Okay, so when you have those properties, and you're using that extreme rising algorithm that I described, you're turning the 70% in five, yeah, you using that you're getting odds estimates that are as accurate 400 days out as the ones you would get from regular forecasts are 150 days out, which means you're still not getting deterministic answers. You're saying well, that know the regulatory forecasters are said assessing and probability as 80% and super podcasters. They're saying that 95% Yeah. Much hinges on where you say your decision threshold for action. That's a decision making function, not a forecasting function. And USA crew. I will when when do I think I have some fish? Yes. Odds to act? Yeah, of course. I, how what what does the probability threshold have to be for my belief about this trial? to change my views about me is yes. So okay, delta fell below 35%. But what I do what I felt 10% over it.
Yeah. Okay. But before we before we unpack that I wanted to so it sounds I just want to quickly summarise so when people who normally disagree, agree, that's when the accuracy of the prediction early stage is strongest. Is that correct?
As an important signal? Yes. Okay. Got it. When Trump and Trump supporters and Trump's critics agree, tells you something.
So the signal from God.
You talked about Charlie Munger a few times in the book, who iss the role model of probably half the audience actually. And he talks about, you know, capital research and management, they had a best ideas fund. And they they the best ideas is where all the portfolio managers are spend their all it's like my CTT, right, like they would take all these portfolio managers and they say, give me your best one and that actually underperformed and he caught... he basically, excoriates... not excoriates, he criticises the amount of time that people spend on something doesn't correlate to the right outcome. So I'm curious, how do you come up with you talk about a little bit of I'd love to have a discussion about it. How do you come up? How do you spend enough time to get to know the subject correctly, but then not get to that point where your incremental information is actually hurting you in some respects? Does does that question make sense?
It makes a lot of sense. Each one of us is constantly tacitly making decisions about how to expand our finite amount of mental energy. And you have to make a judgement call about whether or not investing this amount of mental energy.