Valuation-Informed Indexing #311
by Rob Bennett
Nate Silver posted a fine article a few months ago (How I Acted Like a Pundit and Screwed Up on Donald Trump: Trump’s Nomination Shows the Need for a More Rigorous Approach). The article makes a number of points that are relevant in the investing realm and help to explain why there are still smart and good people recommending Buy-and-Hold strategies 35 years after Shiller published his “revolutionary” (his word) research showing that valuations affect long-term returns.
Silver writes: “I’ve seen a lot of critical self-assessments from empirically minded journalists…about what they got wrong on Trump. This instinct to be accountable for one’s predictions is good since the conceit of ‘data journalism,’ at least as I see it, is to apply the scientific method to the news. That means observing the world, formulating hypotheses about it, and making those hypotheses falsifiable. (Falsifiability is one of the big reasons we make predictions.) When those hypotheses fail, you should re-evaluate the evidence before moving on to the next subject. The distinguishing feature of the scientific method is not that it always gets the answer right, but that it fails forward by learning from its mistakes.”
Silver is describing what I saw as the primary benefit of Buy-and-Hold in the days when I still believed in Buy-and-Hold. Historically, investing advice has been opinion. Some who spoke to the subject offered better reasoned opinions than others, to be sure, and many of the opinions on investing offered in pre-Buy-and-Hold days were rooted in statistical analyses of some kind. The work product of these efforts could fairly be described as something better than mere opinion. But the Buy-and-Holders were the first to transform investing analysis into something at least resembling science.
Buy-and-Holders cite peer-reviewed research to support their claims. Academic researchers in recent decades have made the study of how to invest effectively a matter of systematic study. The work done by one researcher builds on the work of the researchers who came before him. Following such a process, a scientific process, knowledge advances over the course of time.
In theory.
Silver points to the flaw in human nature that caused the downfall not only of those who dismissed Trump’s chances of winning the Republican nomination but of those who grew so emotionally attached to the Buy-and-Hold dogmas that they became incapable of incorporating new insights into their thought processes when they were presented to them. “The distinguishing feature of the scientific method is not that it always gets the answer right, but that it fails forward by learning from its mistakes,” he says. The tragedy of the last 35 years is that the Buy-and-Holders stopped falling forward on the day in 1981 when Shiller showed them that it really is investor emotion that drives stock price changes rather than unforeseen economic developments.
It’s not just that the Buy-and-Holders stopped moving forward when Shiller published his revolutionary research and they elected to ignore the advance. Buy-and-Hold is a numbers-based model. That’s a plus. Numbers-based models are falsifiable. Which means that they can be checked, as Silver notes. Which means that they can be discovered wanting and corrected.
What if they aren’t? A numbers-based model that is not corrected when found to be in error can no longer be said to be scientific. An uncorrected numbers-based model is the opposite of science. It has the appearance of science and thus possesses an unmerited credibility in the eyes of those making use of it (millions of middle-class investors!). But it no longer possesses the reliability of a model that is being corrected on an ongoing basis through regular application of the scientific check-and-correct process.
Numbers-based models that lack scientific integrity get the numbers wrong. They appear authoritative to those who do not have the time to devote enough study to them to discover their flaws. But they generate conclusions that are no better than those generated by guesswork. Models that are not corrected when discovered to be in error are fake science.
Silver writes: “We didn’t just get unlucky: We made a big mistake, along with a couple of marginal ones. The big mistake is a curious one for a website that focuses on statistics. Unlike virtually every other forecast we publish at FiveThirtyEight — including the primary and caucus projections I just mentioned — our early estimates of Trump’s chances weren’t based on a statistical model. Instead, they were what we “subjective odds” — which is to say, educated guesses. In other words, we were basically acting like pundits, but attaching numbers to our estimates. And we succumbed to some of the same biases that pundits often suffer, such as not changing our minds quickly enough in the face of new evidence. Without a model as a fortification, we found ourselves rambling around the countryside like all the other pundit-barbarians, randomly setting fire to things.”
He adds: “Without having a model, I found, I was subject to a lot of the same biases as the pundits I usually criticize. In particular, I got anchored on my initial forecast and was slow to update my priors in the face of new data. And I found myself selectively interpreting the evidence and engaging in some lazy reasoning.”
Those are good words. Silver has learned his lesson. Those of us claiming to offer investing advice rooted in a scientific process need to hear them and learn from them. These words of caution don’t apply only to my Buy-and-Hold friends, of course. Valuation-Informed Indexers can easily fall into the same traps. Science is performed by humans and humans are flawed creatures.
Rob Bennett’s bio is here.
