Michael Mauboussin is the author of The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing (Harvard Business Review Press, 2012), Think Twice: Harnessing the Power of Counterintuition (Harvard Business Press, 2009) and More Than You Know: Finding Financial Wisdom in Unconventional Places-Updated and Expanded (New York: Columbia Business School Publishing, 2008). More Than You Know was named one of “The 100 Best Business Books of All Time” by 800-CEO-READ, one of the best business books by BusinessWeek (2006) and best economics book by Strategy+Business (2006). He is also co-author, with Alfred Rappaport, of Expectations Investing: Reading Stock Prices for Better Returns (Harvard Business School Press, 2001).
Visit his site at: michaelmauboussin.com/
Michael Mauboussin: The Base Rate Book – Integrating The Past To Better Anticipate The Future
“People who have information about an individual case rarely feel the need to know the statistics of the class to which the case belongs.” — Daniel Kahneman
- Successful active investing requires a forecast that is different than what the market is discounting.
- Executives and investors commonly rely on their own experience and information in making forecasts (the “inside view”) and don’t place sufficient weight on the rates of past occurrences (the “outside view”).
- This book is the first comprehensive repository for base rates of corporate results. It examines sales growth, gross profitability, operating leverage, operating profit margin, earnings growth, and cash flow return on investment. It also examines stocks that have declined or risen sharply and their subsequent price performance.
- We show how to thoughtfully combine the inside and outside views.
- The analysis provides insight into the rate of regression toward the mean and the mean to which results regress.
Executive Summary
- The objective of a fundamental investor is to find a gap between the financial performance implied by an asset price and the results that will ultimately be revealed. As a result, investing requires a clear sense of what’s priced in today and possible future results.
- The natural and intuitive way to create forecasts is to focus on an issue, gather information, search for evidence based on our experience, and extrapolate with some adjustment. This is what psychologists call the “inside view.” It is common for the inside view to lead to a forecast that is too optimistic.
- Another way to make a forecast is to consider the outcomes of a relevant reference class. This is called the “outside view.” Rather than emphasizing differences, as the inside view does, the outside view relies on similarity. Using the outside view can be unnatural because you have to set aside your own information and experience as well as find and appeal to an appropriate reference class, or base rate.
- Most executives and investors rely on their memory of prior instances as a basis for comparison. For example, they may deem this private equity deal similar to that prior deal, and hence assume the return on investment will be similar. An appropriate reference class is one that has a sample size that is sufficient to be robust but is similar enough to the class you are examining to be relevant.
- Research in psychology shows that the most accurate forecasts are a thoughtful blend of the inside and the outside views. Here’s a helpful guide: If skill determines the outcome, you can rely more on the inside view. If luck plays a large role, you should place more weight on the outside view.
- Regression toward the mean is a tricky concept that most investors believe in but few fully understand. The concept says that outcomes that are far from average will be followed by outcomes with an expected value closer to the average. Examining correlations allows us to not only acknowledge the role of regression toward the mean, but also to understand its pace. The data in this book not only offer a basis for an assessment of the rate of regression toward the mean, but also document the mean, or average, to which results regress.
- This book provides the base rates of corporate performance for sales growth, gross profitability (gross profits/assets), operating leverage, operating profit margin, earnings growth, and cash flow return on investment (CFROI®). In most cases, the data go back to 1950 and include dead companies. It also examines stocks that have declined or risen sharply, and shows the subsequent price performance based on how the stocks screen on momentum, valuation, and quality.
- Integrating the outside view allows an executive or investor to improve the quality of his or her forecast. It also serves as a valuable reality check on the claims of others.
- This report is the result of a deep collaboration with our HOLT team. HOLT® aims to remove the vagaries of accounting in order to allow comparison of corporate performance across a portfolio, a market, or a universe (cross sectional) as well as over time (longitudinal).
Introduction
The objective of a fundamental investor is to find a gap between the financial performance implied by an asset price and the results that will ultimately be revealed. A useful analogy is pari-mutuel betting in horse racing. The odds provide the probability that a horse will win (implied performance) and the running of the race determines the outcome (actual performance). The goal is not to pick the winner of the race but rather the horse that has odds that are mispriced relative to its likelihood of winning.
As a result, investing requires a clear sense of what’s priced in today and possible future results. Today’s stock price, for example, combines a company’s past financial performance with expectations of how the company will perform in the future. Market psychology also comes into play. The fundamental analyst has to have a sense of a company’s future performance to invest intelligently.
There is a natural and intuitive approach to creating a forecast of any kind. We focus on an issue, gather information, search for evidence based on our experience, and extrapolate with some adjustment. Psychologists call this approach the “inside view.”
An important feature of the inside view is that we dwell on what is unique about the situation.2 Indeed, Daniel Gilbert, a psychologist at Harvard University, suggests that “we tend to think of people as more different from one another than they actually are.”3 Likewise, we think of the things we are trying to forecast as being more unique than they are. The inside view commonly leads to a forecast that is too optimistic, whether it’s the likely success of a new business venture, the cost and time it will take to build a bridge, or when a term paper will be ready to be submitted.
The “outside view” considers a specific forecast in the context of a larger reference class. Rather than emphasizing differences, as the inside view does, the outside view relies on similarity. The outside view asks, “What happened when others were in this situation?” This approach is also called “reference class forecasting.” Psychologists have shown that our forecasts improve when we thoughtfully incorporate the outside view.4
Analysis of mergers and acquisitions (M&A) provides a good example of these contrasting approaches. The executives at the companies that are merging will dwell on the strategic strength of the combined entities and the synergies they expect. The uniqueness of the combined businesses is front and center in the minds of the dealmakers, who almost always feel genuinely good about the deal. That’s the inside view.
The outside view asks not about the details of a specific deal but rather how all deals tend to do. Historically, about 60 percent of deals have failed to create value for the acquiring company.5 If you know nothing about a specific M&A deal, the outside view would have you assume a success rate similar to all deals.
Considering the outside view is useful but most executives and investors fail to do so. Dan Lovallo, Carmina Clarke, and Colin Camerer, academics who study decision making, examined how executives make strategic choices and found that they frequently rely either on a single analogy or a handful of cases that come to mind.6 Investors likely do the same.
Using an analogy or a small sample of cases from memory has the benefit of being easy. But the cost is that it prevents a decision maker from properly incorporating the outside view.
Yet not all instances in a reference class are equally informative. For instance, M&A deals financed with cash tend to do better than those funded with equity. Therefore, a proper analogy, or set of cases, may be a better match with the current decision than a broad base rate. You trade sample size for specificity.
Lovallo, Clarke, and Camerer created a matrix with the columns representing the reference class and the rows reflecting the weighting (see Exhibit 1). The ideal is a large sample of cases similar to the problem at hand.
“Single analogy,” found in the top left corner, refers to cases where an executive recalls a sole analogy and places all of his or her decision weight on it. This is a common approach that substantially over-represents the inside view. As a result, it frequently yields assessments that are too optimistic.
“Case-based decision theory,” the bottom left corner, reflects instances when an executive recalls a handful of case studies that seem similar to the relevant decision. The executive assesses how comparable the cases are to the focal decision and weights the cases appropriately.
The top right corner is reference class forecasting.14 Here, a decision maker considers an unbiased reference class, determines the distribution of that reference class, makes an estimate of the outcome for the focal decision, and then corrects the intuitive forecast based on the reference class. The decision maker weights equally all of the events in the reference class.
Lovallo, Clarke, and Camerer advocate “similarity-based forecasting,” the bottom right corner, which starts with an unbiased reference class but assigns more weight to the cases that are similar to the focal problem without discarding the cases that are less relevant. Done correctly, this approach is the best of both worlds as it considers a large reference class as well as a means to weight relevance.
The scientists ran a pair of experiments to test the empirical validity of their approach. In one, they asked private equity investors to consider a current deal, including key steps to success, performance milestones, and the expected rate of return. This revealed the inside view.
They then asked the professionals to recall two past deals that were similar, to compare the quality of those deals to the project under consideration, and to write down the rate of return for those projects. This was a prompt to consider the outside view.
The average estimated return for the focal project was almost 30 percent, while the average for the comparable projects was close to 20 percent. Every subject wrote a rate of return for the focal project that was equal to or higher than the comparable projects.
Over 80 percent of subjects who had higher forecasts for the focal project revised down their forecasts when given the opportunity. The prompt to consider the outside view tempered their estimates of the rate of return for the deal under consideration. It is not hard to imagine similar results for corporate executives or investors in public markets.
If the outside view is so useful, why do so few forecasters use it? There are a couple of reasons. Integrating the outside view means less reliance on the inside view. We are reluctant to place less weight on the inside view because it reflects the information we have gathered as well as our experience. Further, we don’t always have access to the statistics of the appropriate reference class. As a result, even if we want to incorporate the outside view we do not have the data to do so.
This book provides a deep, empirical repository for the outside view, or base rates, for a number of the key drivers of corporate performance. These include sales growth, gross profitability, operating profit margins, net income growth, and rates of fade for cash flow return on investment (CFROI®). It also offers data for how stocks perform following big moves down or up versus the stock market.
How to Combine the Inside and Outside Views
Daniel Kahneman, a psychologist who won the Nobel Prize in Economics in 2002, wrote a paper with his colleague Amos Tversky called “On the Psychology of Prediction.” The paper, published in Psychological Review in 1973, argues that there are three types of information relevant to a statistical prediction: the base rate (outside view), the specifics about the case (inside view), and the relative weights you should assign to each.7
One way to determine the relative weighting of the outside and inside views is based on where the activity lies on the luck-skill continuum.8 Imagine a continuum where luck alone determines results on one end and where skill solely defines outcomes on the other end (see Exhibit 2). A blend of luck and skill reflects the results of most activities, and the relative contributions of luck and skill provide insight into the weighting of the outside versus the inside view.
For reference, the exhibit shows where professional sports leagues fall on the continuum based on one season. The National Basketball Association is the furthest from luck and the National Hockey League is the closest to it.
For activities where skill dominates, the inside view should receive the greatest weight. Suppose you first listen to a song played by a concert pianist followed by a tune played by a novice. Playing music is predominantly a matter of skill, so you can base the prediction of the quality of the next piece played by each musician on the inside view. The outside view has little or no bearing.
By contrast, when luck dominates the best prediction of the next outcome should stick closely to the base rate. For example, money management has a lot of luck, especially in the short run. So if a fund has a particularly good year, a reasonable forecast for the subsequent year would be a result closer to the average of all funds. There are two analytical concepts that can help you improve your judgment. The first is an equation that allows you to estimate true skill:9
Estimated true skill = grand average + shrinkage factor (observed average – grand average)
The shrinkage factor has a range of zero to 1.0. Zero indicates complete regression toward the mean and 1.0 implies no regression toward the mean at all.10 In this equation, the shrinkage factor tells us how much we should regress the results toward the mean, and the grand average tells us the mean to which we should regress.
Here is an example to make this concrete. Assume that you want to estimate the true skill of a mutual fund manager based on an annual result. The grand average would be the average return for all mutual funds in a similar category, adjusted for risk. Let’s say that’s eight percent. The observed average would be the fund’s result. We’ll assume 12 percent. In this case, the shrinkage factor is close to zero, reflecting the high dose of luck in short-term results for mutual fund managers. You will use a shrinkage factor for one-year risk-adjusted excess return of .10. The estimate of the manager’s true skill based on these inputs is 8.4 percent, calculated as follows:
8.4% = 8% + .10(12% – 8%)
The second concept, intimately related to the first, is how to come up with an estimate for the shrinkage factor. It turns out that the correlation coefficient, r, a measure of the degree of linear relationship between two variables in a pair of distributions, is a good proxy for the shrinkage factor.11 Positive correlations take a value of zero to 1.0.
Say you had a population of violinists, from beginners to concert-hall performers, and on a Monday rated the quality of their playing numerically from 1 (the worst) to 10 (the best). You then have them come back on Tuesday and rate them again. The correlation coefficient would be very close to 1.0—the best violinists would play well both days, and the worst would be consistently bad. There is very little need to appeal to the outside view. The inside view correctly receives the preponderance of the weight in forecasting results.
Unlike the violinists, the correlation of excess returns of mutual funds is low.12 That means that in the short run, returns that are well above or below average may not be a reliable indicator of skill. So it makes sense to use a shrinkage factor that is much closer to zero than to 1.0. You accord the outside view most of the weight in your forecast.
See the full PDF below.
[drizzle][/drizzle]
