For over a century, researchers have developed strategies to forecast equity returns, only to see others determine that such strategies do not outperform the market.

Thorough surveys of the history of these studies can be found in Huang and Zhou (2013), Scholz, Nielsen, and Sperlich (2013), Rapach and Zhou (2012), and Campbell and Thompson (2008). An early notable strategy is the approximately 255 Wall Street Journal editorials written by Charles H. Dow (1851-1902). Though Dow never used the term itself, Dow’s works were organized and represented as “Dow Theory.” Later, Cowles (1933), in “Can Stock Market Forecasters Forecast?” tracked the Dow Theory forecasts and found that they underperformed the market by about 3.5% a year. Cowles also found that recommendations by 24 other publications underperformed by 4% a year. From Cowles (1933) through the mid-1980s, it was generally considered that market returns were not predictable. Major research supporting this view included those of Godfrey, Granger and Morgenstern (1964), Fama (1965), Malkiel and Fama (1970), and Malkiel’s (1973) book, A Random Walk Down Wall Street.

The 1980’s, however, saw a surge of research backing up the claim that market returns could be forecasted. The research supported a variety of variables:

  • Book to Market: Kothari and Shanken (1997), Pontiff and Schall (1998), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Consumption Wealth Ratio: Lettau and Ludvigson (2000), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Corporate Activities: Lamont (1988), Baker and Wurgler (2000), Boudoukh, Michaely,
  • Richardson, and Roberts (2007), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Dividend Yields: Hodrick (1982), Rozeff (1984), Fama and French (1988), Campbell and Shiller (1988a, 1988b), Nelson and Kim (1993), Kothari and Shanken (1997), Lamont (1998), Lettau and Van Nieuwerburgh (2008), Cochrane (2008), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Economic Combined with Technical: Huang and Zhou (2013);
  • Earnings: Fama and French (1988), Campbell and Shiller (1988a, 1988b), Lamont (1998), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Inflation Rate: Nelson (1976), and Fama and Schwert (1977), Campbell and Vuolteenaho (2004), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Interest Rates & Bond Yields: Fama and Schwert (1977), Keim and Stampaugh (1986), Campbell (1987), Breen, Glosten, and Jaganathan (1989), Fama and French (1989), Campbell (1991), Ang and Bekaert, (2007), Welch and Goyal (2008), Campbell and Thompson (2008);
  • Relative Valuations of High and Low Beta Stocks: Polk, Thompson, and Vuolteenaho (2006);
  • Stock Volatility: French, Schwert, and Stambaugh (1987), Guo (2000), Goyal and Santa-Clara (2003), Welch and Goyal (2008), Campbell and Thompson (2008).

Equity returns forecast available through several variables

However, after a growing acceptance that several variables were able to forecast equity returns, arguments disputing those claims returned, the most prominent of which comes from Goyal and Welch (2007). Their study reexamined “the performance of variables that have been suggested by the academic literature to be good predictors of the equity premium,” and, based on extensive out-of-sample testing, they found that these models “would not have helped an investor with access only to available information to profitably time the market.” Goyal and Welch also brought out-of-sample testing to widespread, if not universal, acceptance as a benchmark for testing investment strategies. Goyal and Welch’s findings brought a response from Campbell and Thompson (2008), which accepted the use of out-of-sample results, but “show that many predictive regressions beat the historical average return once weak restrictions are imposed on the signs of coefficients and return forecasts.” Campbell and Thompson’s response appeared to accelerate research into alternative methods of identifying and testing forecasting variables. Rapach and Zhou (2012) covered this topic thoroughly, and, in brief, show that “recent studies provide forecasting strategies that deliver statistically and economically significant out-of-sample gains, including strategies based on:

  • economically motivated model restrictions (e.g., Campbell and Thompson, 2008; Ferreira and Santa-Clara, 2011);
  • forecast combination (e.g., Rapach et al., 2010);
  • diffusion indices (e.g., Ludvigson and Ng, 2007; Kelly and Pruitt, 2012; Neely, Rapach, Tu, and Zhou, 2012);
  • regime shifts (e.g., Guidolin and Timmermann, 2007; Henkel, Martin, and Nadari, 2011; Dangl and Halling, 2012).”

Both efficient market theorists and their critics continue to have strong proponents on each side. Evidence that both sides of the field are highly respected is the concurrent awarding, in 2013, of the Nobel Prize in economics to both Eugene Fama, a proponent of efficient markets, and Robert Shiller, who claims markets are irrational.

forecasting equity returns

See full Forecasting Equity Returns: An Analysis of Macro vs. Micro Earnings in PDF format here.