How Much Time Is Needed To Detect Evidence Of Investment Skill?

How Much Time Is Needed To Detect Evidence Of Investment Skill?

A recent MarketWatch piece cited a talk in Hong Kong by Economics Nobel Prize winner Professor Robert Merton wherein he discussed the challenges of evaluating investment managers. The following article assumes that the above summary of Professor Merton’s talk is accurate. The piece, and assumedly the talk, argued that, given typical nominal portfolio returns and volatilities, it takes impractically long to detect evidence of investment skill. The argument claimed to prove that all manager selection is futile. Instead, it proved that naïve nominal performance metrics are of little use.

Get The Timeless Reading eBook in PDF

Get the entire 10-part series on Timeless Reading in PDF. Save it to your desktop, read it on your tablet, or email to your colleagues.

Check out our H2 hedge fund letters here.

Any test of the effectiveness of manager selection is also a test of the analytical process that distills skill. That nominal investment performance is primarily due to factor (systematic, market) noise and thus reverts is well-known. It is thus unsurprising to find flaws in an approach to manager selection that is as antiquated as Ptolemaic Astronomy.

What Value Investors Can Learn From Walter Schloss And Ben Graham Today

Walter SchlossWalter Schloss isn’t a name many investors will have heard today. Schloss was one of the great value investors who trained under Benjamin Graham and specialized in finding cheap stocks. His track record was outstanding. In Warren Buffett’s 1984 essay, the Super Investors of Graham-and-Doddsville, he noted that between 1956 and 1984, Schloss’s firm returned Read More

In this article, we will illustrate the difference between a naïve attempt to detect evidence of investment skill using nominal returns and a more productive effort relying on alphas (residual, security selection, stock picking returns) isolated using a capable modern multi-factor equity risk model. Whereas the former approach is futile at best, the latter approach is successful. In fact, rather than taking decades, a capable modern system can identify skill with high confidence in months.

Detecting Evidence of Investment Skill Using Nominal Returns

Consider nominal returns of a Portfolio and a Benchmark. The Portfolio is a live long-only fund implementing a Smart Beta active investment strategy:

Portfolio’s and Benchmark’s Cumulative Returns

                           Portfolio Benchmark
 Annualized Return            0.1336    0.1433
 Annualized Std Dev           0.0879    0.1093
 Annualized Sharpe (Rf=0%)    1.5194    1.3115

With a heroic assumption that log returns follow a normal distribution, a t-test appears to confirm Professor Merton’s argument. Even with over six years of data, the returns are too noisy for a statistical inference:

Chart of the distribution of Portfolio’s returns relative to the Benchmark used to detect evidence of investment skill

Distribution of Portfolio’s Returns Relative to the Benchmark

    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 -6.1441 -1.2186 -0.0201 -0.1149  1.2481  5.4068 
      One Sample t-test
 t = -0.4607, df = 78, p-value = 0.6768
 alternative hypothesis: true mean is greater than 0
 95 percent confidence interval:
  -0.5300        Inf

Detecting Evidence of Investment Skill Using Alphas/Residuals

By comparison, consider the same Portfolio’s residual returns, or alphas, for the same period, isolated with the AlphaBetaWorks’ standard Long-Horizon Statistical U.S. Equity Risk Model. These are also the returns Portfolio would have generated if its factor exposures had been fully hedged (its returns factor-neutralized, or residualized) using the Model:

Chart of the absolute cumulative residual (alpha, security selection, stock picking returns) for the Portfolio

Portfolio’s Cumulative Residual/Alpha

With an equally questionable assumption that log residuals follow a normal distribution, a t-test is now highly statistically significant:

Chart of the distribution of Portfolio’s residual returns used to detect evidence of investment skill

Distribution of the Portfolio’s Residuals/Alphas 

    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 -1.5300 -0.2064  0.2643  0.2620  0.7289  2.3663 
      One Sample t-test
 t = 3.3126, df = 78, p-value = 0.0007
 alternative hypothesis: true mean is greater than 0
 95 percent confidence interval:
  0.1303         Inf

Whereas Professor Merton’s argument does indeed apply to nominal returns, it does not apply to their residuals. A critical difference is the lower dispersion of residual returns. Over 90% of the variance of a typical active equity portfolio is due to factor exposures rather than to stock picking. Therefore, using nominal returns to measure skill is like trying to take a baby’s temperature by examining her bath water, rather than the baby herself.

Whereas at least 67 out of 100 monkeys picking stocks at random are expected to outperform the Portfolio, less than 1 out of 1,000 is expected to generate higher residuals – a highly statistically significant result. Thus, with the help of a capable equity risk model, strong evidence of skill can be identified in months rather than in decades.

Converting Residuals into Nominal Outperformance

Assuming the equity risk model uses investable factors, as AlphaBetaWorks’s models do, the residual return stream above is investable. In fact, in the idealized case of costless leverage, positive residual returns can be turned into outperformance relative to any benchmark. Below is the performance of Portfolio after it is hedged to match the factor exposures of the Benchmark. The evidence of skill is now plainly visible in the naïve absolute and relative nominal return metrics:

Chart of the absolute cumulative returns for the Portfolio hedged to match the factor exposures of the Benchmark, the Benchmark, as well as Portfolio’s cumulative return relative to the Benchmark

Cumulative Returns for the Portfolio Hedged to Match the Benchmark and the Benchmark

                          Portfolio with Benchmark Risk  Benchmark
 Annualized Return                                0.1784    0.1433
 Annualized Std Dev                               0.1168    0.1093
 Annualized Sharpe (Rf=0%)                        1.5276    1.3115


  • Since factor noise dominates nominal returns, the use of nominal returns to detect evidence of investment skill takes far too long to be practical.
  • After distilling stock picking performance (alphas, residual returns) from factor noise, statistically significant evidence of investment skill can become evident in months, rather than in decades.
  • Hedging makes it possible to turn positive stock picking returns into nominal outperformance with respect to any benchmark.

The information herein is not represented or warranted to be accurate, correct, complete or timely.

Past performance is no guarantee of future results.

Copyright © 2012-2018, AlphaBetaWorks, a division of Alpha Beta Analytics, LLC. All rights reserved.

Content may not be republished without express written consent.

Article by AlphaBetaWorks Charts

AlphaBetaWorks provides risk management, skill evaluation, and predictive performance analytics. Developed by finance and technology veterans, our proprietary platform combines the latest advances in financial risk modeling, data processing, and statistical analysis. Our Risk Analytics are more robust than alternatives and our Skill Analytics are predictive. Risk Analytics AlphaBetaWorks pinpoints risks missed by other offerings and delivers unique insights. AlphaBetaWorks Risk Analytics were developed by investment professionals seeking usability and a deeper understanding of portfolio exposures. Predictive Performance Analytics Starting with robust, proprietary risk models, AlphaBetaWorks adds layers of attribution and statistical analysis. Our Skill Analytics describe a multitude of specific skills that are strongly predictive of future returns for any fund, manager, or analyst with a sufficient sample of investment history. The AlphaBetaWorks Advantage Our Risk and Performance Analytics provide unique insights: For portfolio managers, we identify overlooked exposures, hidden risk clusters, and crowded bets. Managers can focus on risks in areas where they have proven ability to generate excess returns and avoid undesired risks in areas where they do not. For fund allocators, we identify the skills, crowding, and hidden portfolio bets of individual funds and portfolios of funds. Allocators can identify differentiated and skilled managers that are deploying capital in areas of proven expertise – and more importantly, those that are not. Background As finance professionals, we spent the last decade focused on fundamental investment analysis and the study of great (and seemingly great) investment managers. We asked of ourselves: Where are the unintended risks in a portfolio? What is the chance that a manager possesses true investment skill and was not just lucky? Does investment skill persist and is past skill a predictor of future results? There was no product, service, or technology that rigorously and consistently answered these questions. With decades of fundamental investment analysis, risk management, mathematics, and technology expertise, AlphaBetaWorks professionals have developed risk and skill analytics to address these and related questions.
Previous article Prehistoric Baby Bird Fossil Reveals Many Similarities With Today’s Birds
Next article Five Keys To Entrepreneurial Infrastructure Funding

No posts to display