Can You Trust Investment Research? – Special Edition
April 28, 2016
by Jeffrey Briskin
Investment strategy research has become a cottage industry, with hundreds of studies published every year claiming that actively managed strategies, from factor-based investing to market timing, have greater potential to generate alpha. Those claims are usually based on backtesting performance over various market timeframes.
Campbell Harvey views most of this research with a high degree of skepticism.
Harvey is the J. Paul Sticht Professor of International Business at Duke University’s Fuqua School of Business in Durham, North Carolina, and a research associate of the National Bureau of Economic Research in Cambridge, Massachusetts. He also serves as president of the American Finance Association.
We previously wrote about Harvey’s work here. But in an April 5 presentation at The Journal of Portfolio Management Research Summit in Boston, Harvey elaborated on his research and voiced his skepticism of published investment research that is often misleading or based on false assumptions. His presentation summarized some of the ideas that appeared in Backtesting, an article he co-wrote with Professor Yan Liu of Texas A&M that was originally published in 2015 in The Journal of Portfolio Management.
He began his presentation by demonstrating that researchers – and advisors and investors who act upon their research – often fall victim to committing “type 1” and “type 2” errors.
A type 1 error occurs when someone acts on a belief when the evidence clearly demonstrates that this belief is false. An example is when an antelope flees after hearing what it believes to be the sound of a stalking cheetah when in reality there are no cheetahs for miles around.
Type 2 errors occur when someone fails to act even when his or her belief in a certain reality is proven false. An example would be the same antelope, believing that there are no cheetahs nearby, fails to flee when one suddenly appears in front of him.
Type 1 errors are hard-wired into our DNA. Without them, we wouldn’t react quickly to threats – real or imagined. The cost of making a type 1 error is, in general, relatively small. Fleeing from a false alarm costs time and energy, but we live to tell the tale of our mistake. Committing a type 2 error, on the other hand, can lead to death, injury or other catastrophes.
Investors and advisors often commit type 1 errors, such as timing a trade with the mistaken belief that certain market or economic events will result in a large gain or protect against loss. Of course, sometimes those bets pay off, but over the long run, repeated use of these same tactics – especially in short-term trading situations – rarely results in market-beating performance for anyone other than the most experienced traders with the most sophisticated research and technology at their disposal.
Likewise, investors who commit type 2 errors, such as irrationally refusing to bail out of a free-falling stock of a company on the verge of bankruptcy, can lose their entire investment.
Since most investment professionals aren’t likely to commit many type 2 errors, Campbell’s presentation and paper focused on how advisors can detect type 1 errors in investment research and use this knowledge to determine whether a given strategy merits serious consideration.
PDF | Page 2
