Explore how investors are commonly fooled by current performance metrics. Learn how to correct for these common mistakes and allocate to managers that have the highest probability of repeating good performance and avoid redeeming from high-quality managers that have experienced some bad luck.
Three Forces Contributing to Type I Errors
- There are a huge amount of Type I errors. Why?
- Evolutionary propensity
to tolerate Type I error - Randomness
With enough tests, something will look “significant” - Rare effects
We incorrectly ignore prior beliefs leading to a high error rate
- Evolutionary propensity
Tradeoff
- We have a very high tolerance for Type I error
- There is a tradeoff of Type I and Type II errors
- For example, if we declared all patients pregnant there would be a 0% Type II error, but a very large Type I error
Evolutionary Foundations
- High Type I error (low Type II error) animals survive
- This preference is passed on to the next generation
- This is the case for an evolutionary predisposition for allowing high Type I errors
- Pigeons put in cage. Food delivered at regular intervals – feeding time has nothing to do with behavior of birds.
- Results
- Skinner found that birds associated their behavior with food delivery
- One bird would turn counter-clockwise
- Another bird would tilt its head back
- A good example of overfitting – you think there is pattern but there isn’t
- Skinner’s paper called:
‘Superstition’ in the Pigeon, JEP (1947) - But this applies not just to pigeons or gazelles…
- Coins the term Apophänie. This is where you see a pattern and make an incorrect inference. He associated this with psychosis and schizophrenia.
- Apophany is a Type I error (i.e., false insight)
- Epiphany is the opposite (i.e., true insight)
- Apophany may be interpreted as overfitting
“…nothing is so alien to the human mind as the idea of randomness.” – John Cohen
Article by Campbell Harvey, PhD – Research Affiliates
See the full slides below.
