Are Humans willing to give AI a second chance if it doesn’t get it right the first time?

Updated on


There’s certainly a buzz that can be felt right now in the futures-based Alternative Investment space. Bitcoin Futures are in their first full month of trading, VIX based strategies continue to hold the spotlight, and investors and managers alike are trying to figure out how AI should be used in their strategies. These topics were up for discussion at Opalesque’s most recent Chicago Roundtable filled with talented managers, institutional asset allocators, and RCM’s own Managing Director, Jeff Malec.


Matthias Knab of Opalesque moderated the discussion, and there was lots of good stuff covered, from Bitcoin to emerging managers to VIX to Artificial Intelligence in hedge fund strategies.  We found the AI discussion particularly intriguing, and here’s some of the more intricate and nuanced conversation surrounding that:

Craig Weynand of NuWave Investment Management (QEP) speaks about how the conversation around AI has changed since our technological advances over the past decade. He argues that the markets are telling us something every day, and because of our advances, AI can pick up on exactly what the markets are telling us.

In much the same way as advances in AI and machine learning have transformed speech recognition, we have tried to apply similar concepts to better understand the language of the financial markets.

The markets, after all, speak to all of us every day, and we are all trying to understand what it is saying to predict or forecast where it is headed. We may use different means or methods, or focus on different aspects or data to figure out what the market is saying, and predict where it may be headed, but that is essentially the goal of every investment manager. Each piece of market data, whether technical or fundamental, is like an individual syllable that is spoken, and multiple data points, like multiple syllables, form words and sentences, and a collection of words and sentences in turn form descriptive paragraphs – all of which can lead to important inferences, meaningful predictions, informed forecasts, etc

Simply put, today we [AI] have the means to better understand what the markets may be telling us.

Adam Butler, Chief Investment Officer for emerging manager ReSolve Asset Management (read more about their research here) questions how we know AI is doing a good job? How can we humans trust if it’s doing it’s job if we can’t compare it to anything else? And are we comfortable with the data sets?

Consider Quantopian, for example, which was seeded by Steve Cohen for $250 million, and experienced steady losses after launch. The challenges with unsupervised AI type approaches are: how do you benchmark it? How do you know if the strategy has run its course? How do you determine whether there are sustainable barriers to arbitrage if you don’t have good tools that allow you to identify when the alpha sources have decayed to the point where they no longer cover trading costs?

[Further],  to branch out and try to introduce other nontraditional sources of information introduces vast degrees of freedom. All of a sudden, you’re introducing a large number of new system states. Most of the data is only available over the past 10 years or less, in which case 90% of the data you’re going to test it on came from one major economic environment

So, I think you have to pick and choose where you employ this type of supervised or unsupervised learning. There’s all these tricky degrees of freedom, including hyper parameters, in the machine learning problem that I find fascinating, but also that I think merit caution.

Markian Zyga of Lighthouse Partners followed up with comments that the ease of model creation brings with it different challenges.

I absolutely agree, and…most meaningfully in the situation where signals are changing on an ongoing basis: how do you identify the point of degradation within a machine learning approach? I do not have a perfect answer. We aim to understand qualitatively and quantitatively how these different approaches historically have performed across different market environments through ongoing discussions with our portfolio managers and then mutually agree on a pain threshold, a stop-loss level, at which exposure will be reduced.

The point made about Quantopian is an interesting one, and they are not the only group that is pursuing the path of crowd sourcing trading strategies. I think one of the most meaningful and exciting developments within the quant space over the past several years has been the enhanced computational fire power that exists today. And also the proliferation of open source libraries that allow the quantitative community to pursue the research and testing of new strategies more quickly and efficiently than in the past. These developments also provide new challenges in terms of defining research that is not over fit. Perhaps most intriguing, if we are to assume that new strategies can now be more quickly developed, or that the development process is now becoming more commoditized, how should these different strategies be constructed within a portfolio, and how can you be confident that they are actually different from one another?

RCM Alternatives Jeff Malec chimes in on his being unimpressed with the early examples of AI in the hedge fund space, what would make it more interesting, and what happens when everyone is using AI.

There was a big article a while back on Man AHL using AI, and the two examples of the AI’s great trades were buying the stock market dips in 2015 and after the Trump election. That’s doesn’t make me say,  “Oh my God, that computer is so smart!”, nearly everyone was already on that trade… So it’s rather like “Well, we bought the dip because that had worked the past 20 times.” So to me, that’s not much different than a novice trader going with what has worked recently, and it’s a little worrying, or perhaps the term is unimpressed.

The more interesting thing to me would be a manager who has all the datasets coming in and then works sort of like an AI based macro manager. Instead of focusing just on one market or sector, the AI could be saying, “Hey, bitcoin is up, I’m moving into bitcoin. Toronto real estate’s up. I’m moving into Toronto real estate, there’s no volatility in currencies, I’m avoiding those for now”.  But from what I see this is not done, it is still “I’m trying to figure out this dataset with AI,” instead of “I’m having AI analyze all the datasets to find out where I want to be.” The interesting thing to me would be an unconstrained AI where you just give it the instruction – find the best risk adjust return. Full stop. It has to figure out where and how much to invest.

Finally, I’ll put on my Philosophy major hat here and ask the philosophical question of what happens when everyone has the machines? Can one machine be smarter than the other with the same hardware and physical restrictions? In a future world where the machines create their own hardware and software to improve their intelligence – will that necessarily mean they will all come to the same conclusion theoretically? If they do, will, at some point, the human inputs matter again when all the machines are equal?

Along the same lines, Scot Billington of Covenant Capital says he is a skeptic, and that AI adoption will be more readily accepted by the new generation of millennials:

When I think about AI, first off, I would say there’s no reason that a carbon electro chemical system would be better than silicon. And frankly, we know that already know that computers do almost everything way better than we do. Driverless cars are going to reduce accidents by a factor of hundred thousand and planes are flown almost entirely by computers now…  In trading, the AI approach assumes that there are inefficiencies that a machine could learn, certain anomalies and things that can be predicted from whatever inputs there are. Whether we are taking price or fundamental inputs, there has to be something that a machine could evaluate and say, “Oh look, X and Y mean a higher probability of Z.” That of itself is pretty questionable. So I think one of the most interesting questions is: are there really things that can be learned?

I am exaggerating here, but my kids never bought anything meaningful NOT on a computer. Both my sons trade Bitcoin and they think they’re George Soros right now. They couldn’t be more thrilled with themselves. There is an ease with which younger people deal with machines because as digital natives they have been much more intimate with technology in their lives than our generation

While John Schabillion, Vice President of 50 South Capital questions if humans are willing to give AI a second chance if it doesn’t get it right the first time:

Investors in systematic strategies should understand that even if a machine predicts a probability of a trade one or two percentage points better than a fundamental manager, this does not necessarily mean that the trade will be profitable…. over short time periods in markets luck or randomness often dominates skill regardless of whether you are executing a strategy on a discretionary or systematic basis. I think the biggest thing for allocators getting into the systematic space is making sure that they really understand what the machine is capable of and what it isn’t capable of, so that expectations are set appropriately

One of the interesting things researchers find when they study the relationship between humans and machines is that when machines get something wrong humans are very quick to discredit or discard that machine or system. However, if a person gets something wrong people are much more likely to give them a second chance. I think this difference really has to do with the fact that you can sit across from a person, see the contrition in their eyes, and receive an explanation of what happened and why. But with a machine, especially in the sense of unbounded AI, it may not be immediately intuitive why the machine was positioned a certain way. Additionally, markets can and often do experience exceptional events, which have never been seen previously, and therefore can result in large scale losses for both discretionary and systematic strategies.

Lots to unpack given the recent quant revolution. To read the full conversation about AI, and other topics including VIX strategies, Bitcoin, and emerging managers, visit the Opalesque website, and download the PDF for free.

Leave a Comment