This rule appears to hold for artificial intelligence (ai) and machine learning, which were first employed by hedge funds decades ago, well before the recent hype. First came the “quants”, or quantitative investors, who use data and algorithms to pick stocks and place short-term bets on which assets will rise and fall. Two Sigma, a quant fund in New York, has been experimenting with these techniques since its founding in 2001. Man Group, a British outfit with a big quant arm, launched its first machine-learning fund in 2014. AQR Capital Management, from Greenwich, Connecticut, began using ai at around the same time. Then came the rest of the industry. The hedge funds’ experience demonstrates ai’s ability to revolutionise business—but also shows that it takes time to do so, and that progress can be interrupted.

Ai and machine-learning funds seemed like the final step in the march of the robots. Cheap index funds, with stocks picked by algorithms, had already swelled in size, with assets under management eclipsing those of traditional active funds in 2019. Exchange-traded funds offered cheap exposure to basic strategies, such as picking growth stocks, with little need for human involvement. The flagship fund of Renaissance Technologies, the first ever quant outfit, established in 1982, earned average annual returns of 66% for decades. In the 2000s fast cables gave rise to high-frequency market makers, including Citadel Securities and Virtu, which were able to trade shares by the nanosecond. Newer quant outfits, like AQR and Two Sigma, beat humans’ returns and gobbled up assets.

Machine learning held the promise of still greater fruits. The way one investor described it was that quantitative investing started with a hypothesis: that of momentum, or the idea that stocks which have risen faster than the rest of the index would continue to do so. This hypothesis allows individual stocks to be tested against historical data to assess if their value will continue to rise. By contrast, with machine learning, investors could “start with the data and look for a hypothesis”. In other words, the algorithms could decide both what to pick and why to pick it.

Yet automation’s great march forward has not continued unabated—humans have fought back. Towards the end of 2019 all the major retail brokers, including Charles Schwab, E-trade and TD, Ameritrade, slashed commissions to zero in the face of competition from a new entrant, Robinhood. A few months later, spurred by pandemic boredom and stimulus cheques, retail trading began to spike. It reached a peak in the frenzied early months of 2021 when day traders, co-ordinating on social media, piled into unloved stocks, causing their prices to spiral higher. At the same time, many quantitative strategies seemed to stall. Most quants underperformed the markets, as well as human hedge funds, in 2020 and early 2021. AQR closed a handful of funds after persistent outflows.

When markets reversed in 2022, many of these trends flipped. Retail’s share of trading fell back as losses piled up. The quants came back with a vengeance. AQR’s longest-running fund returned a whopping 44%, even as markets shed 20%.

This zigzag, and robots’ growing role, holds lessons for other industries. The first is that humans can react in unexpected ways to new technology. The falling cost of trade execution seemed to empower investing machines—until costs went to zero, at which point it fuelled a retail renaissance. Even if retail’s share of trading is not at its peak, it remains elevated compared with before 2019. Retail trades now make up a third of trading volumes in stocks (excluding market makers). Their dominance of stock options, a type of derivative bet on shares, is even greater.

The second is that not all technologies make markets more efficient. One of the explanations for Aqr’s period of underperformance, argues Cliff Asness, the firm’s co-founder, is how extreme valuations became and how long a “bubble in everything” persisted. In part this might be the result of overexuberance among retail investors. “Getting information and getting it quickly does not mean processing it well,” reckons Mr Asness. “I tend to think things like social media make the market less, not more, efficient…People don’t hear counter-opinions, they hear their own, and in politics that can lead to some dangerous craziness and in markets that can lead to some really weird price action.”

The third is that robots take time to find their place. Machine-learning funds have been around for a while and appear to outperform human competitors, at least a little. But they have not amassed vast assets, in part because they are a hard sell. After all, few people understand the risks involved. Those who have devoted their careers to machine learning are acutely aware of this. In order to build confidence, “we have invested a lot more in explaining to clients why we think the machine-learning strategies are doing what they are doing,” reports Greg Bond of Man Numeric, Man Group’s quantitative arm.

There was a time when everyone thought the quants had figured it out. That is not the perception today. When it comes to the stock market, at least, automation has not been the winner-takes-all event that many fear elsewhere. It is more like a tug-of-war between humans and machines. And though the machines are winning, humans have not let go just yet.

For more expert analysis of the biggest stories in economics, finance and markets, sign up to Money Talks, our weekly subscriber-only newsletter.

© 2023, The Economist Newspaper Limited. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com