Hedge funds and high-frequency traders are converging
Prop shops versus pod shops
This summer, many systematic, algorithm-powered trading strategies suffered an abrupt and mysterious pummeling that was somewhat reminiscent of the infamous “quant quake” of 2007.
It wasn’t nearly as violent as in 2007 — it was more an unpleasant quiver than an earthquake — but it was enough to fray nerves in some corners of the quantitative hedge fund industry.
The reversal might have been started by a “garbage rally” in heavily shorted stocks, but some think that it might have been exacerbated by one of the biggest new trends in quant investing: the growing overlap between market-making trading firms such as Citadel Securities, Hudson River Trading or Jane Street, and big hedge funds such as DE Shaw, Millennium, Point 72 or Qube Research & Technologies.
Some in the industry are sceptical that this increasing overlap was a factor in the July “quant quiver”, pointing out that the strategies that were the worst hit were mostly longer-term ones, rather the those using quicker signals, where competition is becoming more ferocious. Nonetheless, both proprietary trading firms and hedge funds concede that two industries — that for years virtually operated in separate worlds — are now starting to come together.
As a senior executive at one of the big multi-strategy hedge funds told FT Alphaville:
There are times when an industry’s structure changes. We’re now in the early stages of seeing a reorganisation of systematic trading, where some successful prop trading firms are going to increasingly resemble hedge funds, and some successful hedge funds will start to look like prop trading firms . . . There is an interplay and growing overlap in their skillsets and strategies. It will be interesting to see how it plays out, but they are definitely beginning to converge.
This trend has been quietly emerging since 2020-21, but has become much more apparent in the past year or so. The confluence also has myriad implications for both industries — and the markets where they’re increasingly colliding.
This may test your patience, but to understand how it happened and why it’s so interesting it’s probably worth first diving briefly into the parallel histories of high-frequency trading and the quantitative hedge fund strategy known as “statistical arbitrage”. Feel free to skip the next two sections if you know all this stuff already.
Please use the sharing tools found via the share button at the top or side of articles. Copying articles to share with others is a breach of FT.com T&Cs and Copyright Policy. Email
licensing@ft.com to buy additional rights. Subscribers may share up to 10 or 20 articles per month using the gift article service. More information can be found at
https://www.ft.com/tour.
High-frequency trading has evolved dramatically in the decades since its genesis, whether you think this was the NYSE’s first electronic “designated order turnaround” system in the 1970s, the “bandits” that preyed on the Nasdaq’s Small Order Execution System in the 1980s, or the explosion of automated trading on “electronic communication networks” in the 1990s.
The cottage industry’s first big inflection point came in 2005 with the SEC’s introduction of Regulation National Market System — or RegNMS as it’s usually called. By modernising the US equity market structure and encouraging greater competition, this became “the final structural move that set the stage for the current electronic trading revolution”, as one academic noted in a 2010 study.
Just how far the revolution had come first became apparent to the general public in the 2010 “Flash Crash”, when the US stock market suddenly careened lower at speeds humans struggled to comprehend. The normie view of high-frequency traders as financial parasites that ruin markets was then crystallised by Michael Lewis’ bestselling 2014 book Flash Boys.
Many in the industry — who saw themselves as geeky disrupters that stuck it to Wall Street and made trading cheaper for investors — were horrified at their portrayal. In fact, when Ari Rubinstein, the head of Global Trading Systems, first heard that one of his favourite authors was writing a book about his industry he assumed that they’d naturally be the heroes of the tale. As he told the FT a few years ago:
I thought, finally, someone is going to glorify what we’ve been able to do. A bunch of people were able to disrupt the industry, create a lot of efficiency, save people a lot of money and get rid of the middlemen in the process — and I was like, ‘Holy cow! is he going to call us?’ And then, when I found out that, ‘Oh no, you’re the villain’, I was really surprised.
Politico really nailed the zeitgeist with this illustrative gif back in 2016.
However, the classic view of HFTs as a monolithic group of purely algorithmic, hyperactive speed merchants was never entirely correct, and is now a little outdated.
Pure speed is still essential to swaths of bread-and-butter market-making. What was once measured in milliseconds (thousandths of a second) became nanoseconds (billionths of a second) in the noughties, and is today often done in picoseconds (one trillionth of a second). In this space, microwave towers and “co-location” are still important.
But “low latency trading” — as people in the industry usually call this form of HFT — is butting up against the limits of physics. Moreover, intense competition has made it much less profitable. As one HFT executive says: “There is no alpha, it’s all latency. That gets all the focus, but it doesn’t actually make much money.”
As a result, there’s been a massive amount of consolidation in recent years, with many early HFT pioneers falling by the wayside and others simply stagnating.
The new HFT royalty are therefore primarily (if not exclusively) firms that have evolved into “proprietary” trading firms that also make bets with their own capital — as opposed to only pocketing the spreads between two-sided quotes on securities — and which have broken out from the pure speed game by holding positions for minutes, hours or even days.
The best examples are probably firms such as Jane Street, Citadel Securities, DRW, Susquehanna International Group and Hudson River Trading. Sure, most of these firms still do a lot of classic high-speed, high-frequency, high-volume trading, but increasingly the really big profits are coming from prop trading and slower signals.
And this is starting to bring them into territory historically ploughed by hedge funds that pursue a strategy called “statistical arbitrage”.
📈📉📈📉📈📉📈📉📈📉📈📉
Sometime in the early 1980s a programmer named Gerry Bamberger pioneered something called “pairs trading” at Morgan Stanley. It quickly proved a phenomenon.
Bamberger must have cut an odd figure at Morgan Stanley. This was Wall Street’s Waspiest firm, and he was a tall, cerebral Orthodox Jew with a heavy smoking habit who ate a packed tuna salad sandwich for lunch every single day. But the strategy he developed became a money machine for the bank, which called his new desk Advanced Proprietary Trading.
Pairs trading involved finding pairs of securities that were usually closely correlated — like Pepsi and Coke, Royal Dutch and Shell, or Berkshire Hathaway’s different share classes — but occasionally veer off in opposite directions. You then short one and long the other, betting that the historical link would reassert itself.
Over time this evolved into the broader strategy dubbed statistical arbitrage, where you constantly scour markets for thousands of opportunities like this, hedge out the overall stock market risk and try to just generate pure, sweet market-beating alpha.
These stat-arb bets can range from simple pairs trading to the more complex, such as arbitraging divergences in the price of US equity market exposure through individual stocks, marketwide ETFs, index options and futures. It quickly became a big deal, as former Morgan Stanley risk management supremo Richard Bookstaber later wrote in his book A Demon of Our Own Design:
Thanks to Gerry Bamberger, who started as a programmer on Morgan’s equity desk, the way trading was done and the function it performed had changed. As a result of his work, the computational power for statistical analysis was unleashed on the markets and — using the newfound execution capabilities of the equity market — a machine was created to harvest opportunities to provide liquidity. Bamberger had moved at least one segment of the market from that of hunter-gatherer to farming.
Bamberger later fell out with Morgan Stanley and left for Ed Thorp’s pioneering quant hedge fund Princeton/Newport Partners. Morgan Stanley’s APT desk was taken over by Nunzio Tartaglia, a famously sweary Jesuit-educated former physicist, who for a period took it to new heights. In 1987 APT reportedly made $50mn of profits for Morgan Stanley, a fortune at the time and particularly remarkable given the Black Monday crash that year.
However, by the end of the decade returns started to fizzle, and many of the top quants on Morgan Stanley’s APT desk headed to the exit. Among them was a brilliant technologist called David Shaw. He started his own hedge fund built on statistical arbitrage — which is today’s $70bn DE Shaw.
DE Shaw in turn birthed Two Sigma — another giant of the quant hedge fund industry — while Morgan Stanley’s APT desk was eventually resurrected in the form of Peter Muller’s Process Driven Trading Group. In 2012 this was spun off of the bank as the hedge fund PDT Partners. Between them, DE Shaw, Two Sigma and PDT reportedly manage roughly $150bn, much of it in stat-arb strategies. Renaissance Technologies fabled Medallion fund is also said to mostly consist of statistical arbitrage.
Just like with HFTs, the stat-arb world is not monolithic. Strategies and holding periods can vary enormously. Some hedge funds might use signals that only hold positions for a few hours, but it’s usually days, and they can be weeks.
HFTs also look for often pricing discrepancies that they can arbitrage, but over very different timeframes, and for a long time they overwhelmingly went home “flat” — in other words, most positions were closed out at the end of every trading day. After all, while hedge funds manage other people’s money, prop firms usually only have their own partners’ capital to play with, so they didn’t want to lug around lots of risky positions on their balance sheet for too long.
As a result, they evolved as essentially two different industries, with remarkably little overlap despite being mostly staffed by the same types of programmers, mathematicians and scientists and pursued vaguely similar systematic financial trading strategies.
Until now, at least.
📈📉📈📉📈📉📈📉📈📉📈📉
Industry insiders say the overlap first began in earnest in the wake of the Covid-19 trading boom, when several trading firms made so much money that they had to find new places to deploy it.
After all, in classic “low-latency” strategies there’s a limit to how much capital you can deploy at any given time, given how swiftly securities are traded. What works with $1bn can actually work less well with $10bn. However, longer term trades — and by longer term, we’re obviously talking in relative terms — allow firms to deploy more resources, in terms of capital, people and technology. As the head of one large US trading firm tells Alphaville:
It’s a capacity issue. To keep growing revenues you need to take larger positions. There’s been a lot of investment in research and compute, and those are very high fixed cost investments, so you want as much investment capacity as possible to earn that out.
At the same time, many trading firms saw with envy how much money Jane Street in particular was starting to make, mostly because of its ability and willingness to carry positions for a bit longer than the norm — the result of its role as a major market-maker for ETFs. In the first half of 2020 alone Jane Street notched up $8.4bn of net trading revenues, more than twice that of its rival Citadel Securities.
Soon enough, more and more trading firms started adding “mid-range” trading strategies to their arsenal, which are now said to be particularly strong profit centres at the likes of Hudson River Trading. At HRT these signals are housed a separate unit called Prism, which reportedly notched up profits of more than $2bn last year.
Citadel Securities has largely remained a classic market-maker — given that Ken Griffin’s separate hedge fund Citadel already does plenty of prop trading — but it too is said to be holding positions for longer these days. A spokesperson for the company said: “Depending on size, product, risk and liquidity dynamics, we warehouse this risk over a range of time durations, sometimes up to weeks.”
Naturally, this has meant that many hedge funds have been eyeing with equal jealousy the huge profits these trading firms have made in recent years.
After all, many of the hedge fund industry’s top names are closed to new investment because their existing strategies also have capacity constraints. As a result, they are constantly on the prowl for new ones that might allow them to keep more investor money rather than sending billions of dollars' worth of gains out the door every year.
Moreover, faster, systematic strategies typically boast high “Sharpe ratios” — a measure of investment returns relative to their volatility — which can make a fund’s overall results look prettier, as one senior hedge fund executive notes:
Hedge funds need high Sharpe capacity strategies, because there is a barbell sort of complementarity between high Sharpe strategies with low capacity and high capacity strategies with low Sharpe ratios. So hedge funds want more high Sharpe strategies — and those are typically lower latency strategies — in order to support strategies like commodities or some fixed income.
As a result, hedge funds with stat-arb strategies and prop trading firms are increasingly competing in trading strategies with holding periods ranging from a few hours to a few days.
Here’s a chart from Goldman’s last annual survey of the quant hedge fund industry, which shows how the estimated market footprint of prop traders has expanded dramatically in recent years (zoomable version):
Please use the sharing tools found via the share button at the top or side of articles. Copying articles to share with others is a breach of FT.com T&Cs and Copyright Policy. Email
licensing@ft.com to buy additional rights. Subscribers may share up to 10 or 20 articles per month using the gift article service. More information can be found at
https://www.ft.com/tour.
Some industry insiders argue the convergence is mostly a case of prop trading firms rolling their tanks on to the stat-arb lawn, rather than stat-arb hedge funds also speeding up and encroaching on prop trading turf.
But the head of one major quantitative hedge fund told Alphaville that he was definitely seeing a move towards lower-latency trading signals by his industry.
At the very short end [of latency], opportunities have compressed; at the very long end [of investment strategies], premia are crowded. Naturally, capital and talent migrate towards the middle. So you see prop firms with execution DNA stretching into multi-day signals, while classical stat-arb firms accelerate their cycles.
For now, prop trading firms are said to have been more successful in adding “slower” trading strategies than hedge funds have been at speeding up. As one quant hedge fund manager we spoke to observed: “It’s easier to go from building a Ferrari to building a Volkswagen, than from building a Volkswagen to a Ferrari.”
However, some prop trading firm executives say this understates the apparent success of some hedge funds such as Qube Research & Technologies and long-established quant powerhouses like DE Shaw, and reckon the trend will become even more pronounced in the coming years. As one put it to us:
The Venn diagram never overlapped before, and now it does. It started overlapping in a tiny way maybe five years ago, but in five years’ time the overlap will probably be even bigger than it is today.
📈📉📈📉📈📉📈📉📈📉📈📉
So what does this all mean? Why do we even care? We shouldn’t, on a cosmic scale. But here are a handful of implications that Alphaville thinks are worth bearing in mind, in roughly ascending order of importance.
1️⃣ Hedge funds and prop trading companies are going to be increasingly competing not just for entry-level talent — freshly-graduated computer scientists, mathematicians and general brain boxes — but also for mid-career people.
Millennium’s fateful poaching of two Jane Street traders is therefore probably just the beginning. We’re going to see more of this happening — given that some hedge funds can pony up $100mn packages for portfolio managers — but we’re also likely going to see quant hedge fund people migrate to the prop trading world. After all, even the interns make stupid money.
2️⃣ Prop trading firms are becoming increasingly important clients of Wall Street, and this is just going to become even more pronounced. On the current trajectory they soon might rival hedge funds and private equity for importance.
Historically, prop trading firms operated fairly separately from the banks. They might route trades to or through them, but didn’t rely on them in the symbiotic way that hedge funds do. As Jarrod Yuster, the chair and CEO of trading tech provider Pico, says:
It’s a very technology-intensive business, and generally they don’t hold positions overnight. You don’t need financing for that, so the business they offered banks was just execution and trading fees. Banks therefore valued quant funds more than HFTs.
However, as prop trading firms have begun to spread their wings they need more financing and other services. As a result, they have grown radically in importance to the prime brokerage units of big banks that have usually catered only to hedge funds — even though the trading firms are in many respects rivals to other parts of the same bank.
As Goldman Sachs’ markets supremo Ashok Varadhan told IFR earlier this year:
As you grow and become more relevant, there are going to be times when your clients will be your competitors and you just have to manage through that and have the maturity to realise that you’re going to collaborate in some areas and compete in others.
3️⃣ Prop trading firms are going to be raising more external capital, and hedge funds will become increasingly tempted to hive off their best trading strategies in internal funds for their own partners and employees. In fact, both of these things are already happening — at least in some form.
Citadel Securities, Hudson River Trading and Jane Street have all tapped the debt markets to boost the firepower offered by their retained profits. Given how much money these firms have made in recent years they may never want or need to start more traditional fund-like investment vehicles, but Tower Research — one of the HFT pioneers — has talked to investors about doing so, and several industry insiders predict that “proper” HFT-powered long-short equity funds will inevitably emerge.
At the same time, some of the higher-profile star-arb hedge funds are either already entirely employee money (Renaissance’s Medallion) or probably heading in that direction (DE Shaw’s Valence fund). It’s natural that more and more successful hedge funds start housing their low-capacity, high-Sharpe quasi prop-trading strategies in internal funds — even if it annoys investors no end and needs to be done very carefully.
4️⃣ Increasing competition in mid-frequency trading — generally said to be in the 1-5 day range — could cause crowding in some signals. The dangers of this crowding are ramped up by the growing use of leverage to maximise profits.
There are good reasons to be sanguine about this. A more diverse set of participants is generally a healthy thing for a market. Prop trading firms overwhelmingly deploy their own money — the stickiest capital there is. Both prop firms and quant hedge funds are as a rule pretty obsessive about risk, and particularly assiduous about monitoring for signs of herd-like behaviour.
However, as the head of one big trading firm told Alphaville:
It is a concern how much crowding there is in stat arb signals right now. July was a sign that things could be very crowded . . . It’s unclear how much of it is due to this, but there’s a lot of money chasing similar strategies and signals in similar instruments, which can cause correlated drawdowns.
Where on the Richter scale the next quant quake will measure is beyond Alphaville. But we can see that having nearly two-thirds of US equity trading volume — or ca 20x the total trading volume of the entire long-only investment fund industry — potentially getting locked into correlated drawdowns might not be ideal.
