Public AI Trading Contests Largely Lose Money and Raise New Questions

What happened in the public contests

A run of live experiments has been stress testing large language models as stock pickers, and the early scorecard is rough. Alpha Arena, a tournament run by startup Nof1, pitted eight top-tier frontier models - among them Anthropic's Claude, Google's Gemini, OpenAI's ChatGPT, and Elon Musk's Grok - across four distinct two-week contests focused on US tech stocks.

Each model received $10,000 per challenge to handle tasks like trading off signals, playing defense, responding to competitors, and using high leverage. By the end, the combined portfolio was down by about a third. Across the 32 individual results, only six ended in the black.

In the contest where models could monitor rivals' performance, Grok 4.20 came out on top. It made 158 trades under a single prompt, while Alibaba's Qwen fired 1,418 trades using that same instruction.

These arenas are not lab-grade studies, but they are some of the most visible public trials of whether LLMs can take on the toughest job in finance: beating the market.

What the tests are teaching us

Money managers have been cautious about handing the trading wheel fully to AI, even as LLMs spread elsewhere. From JPMorgan Chase & Co. to Balyasny Asset Management, firms already put these systems to work in quant groups to filter news, draft internal communications for hedge funds, and flag fraudulent activity within major banks. Trading, however, usually keeps a human in the loop.

"LLMs can't really make money by themselves," said Jay Azhang, founder of Nof1. "You need basically a very sophisticated harness and scaffolding and data platform in order to even give them a chance." He said the models do well at research and at choosing and using tools, but they do not yet grasp how much the many stock drivers matter in practice, from analyst ratings and insider activity to shifts in sentiment. In the arena, they tended to mistime entries and exits, size positions poorly, and flip positions too often.

Flat Circle, an AI-focused blog, reviewed 11 market arenas and observed that every one featured at least a single model posting gains. Yet only two of the 11 showed a median model that was profitable, a pattern that rhymes with human fund performance. Bias shows up too. Azhang said that in a recent Alpha Arena run, Claude mostly leaned long, Gemini was comfortable going short, and Qwen was willing to use heavy leverage.

"They have personalities that you have to manage almost like a human analyst," said Doug Clinton, who leads Intelligent Alpha, which operates an LLM-driven fund and publishes a benchmark on AI-based corporate earnings projections. He said reminding a model about its bias can improve results.

Many investors find that new tools can sharpen discipline and uncover fresh opportunities. We're hosting our first ever investor livestream on 11/17 to show you how to use ChatGPT and Claude to invest like a pro, hosted by Briefs Finance CEO Jaspreet Singh. Save your seat.

For its benchmark, Intelligent Alpha supplies 10 AI systems with financial filings, analyst forecasts, earnings call transcripts, macroeconomic data, and as many as 10 web searches. With that tighter brief, the scores look better. He added that the models tend to get stronger with each new release.

Why the methods matter

Measuring AI trading is tricky. Small choices about how often an agent runs or which assets it touches can materially change outcomes. Classic backtests also stumble with LLMs because of lookahead contamination.

If you ask in 2026 how it might have positioned trades for March 2020, the model inevitably knows what actually happened in March 2020. That is why more testing is shifting to live markets, spawning new benchmarks and arenas.

Short runs are the norm, partly because these events are easy to spin up. Many are launched by individuals or startups and double as launchpads for other products. Nof1 is prepping a second season of Alpha Arena that will let agents search the web, think longer, tap more data sources and take multi step actions. The company's main business aims to let retail traders create AI trading agents tailored to their own strategies. As Azhang put it: "Giving an LLM money right now and just having it go - that's not a thing yet."

Most public trials remain too short and too noisy to resolve the question, said Jim Moran, author of the Flat Circle blog and a previous co-founder of alternative data firm YipitData. These setups also face handicaps like limited access to proprietary research and weaker execution. "If you took one of these agents from one of these arenas and you just moved it over to operate inside of a high-end hedge fund, they should perform better," he said.

Alexander Izydorczyk - once the head of data science at Coatue Management and currently with NX1 Capital - recently wrote that, across the AI trading bots he follows, not a single one has shown a durable advantage. His view is that these arenas are hamstrung by gaps in how models are trained, especially the absence of the hands-on quantitative techniques used inside closed-door trading shops. He added that if LLM-based strategies do start to work, they will likely go quiet. "But beginners sometimes see things incumbents cannot," he wrote.

The bottom line for investors

In plain English: the public trials echo a familiar lesson. Without structure and guardrails, LLMs overtrade, disagree with themselves, and often lose money. With a narrow task and curated data, they can spot signals, including earnings direction, that look promising.

For portfolios, that means AI is already useful as a research assistant and tool user, but it still needs a framework and oversight before it can be trusted to trade real capital. Season two of Alpha Arena and more focused benchmarks will test whether more compute, richer data, and better scaffolding can close the gap.

Learning how to use emerging technologies thoughtfully can give investors a lasting edge. On 11/17, our CEO Jaspreet Singh goes live for our first ever investor livestream on how to use ChatGPT and Claude to invest like a pro. Register to join him.