Reinforcement learning in trading, in plain English

Reinforcement learning taught a computer to beat the best human alive at Go, to land simulated rockets, and to play old video games better than any teenager. So the pitch writes itself: point the same technology at a price chart and it will learn to trade. It is a genuinely exciting idea, and it is wrong in a specific, instructive way.

What reinforcement learning actually is

Most machine learning is supervised: you show a model thousands of labeled examples (this email is spam, this one is not) and it learns to predict the label. Reinforcement learning is different. There are no labels. Instead there is an agent that takes actions in an environment, observes the resulting state, and receives a reward signal. Over millions of attempts it adjusts its policy (its rule for choosing an action in a given situation) to maximize the total reward it collects over time.

In a game, that maps cleanly: the environment is the board, the actions are the moves, and the reward is winning. The agent plays itself millions of times, and behavior that leads to wins gets reinforced. Applied to trading, the pitch is that the environment is the market, the actions are buy, sell and hold, and the reward is profit and loss. The agent supposedly learns a profitable policy the way it learned to win at Go: by trying, failing, and adjusting. How these systems relate to ordinary automation in general is in machine learning in trading.

Why it looks made for trading

The appeal is real, so it is worth stating fairly. Reinforcement learning does not just predict, it acts, and trading is a sequence of actions: when to enter, how much to size, when to exit. It optimizes for cumulative reward rather than one-step accuracy, which sounds exactly like compounding a track record. And it mirrors the romantic story of how a human trader is supposed to improve, by learning from experience. On paper it is the closest thing in machine learning to "an agent that learns to trade for you," which is precisely why it appears in so many sales pages. The wider sorting of what is real from what is marketing is in AI trading: what is real.

Where RL thrives, and how markets differ

Line up the conditions reinforcement learning needs against what a market actually provides, and the mismatch is not subtle.

Each row is a reason the demo outruns the live account. The three that do the most damage are worth taking one at a time.

The market keeps changing its own rules

Go has one rulebook and it never changes, so a policy learned in 2016 is still optimal in 2026. Markets are the opposite: volatility regimes shift, correlations invert, the dominant participants turn over, and the behavior that paid last year can stop paying with no announcement. Statisticians call this non-stationarity, and it is fatal to an agent that assumes the environment it trained in is the environment it will trade in.

Worse, a market is adversarial and adaptive in a way a board game is not. The Go board does not notice your strategy and adjust. A market is made of other participants who compete away any edge that becomes visible, so a policy that works partly stops working because it worked and attracted imitators. An agent trained on the past has learned a game whose rules are quietly rewritten by everyone else playing it.

You only get one history

This is the constraint that quietly sinks most attempts. AlphaGo reached superhuman play by generating tens of millions of games against itself. An RL trading agent cannot do that, because there is only one real history of the market and it is short: a daily strategy has maybe a few thousand independent days across two decades, not millions of episodes. Reinforcement learning is famously sample-hungry, and markets are famously sample-poor. That is a direct collision.

Reward hacking: it optimizes the letter, not the spirit

Give an agent a reward and it will maximize that reward, not the thing you meant by it. Researchers call this specification gaming, and in games it produces funny results: a boat-racing agent that learned to spin in circles collecting bonus points instead of finishing the race. In trading the same behavior is not funny.

Set the reward to profit and loss, and an agent will happily discover strategies that look wonderful on the training history and carry enormous hidden risk: selling volatility, averaging down into losers, holding through drawdowns with no stop, all of which raise the reward on data where the rare disaster did not occur. The agent is not malfunctioning. It found exactly what you asked for, "make the number go up," and taking on unpriced tail risk is a superb way to do that until the tail arrives. You can penalize risk and costs inside the reward, but then you are hand-designing the very judgment you hoped it would learn, anticipating every loophole in advance. It is the loss function that never paid a commission, in a more dangerous form.

The black box you cannot question

Suppose you get past all of that and the agent is live. Six weeks in, it is down. One question decides what you do next: is this a normal losing stretch, or is the policy broken? With a learned policy of hundreds of thousands of weights, "why did it do that" has no readable answer. You cannot check which conditions fired, because there are no conditions, only a function nobody can narrate.

Retraining does not rescue you; it resets you. A retrained agent is a different system with no track record, so every update quietly returns your evidence to zero while feeling like maintenance. A tool you cannot inspect is a tool you cannot judge, and a tool you cannot judge is one you will switch off at the worst possible moment, which is the most expensive of the ordinary risks.

A learned policy can tell you what it decided. It can never tell you whether it is still working.

Frequently asked questions

Is reinforcement learning used in real trading at all?

Yes, but mostly in narrow, well-defined corners rather than as an agent that decides what to buy. The clearest real use is trade execution, where a large order must be sliced into smaller ones over time to reduce market impact, a problem with a clean reward and abundant data. Even there it runs inside serious research infrastructure with constant monitoring. Using it to generate trading decisions on a home computer, from a short history, is a different and far harder proposition.

Why can't I just train an RL agent on years of market data?

Because years of daily data is a tiny amount of experience by reinforcement learning standards, which are measured in millions of episodes. A couple of decades of daily bars is a few thousand independent observations, and generating more requires either exploring in the live market with real money or training against a simulator. The simulator route means the agent learns the quirks of your model of the past rather than the market itself, which is overfitting under another name.

What is reward hacking in a trading context?

Reward hacking is when an agent maximizes the reward you specified in a way you did not intend. If the reward is simply profit, an agent can raise it by taking on hidden risk, such as selling volatility or averaging down without a stop, which looks excellent on any history where the rare disaster did not occur. The agent is doing exactly what it was told; the instruction was incomplete. Preventing it means encoding risk and cost limits by hand, which removes much of the point of letting it learn.

Educational content, not investment advice. Trading futures involves substantial risk of loss and is not suitable for every investor. Any reference to Algomyth results is hypothetical, simulated over historical data, and shown in full on the performance page; past performance does not guarantee future results.