What a backtest is, and when you can actually trust one

The good news: it takes an afternoon to learn, and it works on anybody's numbers, ours included.

What a backtest actually is

A backtest is a simulation. You take a set of rules, run them over historical price data as if they had been live, and record what would have happened. Nothing was bought or sold, no money was at risk. It is the standard way to test whether a trading bot's rules had an edge.

That's its strength and its weakness at once. The strength: thousands of trades across conditions nobody could live through in a reasonable time. The weakness: every input is controlled by whoever ran it, and the temptation to keep adjusting until the curve looks beautiful is enormous.

Look-ahead bias: the lie that's hardest to see

Look-ahead bias means the simulation used information that wasn't available at the moment of the trade. It is the most common way a backtest turns beautiful, and it rarely happens on purpose.

• Deciding on a bar you haven't finished. "Buy when the 5-minute candle closes above the open" is fine, as long as the entry is at the next bar, not inside the one used to make the decision.
• Revised data. Economic figures get restated months later. A system reacting to a number must react to the number as first published.
• Knowing the roll in advance. Futures contracts expire, and stitching a continuous price series with hindsight about where volume moved is not what you knew that morning.
• Filters chosen after the fact. "The system skips Fridays" is legitimate if it was decided before the test, and curve fitting in disguise if it appeared after someone noticed Fridays were bad.

Look-ahead is why a strategy looks flawless in testing and falls apart in its first live week. Nothing broke. The simulation was playing with cards it wouldn't have held.

Tick data versus minute data

Picture a one-minute bar whose range covers both your stop and your target. Which was hit first? Minute data cannot tell you. The software has to assume, and every assumption is a policy: optimistic ones credit the target, conservative ones charge the stop. That single assumption can swing the result by a wide margin.

Tick data doesn't assume. It has every trade in sequence, so the simulation knows which price came first. It is heavier, slower and often costs money. It is also the only way a stop-and-target backtest means anything.

If a seller can't tell you what data resolution their backtest ran at, you've learned everything you needed to know.

Commissions, slippage and the zero-cost fantasy

Set commissions to zero and any marginal system becomes profitable. It is the cheapest trick in the industry because it requires no dishonesty: just leaving a field at its default.

Slippage is the other half. Market orders don't always fill at the price on your screen, and stops fill worse in fast conditions. A backtest with perfect fills describes a market that does not exist.

Cherry-picked periods, survivorship and sample size

Three ways of showing a flattering slice of reality.

Survivorship bias. Mostly a stock problem: test on today's index members and you've excluded everything that went bankrupt or got delisted. Index futures dodge it, since the contract is the contract. A related version still applies to systems: you are shown the strategy that survived the developer's own selection. The twelve that failed aren't on the page.

Sample size. Two hundred trades is a story. A thousand is a statistic. With a target at three times the risk, break-even sits at a 25% win rate, so the system does not need to be right most of the time; and the gap between a real edge and luck only becomes visible over many trades.

Curve fitting, the sophisticated version

Curve fitting is tuning parameters until the system fits history so tightly it has memorized it rather than learned anything. The telltales: many parameters, oddly specific values, filters that exclude particular dates, and results that collapse when you nudge an input by 10%.

The defense is a stability test: change a parameter slightly and watch whether performance degrades gently or falls off a cliff. A genuine edge is a plateau; a fitted one is a needle.

What a backtest still can't tell you

Even a clean one has hard limits, and the honest thing is to name them.

• It can't know the future. It describes a period that already happened, and if market structure changes the rules keep applying regardless. That is why judging when a system is broken stays a human job, as covered in automated vs manual trading.
• It can't simulate you. The backtest never switched itself off after four red days. You might.

Frequently asked questions

Is a backtest the same as a track record?

No, and treating them as equivalent is the most expensive mistake in this field. A track record is money that moved in a real account, with real fills and real slippage. A backtest simulates what the rules would have done, prepared with hindsight. Good evidence; not proof of anything that happened.

How long should a backtest be?

Long enough to include several distinct market environments, which for an intraday futures system means years, not months. Trade count matters more than calendar time: a thousand trades across varied conditions tells you more than two calm years. Under a few hundred trades is an anecdote.

What profit factor should I look for?

On an intraday futures system, roughly 1.3 to 1.8 is plausible and defensible. Much above 2 over a long period on a liquid instrument warrants hard questions about data resolution, commissions and parameter count. A high profit factor is not automatically good news; it is a reason to look closer.

Why does my result differ from the published one?

Usually one of four things: a different date range, a different fill resolution, commissions left at zero, or a different data feed. Work through them in that order and the gap almost always closes. If it doesn't, open a ticket through the contact form with your settings.

Educational content, not investment advice. Trading futures involves substantial risk of loss and is not suitable for every investor. Any reference to Algomyth results is hypothetical, simulated over historical data, and shown in full on the performance page; past performance does not guarantee future results.