Learn · Track 5 · Reading the evidence · Lesson 2

A fair test: the same trade without the trick

To know whether a rule helps, compare it with the same trade without the rule, on the same days.

Most trading claims add a trick to a plain trade: a stop, a target, a time to get out, a day to skip. A fair test keeps everything else the same and changes only the trick. The plain trade is the baseline.

The claimput spread with a 2X stopThe baselinesame put spread, no stopsame days, same pricesHolds up, ties, or does worse
Only the rule changes. Everything else is held the same.

The same days matter. A rule measured in a calm month and a baseline measured in a wild month say more about the two months than about the rule.

Compare more than one number. A rule can raise the average P/L and also make the worst losses bigger. 0DTErs reads a balanced score first, then the average P/L, then the size of big losses, then the time at risk.

Then ask whether the gap is bigger than chance. Markets are noisy, and a small edge can be luck. When a gap sits inside the range that chance alone could produce, the honest answer is a tie.

Note. Baseline: the same trade, on the same days, without the rule being tested.

Check yourself (3 questions)
  1. What is the baseline for 'a put spread with a 2X stop'?

    The same put spread on the same days, with no stop.

  2. Why use the same days?

    So the result shows the rule, not a calmer or wilder stretch of the market.

  3. The gap is smaller than chance could explain. What is the verdict?

    A tie.