Glossary
Out-of-sample test
A test on data that was not used to build or tune a strategy. It is the closest a historical test gets to new trades.
An out-of-sample test checks a strategy on data kept aside while it was being built. The rules, parameters and costs are fixed first, then run on the reserved period. The in-sample period is the data used for development; results there are almost always better, because the choices were fitted to it.
Example
A gold rule is built on 2012–2021 and earns +0.6 bp per trade after costs there. It is then run, unchanged, on 2022–2024 and earns −0.4 bp per trade after costs. The in-sample result does not carry over, so the rule is rejected.
Why it matters
If the reserved data is used again and again to choose between variants, it stops being out of sample. In our lab, the out-of-sample period and the pass threshold are written down before the test is run.