Write the question before testing
A backtest applies rules to past data under a model. Before choosing periods, specify the instrument, conditions, execution, costs and evaluation measure. Another person should be able to reconstruct which cases qualify and which actions follow.
Synthetic research example: examine the three periods following two closes above a previously fixed reference. State how the reference is calculated and what data exist at decision time. This description studies a hypothesis without recommending trading the pattern.
Audit data and execution
Record source, period, zone, gaps and transformations. For futures, document contracts and your handling of expiry changes. Cleaning or joining series can change prices and results; preserve a version of the data and procedure.
Do not use an unavailable close to execute at the start of that same candle. If a bar’s high and low cross two thresholds, aggregate data may not reveal which came first. Define an assumption or use suitable data; do not always choose the favourable sequence.
Keep every variant
Changing parameters until you find the best-looking chart increases the risk of fitting the rule to that sample. Record discarded attempts, conditions and costs. Separate development and evaluation, but do not treat a single held-out dataset as a guarantee against overfitting.
Compare results under reasonable changes in costs and assumptions. A result disappearing after small changes needs an explanation. Research on overfitting offers more advanced methods; this introductory procedure does not replace them.
Produce a report that can be questioned
Show rules, data, modelled fills, gross and net results and limitations. Describe the sample’s size and composition without claiming it represents every possible market. Hypothetical historical results neither predict future returns nor reproduce all real conditions.
- Question and rules recorded before results.
- Data source and future-information controls.
- Record of variants, costs and assumptions.
- Reproducible evidence and bounded conclusions.
Sources and references
- Bailey et al. · The probability of backtest overfitting
- CFTC · Limits of hypothetical trading results
Chezur examples and explanations are educational. Consult the original source to verify conditions, data and updates.
Context and review of this explanation
How do I design a reproducible first backtest with explicit rules and costs?
- Topics
- Method and statistics · Units and costs
- Level
- Intermediate
- Skills
- Design a reproducible evaluation · Form a testable hypothesis
- Examples
- Hypothetical situations illustrating the concept.
- Specialist review
- Not documented for this edition. Automated structure and internal-link checks do not establish specialist review.