Who this is for — Anyone who has defined a strategy version and wants to observe it on data arriving after the freeze, without rewriting rules, sample or criteria in light of the results.
A forward test is the prospective evaluation of a frozen version: the specification is identified at a precise moment and, from then on, produces decisions following the real flow of time. Future data cannot be rewound, selected or replaced after their outcome has been seen. The test measures the version's behaviour during the observed period; it does not prove that the edge will continue and does not guarantee the same execution at a larger scale.
Out of sample and forward test answer different questions. OOS describes whether data took part in development: even a historical segment that already exists can be OOS if it truly remained sealed. A forward test instead describes chronological order: the data arrive after the freeze. Paper, shadow and micro-live are implementation environments in which a forward test may be conducted; they are not synonyms for forward or OOS.
Relationship among data, time and environment
| Term | What it classifies | Correct question | Example |
|---|---|---|---|
| Out of sample | Data-development relationship | Did the data influence a choice? | Historical holdout never consulted |
| Forward test | Temporal relationship | Does the decision precede arrival of the data? | Version frozen as of today |
| Paper trading | Simulated environment | Does the order expose real capital? | Demo account with modelled fills |
| Shadow mode | Parallel implementation | Does the pipeline generate but not route orders? | Shadow orders and real-time logs |
| Micro-live | Reduced real environment | Which effects emerge with small real orders? | Routing and fills under tight limits |
The categories can overlap. An observation generated tomorrow by a frozen version and never used to modify it is both forward and OOS relative to that version. If its result guides a new filter, it becomes development information for the next version.
What must be frozen
Freezing “the rules” is not enough. The version includes at least:
- code, dependencies, configurations and relevant seeds;
- universe, data source, calendar, timestamps and missing-data rules;
- features, parameters, signals and handling of no-trade cases;
- portfolio construction, sizing, leverage and constraints;
- order types, fill model, costs, funding and benchmark;
- primary metrics, controls, alert thresholds and completion criterion;
- paper, shadow or live mode and their known differences.
A material change creates a new identifier. Correcting a typo in the documentation may not alter the version; changing features, universe, delay, costs, sizing or order behaviour normally does. The ledger must make the decision verifiable.
Prospective protocol
- Write the mandate. Distinguish statistical objective, pipeline check, execution quality and operational training.
- Apply the freeze. Preserve version, artefact hashes, date and time, responsibilities and predefined modification or stopping criteria.
- Choose the environment. State which events are simulated and which are real; document the fill model, broker, venue, feed and latency.
- Record the entire flow. Preserve received data, null signals, intentions, orders, rejections, fills, positions, costs, alerts and human interventions.
- Do not clean up retrospectively. Errors and outages remain in the record; any exclusions follow rules defined beforehand and are shown.
- Compare with stated expectations. Evaluate distribution, risk, exposure, turnover, execution and benchmark with their uncertainty.
- Classify deviations. Separate data or operational incident, implementation error, execution drift, normal variability and possible model deterioration.
- Close with a traceable decision. Continuing, limiting, suspending, revalidating or creating a new version are outcomes distinct from “profit” and “loss”.
Duration and amount of information
There is no universal duration. The necessary calendar time depends on the strategy's frequency, dependence among signals, position horizon, rarity of events and required precision. Many correlated trades are not equivalent to the same number of independent observations; a few profitable weeks may fail to include material conditions.
The protocol should define criteria based on coverage and the decision: order types observed, relevant sessions and events, volume of information, precision of metrics and absence of unresolved incidents. An end date may be necessary for governance, but that does not make it statistical proof of sufficiency.
Nor can a single profit-factor, Sharpe or drawdown threshold decide the test. Metrics must be read together, net of costs, against a benchmark selected in advance and with intervals consistent with dependence and distribution. An outcome may be favourable, incompatible with the hypothesis or simply inconclusive.
Changes, incidents and purity
Operational defects requiring immediate intervention may emerge during the forward test. Safety takes precedence over statistical purity: a kill switch must not be delayed to “save the test”. The record preserves the incident, decision and data involved. If the correction changes behaviour, a new version begins with a new prospective period.
Repeatedly consulting results and adjusting the strategy turns the forward test into iterative development. Iteration is legitimate, but it must be called by its name; concatenating segments from different versions and presenting them as one uncontaminated track record is incorrect.
Illustrative example
Example explicitly stated as illustrative — Version 1.0 is frozen with signals, sizing, costs and operating thresholds. In shadow mode, it records the order it would have submitted for every event. After launch, data for a shortened session are found to arrive under an incorrect calendar: the incident is retained, the pipeline is corrected and version 1.1 begins. Results from 1.0 are not attributed to 1.1. Numbers of days or trades are not specified because they depend on the objective and information structure.
The next step might be a limited micro-live stage to observe routing and reconciliation. It is neither mandatory nor sufficient in every context and requires appropriate authorisations, risk limits and operational capacity.
Limitations
- A forward test observes only the regimes that occur after the freeze.
- Paper and shadow do not fully reproduce queue position, impact, borrow and economic pressure; micro-live does not demonstrate capacity at target size.
- A frozen version may share data and design biases with the backtest.
- Monitoring many metrics or strategies increases the risk of ex-post selection.
- Prospectivity improves traceability; it does not turn correlation into causality.
Sources
- Leonard J. Tashman, Out-of-sample tests of forecasting accuracy: an analysis and review, 2000 — temporal order, rolling origin and successive out-of-sample evaluations.
- Joseph Simonian, CFA Institute Research Foundation, Investment Model Validation: A Guide for Practitioners, 2024 — proactive validation and reliability of investment models.
- National Futures Association, Interpretive Notice 9025 — Use of Promotional Material Containing Hypothetical Performance Results — limitations of hypothetical performance within its NFA scope.
- U.S. Securities and Exchange Commission, Staff Report on Algorithmic Trading in U.S. Capital Markets, 2020 — automation, venues and order interaction in real markets.
- European Commission, Commission Delegated Regulation (EU) 2017/589, RTS 6 — testing, controls, documentation and deployment within its applicable EU regulatory scope.