Time-series validation and walk-forward

Time-series validation evaluates a procedure on observations later than those used to estimate or select it. Walk-forward analysis repeats this principle at successive origins: at each step it uses only the past permitted by the protocol, produces a decision for the next segment, and then advances through time.

The method more closely emulates a procedure that is recalibrated during its operating life, but it is not definitive proof. The result depends on window type and length, refit frequency, label horizon, selection rules, costs, and the number of configurations considered. If fold results influence choices, they are part of development; concatenating them does not automatically create a pristine final test.

Main designs

There is no universally correct 70/30 split. An intraday system with overlapping labels, a monthly strategy, and a macro model with few recessions contain very different amounts of information and dependence, even with the same number of rows in the dataset.

Rolling or expanding

At origin t, an expanding design uses all observations allowed through t; a rolling design uses only the most recent w. The former maximizes history, while the latter lets the process forget remote data. That choice expresses an assumption about stability, not a mere technical detail.

Every configuration should state:

• training, validation, and test start and end for each fold;
• whether the window is rolling or expanding, and why;
• the step between origins and the evaluated horizon;
• the frequency of refitting, recalibration, and parameter reselection;
• treatment of warm-up, missing values, new instruments, and discontinued instruments;
• aggregation of results and dependence among folds;
• the date on which a decision based on the folds is made.

If each step chooses the best parameter using the next segment, that segment is validation, not test. To estimate the procedure honestly, internal selection must be reproduced using only information available at that origin.

Overlapping labels, gaps, purging, and embargo

In financial problems, an observation can use information over an interval, not a single timestamp. If a label measures the return over the following five days, adjacent labels share days. A row formally before the test can therefore contain outcomes that enter the test information interval.

Purging removes training observations whose information intervals overlap the test. A gap or embargo leaves a justified temporal separation to reduce contamination associated with overlap or dependence. In finance, these tools are associated with the formulation popularized by López de Prado; they are not an automatic cure. Their width should depend on the horizon, feature construction, labels, and operating process, not on a fixed percentage.

Removing rows does not correct preprocessing estimated on the full sample, a universe built from survivors, or future revisions. The control must cover the entire pipeline.

The case of random k-fold

Saying that random k-fold is always invalid for time series would be too strong. Bergmeir, Hyndman, and Koo show conditions under which standard cross-validation can be valid for purely autoregressive models, particularly when errors are uncorrelated. The lesson is not that shuffling is generally safe, but that validity depends on the structure of the problem.

In trading, causal ordering, estimated transformations, changing universes, overlapping labels, and costs normally make a chronological design that reproduces intended use more prudent. Anyone choosing a non-chronological design should establish the conditions that make it appropriate and examine the residuals, rather than merely invoking the method's name.

Illustrative example

A procedure is recalibrated at the start of each month. An expanding fold uses all data available through the end of the previous month, selects a configuration through an internal temporal validation, and generates positions for the following month. Twelve months produce twelve evaluation segments.

If the analyst inspects the twelve results and changes the features, the full set has informed development. The experiment can be described and the process can be repeated on a new period, but those months should no longer be presented as a never-seen final test. The example does not establish that one month is the correct step; that depends on the strategy's horizon and frequency.

Aggregate without losing uncertainty

Adding the returns into a walk-forward curve shows the path of a simulated procedure, but it does not make the folds independent. Useful reporting includes:

• fold-level and aggregate results, not only the mean;
• dates, regimes, instruments, and exposures that concentrate P&L;
• gross and net metrics against the same benchmark;
• the error distribution and stability of the selection;
• turnover, capacity, and recalibration frequency;
• every configuration considered in the process.

Variation across folds can be informative even when the mean is positive. A result dominated by one origin or one crisis calls for a narrower conclusion.

Checklist

1. Reconstruct the complete temporal diagram before calculating metrics.
2. Align windows and step size with actual use and the information horizon.
3. Perform preprocessing, feature selection, and tuning inside each training set.
4. Apply a gap or purging only when justified by interval structure.
5. Count every reuse of fold results as development information.
6. Where possible, preserve a separate final test or prospective phase.
7. Report dependence, dispersion, and concentration of results.

Sources

• Leonard J. Tashman, Out-of-sample tests of forecasting accuracy: an analysis and review — taxonomy and rationale for rolling-origin out-of-sample evaluation.
• Christoph Bergmeir, Rob J. Hyndman, and Bonsoo Koo, A note on the validity of cross-validation for evaluating autoregressive time series prediction — precise conditions under which k-fold can be valid for autoregressive models.
• Jeffrey S. Racine, Consistent cross-validatory model-selection for dependent data: hv-block cross-validation — a block-separation proposal for dependent data, to be used with its assumptions and subsequent methodological discussion.
• Marcos López de Prado, Advances in Financial Machine Learning — official Wiley table of contents — authorial reference for purging and embargo; it is not presented as universal consensus or as an original peer-reviewed paper.

Related pages

• Systematic trading: strategies, backtests, and validation
• Out of sample
• Data leakage and look-ahead bias
• Overfitting
• Forward test