Data leakage and look-ahead bias

Data leakage is contamination of a research procedure by information that should not have been available at the point where it is used. Look-ahead bias is the temporal case: a decision at time t directly or indirectly incorporates data known only after t. The result is not merely optimistic; it simulates a different strategy equipped with a nonexistent information channel.

Essential distinctions

Survivorship bias is distinct: it arises from observing only entities that survived or were selected after the fact. It can coexist with leakage, but correcting timestamps alone does not recreate failed instruments, closed funds or former index members.

Three clocks: signal, order and fill

A trading rule must state at least three moments:

1. Signal time: when the signal is ready; distinguish this from the information cutoff used in its calculation.
2. Order time: creation and submission, followed by arrival and acceptance at the venue.
3. Fill time: the moment or interval when an admitted order can meet compatible prices and quantities.

If the decision requires the final price close[t], it cannot determine participation in the same auction that forms that price. An on-close order submitted earlier must respect venue cutoffs and use information then available: the cited NYSE sheet separates order entry from the auction process. A later fill might have the same numerical price, but not the same time. A bar's final high and low are unavailable as inputs before the bar is complete; an intrabar replay must expose only events already observed at each decision.

For releases, filings and macroeconomic data, the economic-period date is not the dissemination timestamp. The strategy must also have received and be able to process the data. Point-in-time data retain versions (vintages): a later revision must not replace the version available at the decision. ALFRED reconstructs information known on a historical date; it does not by itself certify intraday timing or a particular vendor's latency.

Where leakage hides

Features and transformations

• centred averages, bidirectional filters or interpolation using future observations;
• cross-sectional ranks calculated on a universe reconstructed today;
• winsorisation, imputation or standardisation estimated on the whole dataset;
• fundamentals aligned with quarter-end rather than filing time;
• splits and dividends applied without distinguishing effective date from the date on which they became known.

To standardise a variable: z = (x − mean_train) / standard_deviation_train. Estimate mean and deviation within each fold's training set and reuse them on its test set; handle zero deviation explicitly. The scikit-learn documentation distinguishes fit from transform.

Research and selection

Data snooping reuses a sample for inference or selection: it can produce a chance winner even when every input respects time. It therefore differs from look-ahead.

The test set can be contaminated without any future-looking column. Repeatedly consulting its outcome to select features, thresholds or models gradually turns it into development data. Out-of-sample is not a file label; it is a role in the process. The number of alternatives attempted, rejected decisions and selection criteria must be recorded.

Orders and execution

An OHLC price does not demonstrate that an order would have filled. Stops and limits may be touched without execution; available quantity may be insufficient; queue priority may be unknown; delays and partial fills change the position. Always assigning the most favourable bar price is a form of hindsight, separate from but connected to transaction costs in backtests.

Causal protocol

1. Write the decision before the code. List inputs, timestamps, time zones, cutoffs, frequency and the action produced.
2. Version raw data. Preserve snapshots, vintages, the universe and corporate actions without overwriting history.
3. Compute every feature “as of”. At decision time t, exclude records with available_at > t and select each datum's then-available version; document how available_at is constructed.
4. Fit on train only. Estimate imputation, scaling and feature selection within each fold's training set; apply the learned transformations to its test set. Select parameters using validation while preserving the final test.
5. Respect temporal order. Every training label must already be observable at the training cutoff. Where information intervals overlap the test, assess purging and gaps from actual dates, rather than row proximity alone.
6. Freeze the specification. Record rules, parameters, universe, benchmark and metrics before final evaluation.
7. Separate signal, order and fill. Apply delays, calendars, order types and genuinely observable data.
8. Preserve every experiment. An experiment ledger reduces invisible data snooping.
9. Repeat from a clean snapshot. The result must be reproducible without intermediate files built with future information.

Diagnostic tests

A collapse after these tests is a warning, not automatic proof of the cause. A stable result does not prove the absence of leakage either: some leaks can survive perturbations.

Future labels and training boundaries

The label is the outcome to predict. For example, the return over the five sessions following row t becomes known only after those sessions. Using it as a target in later training is normal; placing it among the inputs at t, or using it in training completed before it matures, introduces future information. Removing labels beyond the cutoff differs from prohibiting any correlation between observations. The scikit-learn gap counts excluded rows: it does not automatically reconstruct each label's duration and availability.

Limitations

Temporal splits, walk-forward procedures and cross-validation do not repair data already contaminated. Purging and embargoes must follow the structure of the labels rather than being applied as rituals. A causal dataset may still suffer measurement errors, multiple selection, regime change, low power or an incomplete universe. Validation therefore requires both statistical controls and an audit of the information path.

Sources

• Halbert White, A Reality Check for Data Snooping, Econometrica, 2000 — inference when the selected outcome emerges from many explored specifications.
• Robert D. Arnott, Campbell R. Harvey and Harry Markowitz, A Backtesting Protocol in the Era of Machine Learning — separation of research, selection and holdout testing under scarce financial data.
• David H. Bailey, Jonathan M. Borwein, Marcos López de Prado and Qiji Jim Zhu, The Probability of Backtest Overfitting — overfitting risk arising from selection among many configurations.
• Federal Reserve Bank of St. Louis, FRED® versus ALFRED® — the difference between historical information available today and the vintage known in the past.
• scikit-learn, Common pitfalls — Data leakage and StandardScaler — training, preprocessing and application to the test set.
• NYSE, Opening and closing auctions, p. 2 — order-entry and closing-auction sequence in the market described.
• scikit-learn, TimeSeriesSplit — time-ordered cross-validation and the gap parameter; official technical documentation.

Links

• Systematic trading
• Backtest
• Out-of-sample
• Point-in-time data
• Survivorship bias
• Transaction costs in backtests