TradehouseBuyer guides & toolsPricingPart of TradeHouseTradeHouse Fidelity Lab · Buyer resources
What is overfitting and underfitting?
Overfitting and underfitting are the two ways a model can fail to generalise. An overfit model is too flexible: it learns the noise in the sample and does badly on new data. An underfit model is too rigid: it misses real structure and does badly on both old and new data. The art is choosing complexity that matches the signal you actually have.
The bias-variance picture
Error on new data splits into bias (the model is systematically wrong because it is too simple) and variance (the model changes a lot depending on which sample it saw). Underfitting is high bias. Overfitting is high variance. Adding parameters lowers bias and raises variance, so test error is U-shaped as complexity grows.
| Underfitting | Overfitting | |
|---|---|---|
| Training performance | Poor | Excellent |
| Performance on new data | Poor | Much worse than training |
| Typical cause | Too few features or too simple a rule | Too many parameters or too many variants tried |
| Gap between train and test | Small | Large |
| Fix | Add informative features, a richer model | Simplify, regularise, gather more independent data, freeze the rule before testing |
A trading example of each
Underfit. A rule that buys whenever price is above a 200-day average and ignores everything else may have a modest edge in one regime and none in another. Both in-sample and out-of-sample results are mediocre, and the rule's drawdowns are about what a buy-and-hold would show. The model is too simple to separate the cases that matter.
Overfit. A rule with a 17-day lookback, a 2.3 standard-deviation entry, a Tuesday filter and a volatility gate chosen from 500 combinations can show a spectacular curve. Each parameter was picked because it improved the historical result. On later data it behaves like a random rule that happens to have fees.
Why overfitting is the bigger risk in finance
Market data has a low signal-to-noise ratio and a limited number of independent observations. Ten years of daily returns is roughly 2,500 data points, far fewer than a typical machine-learning dataset, and they are not independent because of volatility clustering. A flexible model can fit that noise almost perfectly. Meanwhile the incentive to keep the best-looking result is strong.
Diagnosing which one you have
- Compare training and held-out performance. A big gap means overfitting; two weak numbers mean underfitting.
- Check sensitivity. If nudging a parameter wrecks the result, you are on a spike, which is typical of overfitting.
- Count your attempts. The number of variants matters as much as the number of parameters.
- Check the data pipeline for artefacts such as duplicated rows and lookahead.
Where Fidelity Lab helps
Fidelity Lab focuses on the overfitting side. A report on your trade file freezes the rule on training data, scores it on the held-out segment, re-nets costs and applies multiplicity control across everything you submitted. It cannot tell you whether your model is too simple, but a DISSOLVES verdict that cites out-of-sample decay is a strong hint that complexity was ahead of the evidence.
Practical rules of thumb
Start with the simplest rule that expresses the idea and add a parameter only when it fixes a specific, explainable failure rather than a drawdown you disliked. Keep the number of free parameters small relative to the number of independent trades, not rows. Prefer rules with a plateau of acceptable settings. Reserve the most recent period as a final check that you look at once. When training and test results are close and both are modest, you are probably looking at a real but small effect, which is a better outcome than a spectacular training score with a collapse behind it.
Frequently asked questions
Which is worse, overfitting or underfitting?
In trading, overfitting is usually worse because it feels like success until money is at risk. Underfitting is visible immediately.
Does more data fix overfitting?
More independent data helps. More rows from the same few days does not.
What is regularisation?
A penalty on model complexity, such as shrinking coefficients, that trades a little bias for lower variance.