A problem that occurs when a model is too complex, performing well on the training data but poorly on unseen data. Example: A model that has memorized the training data instead of learning general patterns.
Overfitting is the most common way a machine learning project quietly fails. The metrics look excellent, the team ships, and the model falls apart on real traffic — because what it learned was the training set, not the thing the training set was supposed to represent.
How overfitting happens
A model has capacity: the amount of structure it can memorise. When capacity exceeds the signal actually present in the data, the model starts fitting noise — a mislabelled example, an artefact of how the data was collected, a spurious correlation that holds in your 10,000 rows and nowhere else.
The classic diagnostic is the gap between training and validation performance. Training accuracy climbing while validation accuracy flattens or falls is overfitting in progress. Google’s Machine Learning Crash Course has a good visual treatment of the curve.
How to prevent it
Hold out data properly. Split into train, validation, and test, and do not touch the test set until you are done. Every time you tune against validation, you leak a little information into the model — which is why the final test set has to stay sealed.
Regularise. L1 and L2 penalties push weights toward zero. Dropout randomly disables neurons during training so the network cannot rely on any single path. Both make it harder for the model to memorise.
Stop early. Track validation loss and halt when it stops improving, rather than running to a fixed epoch count.
Get more data, or synthesise it with data augmentation. More genuine variation is the most reliable fix there is.
Worth knowing: large language models complicate this picture. They are trained on corpora big enough that classical overfitting is less of a concern during pre-training — but it returns sharply during fine-tuning on a small dataset, where a few hundred examples can teach a model to parrot your samples rather than generalise from them.
Related terms: Machine Learning (ML), Supervised Learning, Optimization, Benchmarking.
