Learn a mapping from examples
Supervised learning fits a function from inputs to labelled outputs: regression for continuous targets, classification for discrete ones. Everything else — architecture, optimiser, tuning — is machinery around this single objective.
Splits and leakage
Data splits into training, validation and test sets. The validation set steers hyperparameters; the test set is touched once. Leakage — any information from the future or from the test set entering training — is the single most common cause of a model that looks excellent offline and fails in production.
Underfitting and overfitting
A model too simple for the data underfits (high bias). A model that memorises training noise overfits (high variance). Regularisation, more data, early stopping and cross-validation move you towards the sweet spot.
- Train error low, validation error high → overfitting.
- Both errors high → underfitting.
- Both low and close → healthy generalisation.