Overfitting

Gist

When a model fits really well to this set of data... But does not generalize well to new data.

Definition

Overfitting means that the model is not only fitting the systematic component of the data, but also the noise.

Analogy I like

"If the suit is tailored to the customer’s exact measurements, it will likely not be useful beyond immediate use, as small fluctuations in weight and body size over time are normal."

Reference

Practical