Error and Residuals

Gist

Error is how the predicted values deviate from the true population value. Therefore, this is something that cannot be observed. We can only observe Residuals

Gist

The vertical distance between the predicted and observed values. Supposed to be the proxy of Error and Residuals

Linear regression

In Linear regression (LM), the residuals are calculated as: $r_i=y_i-ModelOutput$.

Residual standard deviation

Summarizes the scale of the residuals

Other uses

Some people use the residuals from a regression as data but that seems to be a bad idea most of the time (Gary King 1986, Freckleton 2002)

Diagnostic plots (Linear Regression)

Shiny idea: A plot that shows how changing the parameter changes the residual plots?

Time Series Analysis

In Forecasting, the residuals can often be called innovation residuals

Personal notes

Reference

1. Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge university press.
2. Freckleton, R. P. (2002). On the misuse of residuals in ecology: regression of residuals vs. multiple regression. Journal of Animal Ecology, 542-545.
3. Gary King. 1986. “How Not to Lie With Statistics: Avoiding Common Mistakes in Quantitative Political Science”. American Journal of Political Science, 30, Pp. 666–687.

In Time-Series Analysis