Error and Residuals
Gist
Error is how the predicted values deviate from the true population value. Therefore, this is something that cannot be observed. We can only observe the residuals.
Definition
The residual is the vertical distance between the predicted and observed values.
Linear regression
In Linear regression (LM), the residuals are calculated as: $$r_i = y_i - Model Output$$
Residual standard deviation
Summarizes the scale of the residuals
Other uses
Some people use the residuals from a regression as data but that seems to be a bad idea most of the time (Gary King 1986, Freckleton 2002)
Diagnostic plots (Linear Regression)
Shiny idea: A plot that shows how changing the parameter changes the residual plots?
Time Series Analysis
In Forecasting, the residuals can often be called innovation residuals.
- It is important that the autocorrelation after you fit the time-series model is White Noise
Personal notes
Reference
- Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge university press.
- Freckleton, R. P. (2002). On the misuse of residuals in ecology: regression of residuals vs. multiple regression. Journal of Animal Ecology, 542-545.
- Gary King. 1986. “How Not to Lie With Statistics: Avoiding Common Mistakes in Quantitative Political Science”. American Journal of Political Science, 30, Pp. 666–687.