Enter your search text in the box above

Select a result to preview

Advice - Regresion

#statistics #advice #regression

Gist

Advice on anything regression such as Linear regression (LM), Generalized linear models (GLM), and so on.

Model metric

Maximizing the R2 is not the goal!

It is good for comparing models with and without certain predictor variables! It is also good for comparing to a null model!

Model selection

Variable selection

Don't remove variables just because it's not significant

"People sometimes think that if a coefficient estimate is not significant, then it should be excluded from the model. We disagree. It is fine to have nonsignificant coefficients in a model, as long as they make sense." (Gelman and Hill 2007)[1]

Residuals

Non-normal residuals are not that big of a deal

"Although this study strongly recommends the appropriate use of normality tests in linear modeling—which is to evaluate the residuals and not the raw data for normality—our simulations also show that if a normality test is applied to raw data, the subsequent choice of a parametric or non-parametric test has little difference in power." (Midway and White 2025)[2]

  1. Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge university press. ↩︎

  2. Midway, S., & White, J. W. (2025). Testing for normality in regression models: mistakes abound (but may not matter). Royal Society Open Science, 12(4), 241904. ↩︎