Advice - Regresion

#statistics #advice #regression

Gist

Advice on Linear regression

Don't remove variables just because it's not significant

"People sometimes think that if a coefficient estimate is not significant, then it should be excluded from the model. We disagree. It is fine to have nonsignificant coefficients in a model, as long as they make sense." (Gelman and Hill 2007)

Non normal residuals are not that big of a deal

"Although this study strongly recommends the appropriate use of normality tests in linear modeling—which is to evaluate the residuals and not the raw data for normality—our simulations also show that if a normality test is applied to raw data, the subsequent choice of a parametric or non-parametric test has little difference in power." (Midway and White 2025)

References

  1. Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge university press.
  2. Midway, S., & White, J. W. (2025). Testing for normality in regression models: mistakes abound (but may not matter). Royal Society Open Science, 12(4), 241904.