Enter your search text in the box above

Select a result to preview

Stepwise variable selection

#regression #model-selection

Gist

The process of adding or removing predictors based on its p-value. Used in regression for Model selection but highly criticized and never ever recommended now[1]. Though it is curious that some online resources still have a section on it (see 2)

Don't do it!

Seems like a real bad idea to do! People point to either Lasso Regression or Ridge regression

Simulating

In Free Range Stats, the author simulated datasets to show that the significant variables are often not chosen (false negative) and when chosen, the coefficient estimates are inflated.

[!info] Don't do it!

No seriously!

Personal note

Oof, I’m guilty of having done this when I was an undergraduate! I still don’t have an intuitive way of explaining why it’s wrong.

References

  1. Olden, J. D., & Jackson, D. A. (2000). Torturing data for the sake of generality: how valid are our regression models?. Ecoscience, 7(4), 501-510.
  2. https://online.stat.psu.edu/stat462/node/196/
  3. https://freerangestats.info/blog/2024/09/14/stepwise

  1. 1 ↩︎