Enter your search text in the box above

Select a result to preview

Stepwise variable selection

#regression #model-selection #done

Definiton

The process of adding or removing predictors based on its p-value. Used in regression for Model selection but highly criticized and never ever recommended now[1]. Though it is curious that some online resources still have a section on it (see 2)

Don't do it!

Seems like a real bad idea to do! People point to either Lasso Regression or Ridge regression

Proof: Simulating

In Free Range Stats, the author simulated datasets to show that the significant variables are often not chosen (false negative) and when chosen, the coefficient estimates are inflated.

[!info] Don't do it!

No seriously!

Personal note

Oof, I’m guilty of having done this when I was an undergraduate! I still don’t have an intuitive way of explaining why it’s wrong.

References

  1. Olden, J. D., & Jackson, D. A. (2000). Torturing data for the sake of generality: how valid are our regression models?. Ecoscience, 7(4), 501-510.
  2. https://online.stat.psu.edu/stat462/node/196/
  3. https://freerangestats.info/blog/2024/09/14/stepwise

  1. 1 ↩︎