Advice - Regresion
Gist
Advice on anything regression such as Linear regression (LM), Generalized linear models (GLM), and so on.
Purpose
In clinical trial, regression serves to:
- Estimating adjusted effects.
- Predict outcomes
Interactions
Main effects rule: When you include an interaction term, you must also include both “main effects” (the individual terms). Never include without also including and separately. Dropping main effects changes the interpretation of the interaction term in misleading ways.
Do not test every possible interaction – this leads to multiple testing problems and overfitting. Only test interactions that are:
- Pre-specified based on clinical reasoning (e.g., “We hypothesise the drug works differently in men vs. women”)
- Scientifically plausible (there is a biological mechanism for effect modification)
- Adequately powered (interaction tests require much larger samples than main effect tests – roughly 4x the sample size)
Model metric
Maximizing the R2 is not the goal!
It is good for comparing models with and without certain predictor variables! It is also good for comparing to a null model!
Model selection
Variable selection
Don't remove variables just because it's not significant
"People sometimes think that if a coefficient estimate is not significant, then it should be excluded from the model. We disagree. It is fine to have nonsignificant coefficients in a model, as long as they make sense." (Gelman and Hill 2007)[1]
Residuals
Non-normal residuals are not that big of a deal
"Although this study strongly recommends the appropriate use of normality tests in linear modeling—which is to evaluate the residuals and not the raw data for normality—our simulations also show that if a normality test is applied to raw data, the subsequent choice of a parametric or non-parametric test has little difference in power." (Midway and White 2025)[2]