Drop the Weak Ones
df has a column y and several predictor columns. Trim the model by backward elimination:
1. Fit y on all predictors still in the model (with an intercept).
2. If the largest p-value among the predictors is above 0.05, remove that predictor and go back to step 1.
3. Otherwise stop.
Return the names of the predictors left, in alphabetical order (possibly none).
Example 1
df {"a":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15],"b":[5,3,8,1,9,2,7,4,6,0,3,8,5,1,9],"y":[3.2,4.9,7.1,9.2,10.8,13.1,15.2,16.8,19.1,21.0,23.2,24.9,27.1,28.8,31.2]}
returns ["a"]
y climbs by about 2 per unit of a. Column b is noise: its p-value is far above 0.05, so it goes, and a stays.
Example 2
df {"a":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15],"b":[5,3,8,1,9,2,7,4,6,0,3,8,5,1,9],"y":[6.2,4.1,5.9,7.2,4.8,6.1,5.2,6.8,4.1,5.0,7.2,4.9,6.1,5.8,5.2]}
returns []
Neither column explains y. b goes first, then a.
Constraints
- 2 to 5 predictors with one-letter names
15 <= len(df) <= 50- No p-value is close to 0.05, and the two largest are never close to each other
Write it in your browser, against the clock. The model solution, stepped through line by line, comes after.
More statsmodels problems
- Effect of the ProgramHard
- Enough PeopleHard
- Honest ErrorHard
- Next Three ValuesHard
- Slope Through TimeHard
- The Repeating PartHard
- Which Findings SurviveHard
- How Much Is ExplainedBeginner