The leak that does matter
Machine Learning · from Capstone: diagnose a broken model
leakagebaselinesmetric choicestratifybest_score_overfittingPutting the kettle on…
Starting up…
Exercise 3
The leak that does matter. Build 200 columns of pure noise, select the 5 "best" using all of y, then cross-validate. Return [leaked_auc, honest_pipeline_auc], rounded to 4 places — the first should be well above 0.5 on data with no signal at all.
your answer
stuck? the lesson behind it
Capstone: diagnose a broken model
Six faults in one notebook, and two that are real but too small to see