Breast cancer — check the label order

569 biopsies from the University of Wisconsin, 30 measurements each, and a target that means something: malignant or benign. Everything from module 2 applies here with consequences attached.

Check the label order first

cancer = load_breast_cancer(as_frame=True)
cancer.target_names      # array(['malignant', 'benign'])
np.bincount(cancer.target)   # [212, 357]

0 is malignant. 1 is benign. The class you care about is coded zero, and the class the library will treat as "positive" by default is the one where nothing is wrong.

Which quietly breaks every metric

recall_score(y_test, pred)                 # 0.9667  ← recall on BENIGN
recall_score(y_test, pred, pos_label=0)    # 0.9434  ← recall on MALIGNANT

scikit-learn's default is pos_label=1. So a bare recall_score tells you how reliably you identify healthy tissue — the opposite of the medical question — and it returns a healthy-looking number while doing it.

Nothing errors. Nothing warns. The same applies to precision_score, f1_score, average_precision_score, and to which column predict_proba you take. The only defence is reading target_names before you write a metric — on any dataset you didn't build yourself.

(If you'd rather not think about it every time, relabel once at the top: y = (cancer.target == 0).astype(int) makes malignant the positive class and every default correct thereafter.)

The honest numbers

accuracy              0.9580
dummy                 0.6294
recall (malignant)    0.9434

confusion matrix, rows = truth:
   [[50,  3],      50 malignant caught,  3 MISSED
    [ 3, 87]]       3 false alarms,     87 benign correct

A respectable model. And the only number worth acting on is the 3 in the top right: three malignancies called benign.

Three is not three

Out of 53 malignancies in the test set. In a spreadsheet that's a rounding error; in a clinic it is three people told they're fine.

And the two error types are wildly asymmetric. A false alarm means another test — unpleasant and expensive. A miss means a delayed diagnosis. This is where lesson 7's threshold stops being an exercise.

Move the cut

flag malignant if p(malignant) >= …

  0.5      3 missed,   3 false alarms
  0.3      2 missed,   5 false alarms
  0.2      2 missed,   6 false alarms
  0.1      1 missed,  10 false alarms

Going from 0.5 to 0.1 takes missed malignancies from 3 to 1 for seven extra false alarms. In a screening context that trade is obviously worth taking — and note that no model improvement was needed, only the willingness to move a number that defaults to 0.5.

This is also why screening programmes are deliberately over-sensitive, and why "the test came back positive" usually means "we need another test" rather than "you have it". Lesson 6's precision/recall trade, running in the real world.

Thirty correlated columns

The features come in three groups of ten: the mean, the standard error and the worst value of ten cell-nucleus measurements. So they're heavily correlated by construction — mean radius, mean perimeter and mean area are three views of one thing.

Which is exactly the situation lesson 25 warned about: permutation importance will show all three as unimportant, because permuting one leaves the model two others to read. Check correlations before interpreting anything here.

And what a model cannot do

These 30 features were computed from digitised images by researchers in 1993. A model trained on them predicts their labels, on their patient population, with their imaging equipment.

None of that transfers automatically to a different hospital, a different scanner, or a different decade — and no cross-validation score will tell you it doesn't, because every fold is drawn from the same 569 people. That limitation belongs in writing, which is lesson 35.