교차 검증: 하나의 열차/테스트 스플릿이 거짓말을 하는 이유

You split your data 80/20, get 91% accuracy, and ship it. But was that 91% luck or skill? A single split can fool you. Cross-validation gives you a trustworthy number. Here’s k-fold, visualized.

🔁 Watch the folds rotate: https://dev48v.infy.uk/ml/day18-cross-validation.html

The problem with one split

One train/test split is high-variance: a lucky test set flatters your model, an unlucky one trashes it. You’re judging on a single roll of the dice.

k-fold cross-validation

Split the data into k equal folds. Then, k times: train on k−1 folds, validate on the held-out one. You get k scores — report the mean ± std. Every data point gets used for both training and validation (in different rounds), so the estimate is stable.

The demo rotates each fold through validation, fits a real model per fold, fills in the per-fold scores, and shows the average — next to a single split you can reshuffle to watch it swing.

The disciplines that matter

• Use CV to tune hyperparameters, but keep a final test set you never touch.
• Fit scalers/encoders inside each fold (or you leak).
• Stratify for imbalanced classes; don’t shuffle time-series.

Cost: k× the training. Worth it for an honest score.

🔨 Built from scratch (split into folds → train/score each → mean±std → grid-search) on the page: https://dev48v.infy.uk/ml/day18-cross-validation.html

Part of MachineLearningFromZero. 🌐 https://dev48v.infy.uk