Module 6: Model Evaluation & Hyperparameter Tuning - Machine Learning for Intelligent Systems & Robotics

Best practices for knowing whether your model is actually good: honest evaluation protocols, systematic hyperparameter search, and the diagnostic tools that tell you why a model underperforms.

What You’ll Learn¶

Pipelines for leak-free preprocessing + modeling
K-fold cross-validation and stratification
Learning curves and validation curves for diagnosing bias vs. variance
Grid search and randomized/successive-halving hyperparameter search
Confusion matrices, precision/recall, F1, ROC-AUC, and metrics for imbalanced classes

Lecture Materials¶

📝 Lecture Notes (PDF)
📓 Chapter notebook: Learning Best Practices for Model Evaluation and Hyperparameter Tuning

Lecture Videos¶

Part 1: Introduction and Module Overview | CAP 6606 — Module 6, Part 1¶

Part 2: Streamlining Workflows with Pipelines | CAP 6606 — Module 6, Part 2¶

Part 3: Holdout and K-Fold Cross-Validation | CAP 6606 — Module 6, Part 3¶

Part 4: Learning and Validation Curves | CAP 6606 — Module 6, Part 4¶

Part 5: Hyperparameter Search with Grid and Randomized Search | CAP 6606 — Module 6, Part 5¶

Part 6: Nested Cross-Validation for Algorithm Selection | CAP 6606 — Module 6, Part 6¶

Part 7: The Confusion Matrix | CAP 6606 — Module 6, Part 7¶

Part 8: Optimizing Precision and Recall | CAP 6606 — Module 6, Part 8¶

Part 9: ROC Curves and AUC | CAP 6606 — Module 6, Part 9¶

Part 10: Multiclass Metrics, Class Imbalance, and Summary | CAP 6606 — Module 6, Part 10¶

Prerequisites¶

Modules 3–5

Reference¶

Raschka & Mirjalili, Machine Learning with PyTorch and Scikit-Learn, Chapter 6.