Module 2: Linear Classifiers

This module builds the first trainable models of the course from scratch: the perceptron and adaptive linear neurons (Adaline). These simple algorithms introduce the two ideas everything later depends on — decision boundaries and gradient-based learning.

What You’ll Learn¶

• The perceptron learning rule and its convergence behavior

• Adaline and minimizing a continuous loss with gradient descent

• Batch vs. stochastic gradient descent

• Feature scaling and why it matters for optimization

• Implementing a classifier in NumPy from first principles

Lecture Materials¶

• 📊 Lecture Slides (PDF)

• 🧮 Math Notes (PDF)

• 📓 Chapter notebook: Training Simple Machine Learning Algorithms for Classification

Lecture Videos¶

Part 1: Artificial Neurons and the Perceptron | CAP 6606 — Module 2, Part 1¶

Part 2: Implementing a Perceptron in Python | CAP 6606 — Module 2, Part 2¶

Part 3: Adaline and Gradient Descent | CAP 6606 — Module 2, Part 3¶

Part 4: Implementing Adaline and Feature Scaling | CAP 6606 — Module 2, Part 4¶

Part 5: Batch, Stochastic, and Mini-Batch Gradient Descent | CAP 6606 — Module 2, Part 5¶

Prerequisites¶

• Module 1

• Comfort with vectors, dot products, and partial derivatives

Reference¶

Raschka & Mirjalili, Machine Learning with PyTorch and Scikit-Learn, Chapter 2.