Module 15: Transformers

The architecture behind modern NLP and large language models. This module develops attention from first principles through the full Transformer, and connects it to the pretrained models used in practice.

What You’ll Learn¶

• Self-attention: queries, keys, values, and scaled dot-product attention

• Multi-head attention and positional encodings

• The Transformer encoder–decoder architecture

• Pretraining and finetuning paradigms (BERT-style vs. GPT-style)

• Working with pretrained transformers in practice

Lecture Materials¶

• 📊 Lecture Slides (PDF)

• 📓 Chapter notebook: Transformers — Improving NLP with Attention Mechanisms

Prerequisites¶

• Module 13 (sequence modeling)

Reference¶

Raschka & Mirjalili, Machine Learning with PyTorch and Scikit-Learn, Chapter 16. Continue to Module 16 for the full LLM lecture sequence.