Course syllabus
LLM Inference Systems
Learn how a transformer becomes a fast, reliable serving system—from the first attention layer to distributed prefill and decode.
Implementing a Transformer
Build a minimal decoder-only language model from first principles.
Inference as a Systems Problem
Define what a fast, efficient inference system must optimize.
GPU Architecture for LLM Inference
Develop the hardware intuition needed to reason about inference performance.
Efficient Attention
Understand how modern attention kernels reduce memory traffic and latency.
KV Cache Systems
Manage the dominant source of dynamic memory in autoregressive serving.
Serving a Model with vLLM
Study a production serving engine from request arrival to token output.
Batching and Scheduling
Schedule competing requests while meeting latency and throughput goals.
Faster Decoding and Model Optimization
Reduce the work required to produce each useful output token.
Distributed LLM Inference
Scale models and workloads across multiple accelerators and nodes.
Disaggregated Prefill and Decode
Separate the two inference phases so each can scale independently.
Advanced Serving Workloads
Extend the serving stack to emerging model architectures and workloads.
Operating an Inference Platform
Run inference reliably under real production constraints.
Capstone: Meet the SLO
Design and optimize a serving system against a fixed workload and GPU budget.
- Define a workload and service-level objective
- Establish a single-GPU baseline
- Implement a minimal serving engine
- Profile the end-to-end system
- Optimize batching and KV cache management
- Evaluate quantization and speculative decoding
- Design a distributed or disaggregated deployment
- Report performance, quality, and cost