Course links
| Course | Links | Role |
|---|---|---|
| DeepMind x UCL Reinforcement Learning Lecture Series, 2021 — Hado van Hasselt, Diana Borsa, and Matteo Hessel | Lecture playlist · Unofficial slide mirror | Main course: thirteen lectures on the foundations, from bandits, Markov decision processes, and the theory of dynamic programming to model-free prediction and control, function approximation, planning and models, policy gradients, multi-step and off-policy learning, and deep RL (chapters 1–3, 5–13, 16, and 23). YouTube labels the playlist as a deep-learning series, but it holds the RL lectures. No official slides remain online; the mirror collects the thirteen decks. |
| UCL, 2015 — David Silver, Introduction to Reinforcement Learning | Course page with slides · Lecture playlist · Easy21 assignment · PDF | Main course: the classic ten-lecture course, with slides for every lecture: MDPs, dynamic programming, model-free prediction and control, value function approximation, policy gradients, learning and planning, exploration, and games (chapters 1–3, 5–8, 10–11, 13, 24, and 27). Its Easy21 assignment applies Monte Carlo control, SARSA(λ), and linear function approximation to a card game. |
| UC Berkeley CS285, Fall 2023 and Spring 2026 — Sergey Levine, Deep Reinforcement Learning | Course page with slides · Lecture playlist · Spring 2026 course · Spring 2026 playlist | Main course: the standard deep RL course: imitation, policy gradients, actor-critic, Q-learning, optimal control, model-based RL, exploration, offline RL, inverse RL, control as inference, and RL theory (chapters 1, 11, 13, 15–16, 19–23, 25–26, and 29–31). Each Fall 2023 lecture is split into several videos. The Spring 2026 edition supplies the lectures on Q-learning in practice and on RL with sequences and language models (chapters 18 and 28). Its homeworks implement imitation learning, policy gradients, Q-learning and actor-critic methods, model-based RL, and exploration and offline RL. |
| Stanford CS234, Spring 2024 — Emma Brunskill, Reinforcement Learning | Course page with slides · Lecture playlist | Supplement: sixteen lectures, strongest on data-efficient RL: tabular planning, model-free evaluation and control, policy search, bandits and regret, Bayesian and PAC exploration, offline RL and policy evaluation, imitation and learning from preferences, and Monte Carlo tree search (chapters 1–5, 7, 9–10, 13, 20, 22, and 24–26). |
| Stanford CS224R, Spring 2025 and 2026 — Chelsea Finn, Deep Reinforcement Learning | Course page with slides · Spring 2025 playlist | Supplement: deep RL for robotics and language models: imitation, off-policy actor-critic, offline RL, reward learning, RL for language-model reasoning, multi-task and goal-conditioned RL, meta-RL, hierarchy, sim-to-real, and RL fine-tuning of vision–language–action models (chapters 21, 25–26, 28–29, and 31). The reading plan links the 2026 slides; the 2025 recordings cover the same lectures under the same numbers. |
| Pieter Abbeel, Foundations of Deep RL, 2021 | Lecture playlist | Supplement: six compact lectures: exact methods and maximum-entropy RL, deep Q-learning, policy gradients and advantage estimation, TRPO and PPO, DDPG and SAC, and model-based RL (chapters 13, 16, 20–21, and 23). |
| University of Alberta — Martha White and Adam White, Reinforcement Learning Specialization | Coursera | Supplement for Part I: four short courses that follow Sutton and Barto closely: bandits, MDPs, and dynamic programming; Monte Carlo, TD, Q-learning, and Dyna; prediction and control with function approximation and policy gradients; and a capstone (chapters 1–3, 5–7, 10–11, and 13). |
| UC Berkeley CS287, Fall 2019 — Pieter Abbeel, Advanced Robotics | Course page with slides and videos | Supplement for control and partial observability: exact methods and function approximation for MDPs, LQR and iterative LQR, trajectory optimization and model-predictive control, Bayes filters, POMDPs, imitation, and sim-to-real transfer (chapters 11, 14–15, 25, and 31). |
| CMU 16-745, 2025 — Zac Manchester, Optimal Control and Reinforcement Learning | Lectures and notes · Lecture playlist · Notebooks | Supplement for control: LQR from three viewpoints, convex MPC, differential dynamic programming, and direct trajectory optimization, with lecture notebooks (chapter 15). |
| Harvard CS2824, Spring 2026 — Kianté Brantley and Sham Kakade, Foundations of Reinforcement Learning | Course page with slides | Supplement for theory: sample complexity, exploration, function approximation, policy optimization, and offline RL with proofs, taught by two of the authors of RLTA (chapter 30). |
| Cornell CS6789, Fall 2024 — Wen Sun | Course page with slides | Supplement for theory: slides on value iteration, policy iteration and linear programming, multi-armed, contextual, and linear bandits, fitted Q-iteration, sample complexity, UCBVI, and Bellman rank (chapters 2–4, 12, and 30). |
| University of Alberta CMPUT 653 — Csaba Szepesvári | Lecture notes | Supplement for theory: lecture notes on the theory of planning and learning in MDPs, with function approximation (chapter 30). |
| Princeton ECE524, Spring 2024 — Chi Jin, Foundations of Reinforcement Learning | Lecture playlist | Supplement for theory: recorded lectures on the foundations of RL theory (chapter 30). |
| Stefano Albrecht, Filippos Christianos, and Lukas Schäfer, Multi-Agent Reinforcement Learning lectures | Lecture playlist · Slides | Supplement for multi-agent RL: lectures that follow the authors’ book on games, solution concepts, and multi-agent deep RL (chapter 27). |
| Nathan Lambert, Reinforcement Learning from Human Feedback | Book · Course | Supplement for language models: the book and lectures on post-training, with policy-gradient algorithms for language models and reasoning; lectures 3–5 and chapter 6 of the book serve chapter 28. |
Code resources complement the courses:
| Resource | Link | Role |
|---|---|---|
| CS285 homeworks | Course page | Five homeworks with starter code: imitation learning (chapter 25), policy gradients (chapter 13), Q-learning and actor-critic algorithms (chapters 16 and 21), model-based RL (chapter 23), and exploration and offline RL (chapters 22 and 26). |
| Easy21 | Assignment · PDF | Silver’s assignment: Monte Carlo control, SARSA(λ), and linear function approximation on a card game like blackjack (chapters 5, 8, and 11). |
| 16-745 notebooks | Repository | The lecture notebooks of CMU 16-745 for LQR, MPC, differential dynamic programming, and trajectory optimization (chapter 15). |
| Hugging Face Deep RL course | Course | Notebooks for Q-learning, DQN, REINFORCE, A2C, PPO, and self-play (chapters 6–7, 13, 16, 19–20, 23, and 27). |
| ShangtongZhang, reinforcement-learning-an-introduction | Repository | Sutton and Barto’s figures reproduced in Python (chapters 1–3 and 5–13). |
| CleanRL | Documentation | Single-file, benchmarked implementations of DQN, C51, Rainbow, PQN, PPO, DDPG, TD3, SAC, and RND (chapters 16–18 and 20–22). |
The reading plan maps individual lectures, slides, and readings to the study sequence; its topic N is chapter N. Video links arranges the recordings by chapter, Books and documentation maps the reference books, libraries, and tutorials to the chapters, and the RL overview lists the chapters and labs.