Background Notes on Machine Learning and AI

Supplementary notes for the course. They develop in full the background that the lectures assume: the mathematics of learning, machine learning, deep learning, classical AI, NLP and large language models, generative models, and reinforcement learning. Modules marked work in progress are still being revised.

• Foundations
  • 1. Terminology and Mathematical Language
  • 2. Linear Algebra
  • 3. Calculus and Optimization
  • 4. Probability and Statistics
  • 5. Information and Learning Theory
  • 6. Numerical Computing with NumPy and PyTorch
• Machine Learningwork in progress
  • 1. Learning Problems and Nearest Neighbors
  • 2. The Perceptron and Linear Separation
  • 3. Linear Regression and Regularization
  • 4. Generative Classifiers
  • 5. Logistic Regression and Probabilistic Prediction
  • 6. Losses, Model Selection, and Evaluation
  • 7. Statistical Learning Theory
  • 8. Support Vector Machines and Kernels
  • 9. Decision and Regression Trees
  • 10. Bagging and Random Forests
  • 11. Boosting
  • 12. Principal Components and Dimensionality Reduction
  • 13. Clustering
  • 14. Gaussian Mixtures and Expectation Maximization
  • 15. Gaussian Processes
  • 16. Semi-Supervised and Active Learning
  • 17. Smoothing, Density Estimation, and Basis Expansions
• Deep Learningwork in progress
  • 1. Deep Feedforward Networks
  • 2. Initialization and Signal Propagation
  • 3. Optimization for Deep Networks
  • 4. Normalization and Residual Connections
  • 5. Regularization and Generalization in Deep Networks
  • 6. Convolutional Networks
  • 7. Convolutional Architectures and Transfer Learning
  • 8. Recurrent Networks
  • 9. Attention and Transformers
  • 10. Self-Supervised Representation Learning
  • 11. Training at Scale and Efficient Inference
  • 12. Practical Methodology
  • 13. Graph Neural Networks
  • 14. Detection and Segmentation
  • 15. Infinite Width and the Neural Tangent Kernel
• Artificial Intelligencework in progress
  • 1. Agents and Uninformed Search
  • 2. Heuristic Search
  • 3. Constraint Satisfaction and Local Search
  • 4. Adversarial Search and Games
  • 5. Propositional Logic and Satisfiability
  • 6. First-Order Logic and Knowledge Representation
  • 7. Automated Planning
  • 8. Bayesian Networks and Markov Networks
  • 9. Exact Inference
  • 10. Approximate Inference
  • 11. Temporal Probabilistic Models
  • 12. Decision Theory and the Value of Information
  • 13. Causal Inference
  • 14. Learning Graphical Models
  • 15. Game Theory and Multiagent Systems
• NLP and Large Language Modelswork in progress
  • 1. Text, Tokens, and Tokenization
  • 2. N-gram Language Models and Perplexity
  • 3. Word Embeddings
  • 4. Transformer Language Models
  • 5. Pretraining and Transfer
  • 6. Pretraining Data
  • 7. Scaling Laws
  • 8. Decoding and Text Generation
  • 9. In-Context Learning and Prompting
  • 10. Fine-Tuning and Parameter-Efficient Adaptation
  • 11. Learning from Human Preferences
  • 12. Reasoning and Test-Time Compute
  • 13. Retrieval, Tools, and Agents
  • 14. Evaluating Language Models
  • 15. Machine Translation and Multilingual Models
  • 16. Syntactic Parsing
• Generative AIwork in progress
  • 1. Foundations of Generative Modeling
  • 2. Autoregressive Models
  • 3. Variational Autoencoders
  • 4. Normalizing Flows
  • 5. Generative Adversarial Networks
  • 6. Energy-Based Models and Score Matching
  • 7. Denoising Diffusion Models
  • 8. Diffusion SDEs and Fast Sampling
  • 9. Flow Matching
  • 10. Guidance and Conditional Generation
  • 11. Latent Diffusion and Large-Scale Generation
  • 12. Discrete Tokens and Multimodal Generation
  • 13. Evaluating Generative Models
  • 14. Generative Models for Science and Control
  • 15. The Theory of Diffusion Models
• Reinforcement Learningwork in progress
  • 1. Markov Decision Processes
  • 2. Dynamic Programming
  • 3. Multi-Armed Bandits
  • 4. Contextual, Bayesian, and Adversarial Bandits
  • 5. Monte Carlo Methods
  • 6. Temporal-Difference Learning
  • 7. Model-Free Control
  • 8. Multi-Step Bootstrapping and Eligibility Traces
  • 9. Off-Policy Learning
  • 10. Planning and Learning with Tabular Models
  • 11. Value Function Approximation
  • 12. The Deadly Triad and Gradient-TD Methods
  • 13. Policy Gradient and Actor-Critic Methods
  • 14. Partially Observable Environments
  • 15. Optimal Control and Trajectory Optimization
  • 16. Deep Q-Learning
  • 17. Distributional Reinforcement Learning
  • 18. Data-Efficient and Scalable Value-Based Agents
  • 19. Deep Actor-Critic and Distributed RL
  • 20. Trust Regions and Proximal Policy Optimization
  • 21. Continuous Control and Maximum-Entropy RL
  • 22. Exploration in Deep RL
  • 23. Model-Based RL and World Models
  • 24. Planning with Learned Models
  • 25. Imitation Learning and Inverse RL
  • 26. Offline Reinforcement Learning
  • 27. Multi-Agent RL and Self-Play
  • 28. Reinforcement Learning for Language Models and Reasoning
  • 29. Generalist Agents, Meta-RL, and Open-Endedness
  • 30. The Theory of Reinforcement Learning
  • 31. Reinforcement Learning in the Real World
  • Labs (17)