Books and documentation
The chapter notes develop the module’s main material. These books and official documentation provide alternative explanations, fuller derivations, and the details of the library interfaces used in the code. The reading plan lists the lecture slides and readings for each topic.
Main references
| Source | Relevant material |
|---|---|
| Prince, Understanding Deep Learning (MIT Press, 2023) | The primary textbook: chapters 3–4 (chapter 1), 6–7 (chapters 2–3), 8–9 (chapter 5), 10 (chapter 6), 11 (chapters 4 and 7), 12 (chapter 9), 13 (chapter 13), 14 (chapter 10), and 20 (chapters 5 and 15). The free PDF and Python notebooks are linked from the book page. |
| Goodfellow, Bengio, and Courville, Deep Learning (MIT Press, 2016) | Chapters 6 (chapter 1), 7 (chapter 5), 8 (chapters 2–4), 9 (chapter 6), 10 (chapter 8), 11 (chapter 12), and 14–15 (chapter 10). Older, but thorough on the classical material; free HTML chapters. |
| Fleuret, The Little Book of Deep Learning | A compact summary of components, architectures, training, and efficient computation that accompanies the UNIGE course. |
| Zhang, Lipton, Li, and Smola, Dive into Deep Learning | An interactive book with runnable PyTorch code for every model: optimization, convolutional and recurrent networks, attention and transformers, computational performance, and computer vision. |
Specialized books and surveys
| Source | Relevant material |
|---|---|
| Bronstein, Bruna, Cohen, and Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges | Architectures derived from the symmetries of their domain: convolutional networks, transformers, and graph networks in one framework (chapters 6, 9, and 13). |
| Hamilton, Graph Representation Learning | Graph neural networks, spectral methods, and the Weisfeiler–Lehman connection (chapter 13). |
| Roberts, Yaida, and Hanin, The Principles of Deep Learning Theory | Signal propagation at initialization, the infinite-width limit, and its \(1/n\) corrections (chapters 2 and 15). |
| Balestriero et al., A Cookbook of Self-Supervised Learning | The families of self-supervised methods and the training details that make them work (chapter 10). |
| Phuong and Hutter, Formal Algorithms for Transformers | Precise pseudocode for every transformer variant (chapter 9). |
| Nagel et al., A White Paper on Neural Network Quantization | Quantization schemes, post-training quantization, and quantization-aware training (chapter 11). |
Notes and guides
| Source | Relevant material |
|---|---|
| Horace He, Making Deep Learning Go Brrrr From First Principles | Compute, memory bandwidth, and overhead as the three costs of a GPU program (chapter 11). |
| Olah, Understanding LSTM Networks | Illustrated LSTM and GRU gates (chapter 8). |
| Rush et al., The Annotated Transformer and Karpathy, nanoGPT | Compact, complete transformer implementations (chapter 9). |
| Weng, Contrastive Representation Learning and Some Math behind Neural Tangent Kernel | Surveys of contrastive objectives (chapter 10) and derivations of the NTK results (chapter 15). |
| Dumoulin and Visin, A Guide to Convolution Arithmetic for Deep Learning | Output sizes and illustrations for padding, stride, dilation, and transposed convolutions (chapters 6 and 14). |
| Odena, Dumoulin, and Olah, Deconvolution and Checkerboard Artifacts | Why transposed convolutions produce checkerboard patterns (chapter 14). |
Library documentation
| Source | Relevant material |
|---|---|
| PyTorch documentation | torch.nn layers and initialization, autograd, torch.optim and learning-rate schedulers, torch.nn.functional.scaled_dot_product_attention, and torch.utils.checkpoint. |
PyTorch automatic mixed precision and torch.compile | torch.autocast, gradient scaling, and compilation with kernel fusion (chapter 11). |
| PyTorch distributed overview | Distributed data parallelism, fully sharded data parallelism, and tensor parallelism (chapter 11). |
| PyTorch reproducibility notes | Seeds, deterministic algorithms, and sources of nondeterminism (chapter 12). |
| NetworkX documentation | Graph construction, random graph models, and the karate-club graph used in chapter 13. |
| scikit-learn datasets | The bundled digits dataset used throughout, and load_sample_image (chapter 9). |