Tutorials
|
Deadline: This module covers the topic of geometric deep learning, touching upon all its five G's (Grids, Groups, Graphs, Geodesics, and Gauges) but with a strong focus on group equivariant deep learning. The impact that CNNs made in fields such as computer vision, computational chemistry and physics, can largely be attributed to the fact that convolutions allow for weight sharing, geometric stability, and a dramatic decrease in learnable parameters by leveraging symmetries in data and architecture design. These enabling properties arise from the equivariance property of convolutions. That is, if the input image is translated, the output of a convolution is translated accordingly, which in turn means that local information does not get lost in the neural network upon an input transformation (it is just shifted to a different location). With group equivariant deep learning we can hard-code stability and weight sharing over transformations beyond just translations. E.g., it allows for sharing of weights (representing complex patterns/representations) over poses and symmetries represented by transformations such as translation + rotation + scaling. Tutorials:
Regular Group Convolutions
GDL - Steerable CNNs
Equivariant graph tutorial
Equivariant graph tutorial (without solutions)
Geometric Latent Spaces Tutorial
Lecture recordings: |
|
|
Deadline: Usually we train neural networks (NNs) by way of point estimation. After running gradient descent, each weight is represented by a particular value. In this module, we consider giving NNs a Bayesian treatment. We start by specifying a probability distribution for each weight---p(W)---and the goal of training is to obtain the posterior distribution p(W | D), where D denotes the training data, via Bayes rule. The Bayesian approach provides natural mechanisms for model selection, complexity control, incorporating prior knowledge, uncertainty quantification, and online learning. Yet, Bayesian inference is computationally demanding, and much of the module will focus on scalable methods for estimating the posterior distribution. Tutorials:
Tutorial 1: Bayesian NNs with Pyro (with solutions)
Tutorial 1: Bayesian NNs with Pyro (without solutions)
Tutorial 2: Comparing to Non-Bayesian Methods for Uncertainty (with solutions)
Tutorial 2: Comparing to Non-Bayesian Methods for Uncertainty (without solutions)
Lecture recordings:
Recordings will be added soon.
|
|
|
Deadline: In this module you learn to view data as a byproduct of probabilistic experiments. You will parameterise joint probability distributions over observed random variables and perform parameter estimation by regularised gradient-based maximum likelihood estimation Tutorials:Lecture recordings:
No recordings.
|
|
|
Deadline: In this session you learn to model complex data along with unobserved discrete random variables. Examples you are probably already familiar with include mixture models, factor models, and HMMs. You will learn how to approximate intractable inference using stochastic variational methods and derive efficient gradient estimators for parameter estimation via backpropagation. Tutorials:Lecture recordings:
No recordings.
|
|
|
Deadline: In this session you will learn about techniques to reduce variance in gradient estimation for models with discrete unobserved random variables. We will look into: control variates, Rao-Blackwellisation, continuous relaxations and proxy gradients, sparse projections and mixed random variables. Tutorials:
No documents.
Lecture recordings:
No recordings.
|
|
|
Deadline: This module will focus on the connections between causality and deep learning. We will start by introducing the basic concepts in causality (e.g. causal graphs, d-separations, interventions). Then we will focus on how can DL help causality, specifically for the task of causal discovery (learning the causal graph from data), which will be also the main topic of the tutorial. Finally we will discuss how can causality (or ideas from causality research) help DL and RL, focusing in particular on dealing with distribution shifts. Tutorials:Lecture recordings:
No recordings.
|
|
|
Deadline: In the high performance module, we will investigate how to use scale up your deep learning performance. By the end of this course you will be able to run your projects on (super)computers with large efficiency and minimal human intervention. We will also discuss how to track the performance of your jobs and find possible bottlenecks. The topics include:
|
|
|
Deadline: In this module we will study generative models beyond variational autoencoders: normalizing flows, energy-based models like scored-matching neural networks and diffusion models, Hopfield networks, and so forth. Tutorials:Lecture recordings:
No recordings.
|
|
|
Deadline: In this module we will study the interface and overlap between neural networks, dynamical systems, ordinary/partial/stochastic differential equations, and physics-based neural networks. We will study how and where dynamical systems be found in neural networks with implicit functions and neural ODEs. We will also see how neural networks can be used to model dynamical systems like Navier-Stokes with physics-informed neural networks, as well as with Fourier-inspired architectures and autoregressive neural networks. Tutorials:Lecture recordings:
No recordings.
|
|
|
Deadline: Neural Information Retrieval is the application of deep learning models to ranking tasks, such as ranking documents by their relevance to a given query. After briefly introducing IR basics, this module will cover the three families of models commonly used for this task and their trade-offs. Tutorials:Lecture recordings:
No recordings.
|
|