Tutorial 1: Learning by Pruning and the Hunt for Lottery Tickets
Lecturer: Dimitris Papailiopoulos
Title: Learning by Pruning and the Hunt for Lottery Tickets
Abstract: It has been widely observed that highly overparameterized neural networks can be significantly sparsified, with little accuracy loss, through a time-consuming "train, prune, re-train" approach. In 2018, Frankle and Carbin conjectured that we may be able to avoid this by instead training lottery tickets: sparse subnetworks, found right at initialization, trainable to high accuracy. This led to a flurry of algorithmic work attempting to find lottery tickets, and algorithms for pruning at initialization. In this tutorial I will survey these recent findings along with theoretical justifications for the existence of good subnetworks at initialization. I will list several algorithmic challenges for finding lottery tickets and will discuss open problems that fall in the intersection of information theoretic bounds and achievability results.
Bio: Dimitris Papailiopoulos is an Assistant Professor of Electrical and Computer Engineering at the University of Wisconsin-Madison. His interests span machine learning, information theory, and ML systems, with a current focus on efficient large-scale training algorithms. Before UW-Madison, Dimitris was a postdoctoral researcher at UC Berkeley. He earned his Ph.D. in ECE from UT Austin, under the supervision of Alex Dimakis. He received his ECE Diploma and M.Sc. degree from the Technical University of Crete, in Greece. Dimitris is a recipient of the NSF CAREER Award (2019), three Sony Faculty Innovation Awards (2019, 2020, 2021), a joint IEEE ComSoc/ITSoc Best Paper Award (2020), an IEEE Signal Processing Society, Young Author Best Paper Award (2015), the Vilas Associate Award (2021), the IEEE Education Society Mac Van Valkenburg Early Career Teaching Award (2021), the Emil Steiger Distinguished Teaching Award (2021), and the Benjamin Smith Reynolds Award for Excellence in Teaching (2019). In 2018, he co-founded MLSys, a new conference that targets research at the intersection of machine learning and systems.
Tutorial 2: A tutorial on Optimal Transport Theory
Lecturer: Augusto Gerolin
Title: A tutorial on Optimal Transport Theory
Abstract: Optimal transport theory is the branch of mathematics and economics that nowadays is becoming a fundamental tool in Machine Learning research. In simple terms, it aims at studying and generalizing the idea of optimal matchings or assignments between two (or more) groups of observations. This tutorial aims to provide a gentle introduction to mathematical and computational aspects of the theory, motivating its use in Machine Learning with some examples, following with some of the best-known results in the field. Focus will be given to the Entropy-regularization of Optimal Transport, whose solutions can be easily computed via the Sinkhorn algorithm. This tutorial has few prerequisites and definitively no contraindications. Therefore, graduate students are warmly encouraged to attend. During the tutorial, the participants will be stimulated to be active in solving pedagogical problems, which can vary from pen & paper and coding - so bring your laptops with you!
Bio: Augusto Gerolin is a Canada Research Chair in Artificial Intelligence at the Interface of Chemistry and Mathematics and is an Assistant Professor jointly appointed to the uOttawa Departments of Mathematics and Statistics and of Chemistry and Biomolecular Sciences. Previously, he was a Marie Skłodowska-Curie fellow at the Theoretical Chemistry section of the Vrije Universiteit Amsterdam in The Netherlands. He obtained his PhD degree at the University of Pisa in Italy. Gerolin's research interests include Optimal Transport Theory, Mathematical Physics, Theoretical and Computational Chemistry and Mathematical aspects of Machine Learning.