Mark Schmidt

About

Mark Schmidt is a Professor in the Department of Computer Science at the University of British Columbia. He holds a Canada Research Chair in Large-Scale Machine Learning and is a CIFAR AI Chair at Amii. His research focuses on optimization algorithms for machine learning, including stochastic gradient methods and adaptive step sizes, as well as probabilistic machine learning and applications in computer vision.

Education

• Ph.D. Computer Science, University of British Columbia, 2010
• M.Sc. Computer Science, University of Alberta, 2005

Selected publications

• Flatland: The Adventures of Gradient Descent with Large Step Sizes
• Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
• Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
• Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
• Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
• Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models
• Noise is not the main factor behind the gap between SGD and Adam on transformers, but sign descent might be
• Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
• Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
• Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)