Pavel Izmailov
Contact: pi390@nyu.edu, Twitter
I'm a Research Scientist at OpenAI, working on AI alignment.
Starting in Fall 2024, I will be joining NYU as an Assistant Professor in the Tandon CSE department, and Courant CS department by courtesy.
In my research, I aim to build foundational understanding of models, training procedures, and their limitations. I use this understanding to develop practically impactful, interpretable, robust and broadly applicable methods and models. My interests include interpretability, large-scale models, out-of-distribution generalization, probabilistic deep learning, representation learning, and other topics.
In 2023, I defended my PhD in Computer Science at NYU, under the supervision of Andrew Gordon Wilson. In years 2017–2019 I was a PhD student in Operations Research and Information Engineering at Cornell University, after which I received an MSc degree and transferred to NYU. I received a BSc in applied math and computer science from the faculty of Computational Mathematics and Cybernetics of Lomonosov Moscow State University, where I was working at the Bayesian Methods Research Group under supervision of Dmitry Vetrov and Dmitry Kropotov.
In the summer of 2019 I completed a research internship at Amazon AWS in Palo Alto, working with Bernie Wang and Alex Smola. In the summer of 2020 I worked with Matt Hoffman at Google AI. Between June 2021 and February 2022 I worked with Alex Alemi and Ben Poole at Google as a research intern and a student researcher. In the summer of 2022 I worked with Lucas Beyer and Simon Kornblith at Google Brain.
Our work on Bayesian model selection was recently recognized with an Outstanding Paper Award 🏆 at ICML 2022!
Links
- [Home, Publications, Talks, CV, GitHub, Google Scholar, Semantic Scholar]
Talks
-
Neural network loss surfaces and Bayesian neural nets
Caltech, guest lecture; May, 2023 -
Feature Learning and Distribution Shift
Stanford, Chelsea Finn’s group; December, 2022 -
Understanding Knowledge Distillation
MIT, Tommi Jaakkola’s group; November, 2022 -
Feature Learning and Spurious Correlations
University of Washington, Ludwig Schmidt’s group; November, 2022 -
On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
Google Research, Shannon’s Bandwagon meeting; July, 2022 -
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations
Google Research, Sample Efficient Learning meeting; July, 2022 -
What Are Bayesian Neural Network Posteriors Really Like?
AABI Invited Talk [video] and
Max Plank Institute MIS and UCLA joint Seminar: Math Machine Learning [video] and
Teams at Google Brain and Perception and
Oxford Applied and Theoretical Machine Learning Group and
Bayesgroup seminar and
International Conference on Machine Learning (ICML);
2021 -
Does Knowledge Distillation Really Work?
Teams at Google Brain and Perception; 2021 -
Bayesian Deep Learning and a Probabilistic Perspective of Generalization
University of Freiburg, Frank Hutter's group; 2021 -
How Do We Build Neural Networks We Can Trust?
Broad Institute of MIT and Harvard MIA Seminar; Invited Talk 2019
[video] -
Scalable Bayesian inference in low-dimensional subspaces
Bayesgroup seminar; 2019 -
Subspace Inference for Bayesian Deep Learning
Harvard, Finale Doshi-Velez group; 2019 -
Averaging Weights Leads to Wider Optima and Better Generalization
UAI Oral presentation, 2018
[video] -
Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition
Artificial Intelligence and Statistics (AISTATS) Oral presentation; 2018