Vidya Muthukumar (Georgia Tech): “Distributional robustness under the microscope: Empirical and theoretical case studies in old-school neural network learning”

Abstract:

Distribution shift, i.e. a change in the distribution of input/output data between training and test time, remains an enduring challenge in modern ML. In practice, methods to improve robustness of ML models to distribution shift are frequently heuristic and require considerable side information about the shifted (target) test distribution to be effective. In theory, fundamental questions like why ML models are initially brittle to distribution shifts, how to best mitigate this brittleness, and how much target data is required for this mitigation remain open. In this talk, we put distributional robustness under the microscope by studying the “group robustness” setting – where data is comprised of majority and minority groups whose relative proportions shift between training and test time.

We present two case studies:

On the empirical side, we show that simply retraining the last layer of state-of-the-art neural networks on a class-balanced held-out set is surprisingly effective – even if the held-out set remains sizably group-imbalanced – on well-established group robustness benchmarks. We also present a data-driven, active learning-inspired methodology to select and upweight minority group examples which achieves comparable group robustness on these same benchmarks to state-of-the-art methods that do require group labels.

On the theoretical side, we provide the first end-to-end analysis showing that shallow neural networks learn a simpler spurious correlation and suppress the true (core) predictor. We consider the core predictor to be the nonlinear XOR function while the spurious correlation is linear. Our analysis and techniques reveal a striking phase transition between the situations of “extreme” correlation and “moderate” correlation, and could explain the success of existing robustness-advancing methods as well as inspire new ones.

This is joint work with Tyler LaBonte, John C. Hill, Xinchen Zhang and Abhishek Kumar.

Biography:

Vidya Muthukumar is the Harold R. and Mary Anne Nash Early Career Professor and Assistant Professor in the School of Electrical and Computer Engineering and H. Milton Stewart School of Industrial and Systems Engineering at Georgia Institute of Technology. Dr. Muthukumar’s broad interests are in game theory, online and statistical learning. She is particularly interested in designing learning algorithms that provably adapt in strategic environments, fundamental properties of overparameterized models, and the foundations of multi-agent decision-making.

Dr. Muthukumar received the B.Tech (with honors) degree from the Indian Institute of Technology, Madras and the Ph.D. degree in Electrical Engineering from University of California, Berkeley. She interned at IBM Research in the summer of 2018 as a Science for Social Good fellow. Before joining Georgia Tech, she spent a semester at the Simons Institute for the Theory of Computing as a research fellow for the program “Theory of Reinforcement Learning.” She is the recipient of an Amazon Research Award, NSF CAREER Award, Adobe Data Science Research Award, Simons-Berkeley Google Research Fellowship, and the UC Berkeley EECS Outstanding Course Development and Teaching Award.

Dr. Muthukumar has served/is serving as an Area Chair for COLT (2021-2024), NeurIPS (2023 and 2024), ICML (2025), ALT (2024-2025) and UAI (2024-2025). She also co-organizes several mentorship workshops as part of the Learning Theory Alliance team, and organized the ITALT symposium in 2024.

View the Seminar Recording here!