Shanghua Teng, USC

Title: Understanding and Characterizing Regularization: Learnability and Physics-Based Energy Guidance

Abstract: The quintessential learning algorithm of empirical risk minimization (ERM) is known to fail in various settings for which uniform convergence does not characterize learning. Relatedly, the practice of machine learning is rife with considerably richer algorithmic techniques, perhaps the most notable of which is regularization. Nevertheless, no such technique or principle has broken away from the pack to characterize optimal learning in these more general settings. The purpose of this research direction is to understand the role of regularization in data-driven machine learning.

First, we focus on an abstract statistical learning framework, present our work on characterizing the power of regularization in perhaps the simplest setting for which ERM fails: multiclass learning with arbitrary label sets. Using one-inclusion graphs (OIGs), we exhibit a local-regularization approach to obtain optimal learning algorithms that dovetail with tried-and-true algorithmic principles: Occam’s Razor as embodied by structural risk minimization (SRM), the principle of maximum entropy, and Bayesian inference.

Second, we share some of our on-going progress on designing physics-guided energy regularization for data-driven learning of the weak solution to parabolic PDEs. The goal here is to extend semi-supervised learning by exploiting auxiliary data and the underlying physical model to construct stronger regularization, enabling more efficient learning with optimal estimation and faster generalization.

Joint work (COLT 2024) with Julian Asilis, Siddartha Devic, Shaddin Dughmi, and Vatsal Sharan

Joint work with Xiaohui Chen and Zixiang Zhou.