PhD Defence Margherita Lazzaretto

Titel:  Exploiting invariance to learn under distribution shifts

Abstract: 

The concept of invariance plays a central role in robust statistics, serving as a modeling principle to isolate essential structure from variation induced by interventions, environmental heterogeneity, or time. This thesis investigates the advantages of leveraging structural invariance to guide learning in evolving environments. The unifying throughout this work is that the presence of invariance supports robust inference, facilitates dimension reduction, and enables adaptation.

 Chapter 1 elaborates on these ideas and places them within the literature on prediction under distribution shifts. In particular, it highlights strengths and limitations of invariance-based approaches for distribution generalization, thereby motivating the search for forms of invariance that remain useful for adaptation to new environments. It then extends this perspective to online learning and contextual bandits, and discusses limitations of the works presented in this thesis together with future research directions.
  Chapter 2 introduces a framework that separates invariant and time-varying effects in the conditional mean of a response in a non-stationary linear system. The framework, called invariant subspace decomposition (ISD), provides a unified approach to both zero shot prediction and time adaptation. We introduce a practical estimation procedure for the proposed decomposition, establish finite-sample guarantees, and show empirically that exploiting invariance can improve performance over methods that rely only on recent observations or that use invariance without adaptation.

Chapter 3 extends this perspective to stochastic non-stationary contextual bandits. Building on the ISD framework, it introduces an algorithm, ISD-linUCB, which uses offline data to learn an invariant component of the reward model and then performs online
adaptation in a lower-dimensional residual subspace. This decomposition leads to regret bounds that scale with the residual dimension rather than the full problem dimension. We complement the theoretical analysis with simulation experiments showing that exploiting invariance can substantially improve online performance, especially in rapidly changing environments and when sufficient offline data is available.

The thesis will be announced later.

Supervisor: 

Professor Niels Richard Hansen, University of Copenhagen

Professor Jonas Peters ETH Zürich

Associate Professor Niklas Pfister, University of Copenhagen (now Lakera)

Assessment Committee:

Professor Line Clemmensen (chair), University of Copenhagen
Associate Professor Niki Kilbertus, Technical University of Munich
Associate Professor Søren Wengel Mogensen, Copenhagen Business School