UCPH Statistics Seminar: Luigi Gresele

Speaker: Luigi Gresele from the Max Planck Institute for Intelligent Systems

Title: Learning Identifiable Representations: Independent Influences, Multiple Views, Latent Causal Models

Abstract: Learning systems often receive unstructured information from the external world---e.g., neural networks trained for object recognition take collections of pixels as inputs. Sophisticated signal processing is required to extract meaningful features (such as the position, dimension, and colour of an object) from these inputs: this motivates the field of representation learning. What features should be deemed meaningful? One way to answer is to postulate a data-generating process in the form of a latent-variable model. It is then possible to study the recoverability of the latent variables which generated the observed data---formalized as identifiability. 

I will present three contributions to the field of identifiable representation learning. In the first one, we seek to learn representations where the latent components influence the observations independently. Here, the term “independently” is used in a non-statistical sense—which can be thought of as absence of fine-tuning between distinct elements of a generative process. Next, we study representations which can be learned from paired observations, or views, corresponding to mixtures of (some perturbed version of) the same latent variables. Finally, I will present our recent work on Causal Component Analysis (CauCA), a generalization of Independent Component Analysis (ICA) that models observed variables as mixtures of causally dependent components, with a known graph encoding their causal relationships. I will discuss the implications of these works for representation learning, as well as for probabilistic and causal modelling.