Implicit Variational Inference for Uncertainty Estimation: the Parameter and the Predictive Space

Our first contribution is a simple approach to estimating the similarity between two distributions in the predictor space relying solely on samples. This leads to an approximation of the KL divergence based on a kNN estimation, both in the parameter space and in the predictor space. Secondly, we explore two implicit variational methods based on a deep generative network: one in parameter space and the second one directly in predictor space.

Date
  • 16 juin 2020
Heure

15h00 à 16h00

Localisation

En téléprésence

Coûts

Conférence de Yann Pequignot, stagiaire postdoctoral associé au Groupe de recherche en apprentissage automatique de l’Université Laval (GRAAL)

Having access to accurate confidence levels along with the predictions allows conducting risk assessment, and decide whether making a decision is worth the risk. Under the Bayesian paradigm, the posterior distribution over parameters is used to capture model uncertainty, a valuable information that can be translated into predictive uncertainty. However, computing the posterior distribution for high capacity predictors – such as neural networks (NN) – is generally intractable, making approximate methods such as variational inference a promising alternative. While most methods perform inference in the space of parameters, we explore the benefits of carrying inference directly in the space of predictors.

Our first contribution is a simple approach to estimating the similarity between two distributions in the predictor space relying solely on samples. This leads to an approximation of the KL divergence based on a kNN estimation, both in the parameter space and in the predictor space. Secondly, we explore two implicit variational methods based on a deep generative network: one in parameter space and the second one directly in predictor space. Given a prior and a likelihood function, we compare the approximate posterior distributions learned with our two methods to the ones obtained by Hamiltonian Monte Carlo (HMC); both as distributions on the parameter space and as distributions on the predictor space.

Our results show that the variational distributions of predictors learned via an inference in the predictor space are closer to the models obtained by HMC than those trained in the parameter space. We engage in a discussion about the implications of working in the parameter space induced by a NN and the limitations of the commonly used metrics for evaluating the quality of the inferred uncertainty.

Restons en contact!

Vous souhaitez être informé des nouvelles et activités de l'IID? Abonnez-vous dès maintenant à notre infolettre mensuelle.