Chapter 8
Chain Rule Optimal Transport
Frank Nielsen and Ke Sun
Abstract We define a novel class of distances between statistical multivariate distributions by modeling an optimal transport problem on their marginals with respect
to a ground distance defined on their conditionals. These new distances are metrics
whenever the ground distance between the marginals is a metric, generalize both the
Wasserstein distances between discrete measures and a recently introduced metric
distance between statistical mixtures, and provide an upper bound for jointly convex distances between statistical mixtures. By entropic regularization of the optimal
transport, we obtain a fast differentiable Sinkhorn-type distance. We experimentally evaluate our new family of distances by quantifying the upper bounds of several jointly convex distances between statistical mixtures, and by proposing a novel
efficient method to learn Gaussian mixture models (GMMs) by simplifying kernel
density estimators with respect to our distance. Our GMM learning technique experimentally improves significantly over the EM implementation of sklearn on the
MNIST and Fashion MNIST datasets.
Keywords Optimal transport · Wasserstein distances · Information geometry ·
f -divergences · Total Variation and Jensen–Shannon divergence · Bregman
divergence · Rényi divergence · Statistical mixtures · Joint convexity
8.1 Introduction and Motivation
Calculating dissimilarities between statistical mixtures is a fundamental operation
met in statistics, machine learning, signal processing, and information fusion [6]
among others. Minimizing the information-theoretic Kullback–Leibler divergence
F. Nielsen (B)
Sony Computer Science Laboratories, Tokyo, Japan
e-mail: Frank.Nielsen@acm.org
K. Sun
CSIRO’s Data61, Sydney, Australia
e-mail: sunk@ieee.org
© Springer Nature Switzerland AG 2021
F. Nielsen (ed.), Progress in Information Geometry,
Signals and Communication Technology,
https://doi.org/10.1007/978-3-030-65459-7_8
191
Chain Rule Optimal Transport
Frank Nielsen and Ke Sun
Abstract We define a novel class of distances between statistical multivariate distributions by modeling an optimal transport problem on their marginals with respect
to a ground distance defined on their conditionals. These new distances are metrics
whenever the ground distance between the marginals is a metric, generalize both the
Wasserstein distances between discrete measures and a recently introduced metric
distance between statistical mixtures, and provide an upper bound for jointly convex distances between statistical mixtures. By entropic regularization of the optimal
transport, we obtain a fast differentiable Sinkhorn-type distance. We experimentally evaluate our new family of distances by quantifying the upper bounds of several jointly convex distances between statistical mixtures, and by proposing a novel
efficient method to learn Gaussian mixture models (GMMs) by simplifying kernel
density estimators with respect to our distance. Our GMM learning technique experimentally improves significantly over the EM implementation of sklearn on the
MNIST and Fashion MNIST datasets.
Keywords Optimal transport · Wasserstein distances · Information geometry ·
f -divergences · Total Variation and Jensen–Shannon divergence · Bregman
divergence · Rényi divergence · Statistical mixtures · Joint convexity
8.1 Introduction and Motivation
Calculating dissimilarities between statistical mixtures is a fundamental operation
met in statistics, machine learning, signal processing, and information fusion [6]
among others. Minimizing the information-theoretic Kullback–Leibler divergence
F. Nielsen (B)
Sony Computer Science Laboratories, Tokyo, Japan
e-mail: Frank.Nielsen@acm.org
K. Sun
CSIRO’s Data61, Sydney, Australia
e-mail: sunk@ieee.org
© Springer Nature Switzerland AG 2021
F. Nielsen (ed.), Progress in Information Geometry,
Signals and Communication Technology,
https://doi.org/10.1007/978-3-030-65459-7_8
191