Entropy differential metric, distance and divergence measures in probability spaces: a unified approach

Abstract

The paper is devoted to metrization of probability spaces through the introduction of a quadratic differential metric in the parameter space of the probability distributions. For this purpose, a φ-entropy functional is defined on the probability space and its Hessian along a direction of the tangent space of the parameter space is taken as the metric. The distance between two probability distributions is computed as the geodesic distance induced by the metric. The paper also deals with three measures of divergence between probability distributions and their interrelationships.