Title:On the Rényi Divergence and a Related Problem on the Joint Range of Relative Entropies

Abstract:This paper starts with a study of the minimum of the Rényi divergence, subject to a fixed (or minimal) value of the total variation distance. Relying on the solution of this minimization problem, and using a simple identity which expresses the Rényi divergence as a linear combination of relative entropies, a related problem on the joint range of relative entropies is solved in this work. Its solution provides an exact characterization of this joint range where three probability distributions are involved in the formulation of this problem, subject to a constraint on the total variation distance between two of these distributions. As a byproduct of this characterization, we provide a geometric interpretation of the minimal Chernoff information subject to a minimal total variation distance. Following a recent work by Gil {\em et al.}, new relations between the Rényi divergence and the centered moments of the log-likelihood ratio are explored.