Title:On the Rényi Divergence and the Joint Range of Relative Entropies

Abstract:This paper starts with a study of the minimum of the Rényi divergence subject to a fixed (or minimal) value of the total variation distance. Relying on the solution of this minimization problem, we determine the exact region of the points $\bigl( D(Q||P_1), D(Q||P_2) \bigr)$ where $P_1$ and $P_2$ are any probability distributions whose total variation distance is not below a fixed value, and the probability distribution $Q$ is arbitrary (none of these three distributions is assumed to be fixed). It is further shown that all the points of this convex region are attained by a triple of 2-element probability distributions. As a byproduct of this characterization, we provide a geometric interpretation of the minimal Chernoff information subject to a minimal total variation distance.