Title:A Generalized Tangent Approximation Framework for Strongly Super-Gaussian Likelihoods

Abstract:Tangent approximation form a popular class of variational inference (VI) techniques for Bayesian analysis in intractable non-conjugate models. It is based on the principle of convex duality to construct a minorant of the marginal likelihood, making the problem tractable. Despite its extensive applications, a general methodology for tangent approximation encompassing a large class of likelihoods beyond logit models with provable optimality guarantees is still elusive. In this article, we propose a general Tangent Approximation based Variational InferencE (TAVIE) framework for strongly super-Gaussian (SSG) likelihood functions which includes a broad class of flexible probability models. Specifically, TAVIE obtains a quadratic lower bound of the corresponding log-likelihood, thus inducing conjugacy with Gaussian priors over the model parameters. Under mild assumptions on the data-generating process, we demonstrate the optimality of our proposed methodology in the fractional likelihood setup. Furthermore, we illustrate the empirical performance of TAVIE through extensive simulations and an application on the U.S. 2000 Census real data.