Title:Auxiliary Problem Methods with Varying Core Functions for Large-Scale Structured Convex Problems

Abstract:The auxiliary problem principle (APP), proposed by Cohen (1978, 1980), Cohen and Zhu (1984), aims to find the solution of an optimization problem through a sequence of auxiliary problems. The merits of this approach are two folds. First, the core function is usually separable, which makes the subproblems at each step decomposable and particularly attractive for parallel computing. Second, the choice of the core function is quite flexible. Consequently, by carefully specifying this function, APP may reduce to many standard optimization algorithms. In this paper, we pursue enhancing such flexibility by allowing the core function to be non-identical at each step of the algorithm, and name it varying auxiliary problem principle (VAPP). The convergence of VAPP for convex problem with coupling constraints is proved. Moreover, if this function is specialized to be quadratic, an o(1/k) convergence rate can be established. Interestingly, the new VAPP framework can cover several variants of Jacobian type ADMM as special cases. In the absence of the proximal terms, the condition used in our proof is weaker than that required by other literature for ADMM. Furthermore, our technique works for the convex problem with nonseparable objective and coupled linear constraints, which usually can not be handled by ADMM. Numerical results are presented to illustrate the efficiency of the proposed new framework.