Title:First-Order Primal-Dual Augmented Lagrangian Method for Nonlinear Cone Constrained Composite Convex Optimization

Abstract:Nonlinear convex cone programs (NCCP) are important problems with many practical applications, but these problems are hard to solve by using existing methods. This paper introduces a flexible first-order primal-dual augmented Lagrangian decomposition algorithm Varying Auxiliary Problem Principle (VAPP) for solving NCCP (objective or constraints are nonseparable, nonsmooth possible). The iterative scheme of VAPP embeds linearization and regularization into each iteration of the augmented Lagrangian method (ALM). In each iteration, a nonlinear approximation, which incorporates linearization and a variable Bregman-like function (or auxiliary core function), is used to approximate the primal problem of augmented Lagrangian method. In this way, the primal problem can be decomposed into smaller subproblems, each of which has a closed-form solution or an easily approximated solution. Moreover, these subproblems can be solved in a parallel way. This paper proves convergence and an $O(1/t)$ convergence rate in average for the primal suboptimality, feasibility and dual suboptimality. Moreover, a backtracking scheme is discussed to treat the case where the Lipschitz constants are not known or computable.