Mathematics > Optimization and Control
[Submitted on 14 Dec 2015 (v1), revised 26 Dec 2015 (this version, v2), latest version 16 Jan 2017 (v4)]
Title:Auxiliary Problem Methods with Varying Core Functions for Large-Scale Structured Convex Problems
View PDFAbstract:The auxiliary problem principle (APP), proposed by Cohen (1978, 1980), Cohen and Zhu (1984), aims to find the solution of an optimization problem through a sequence of auxiliary problems. The merits of this approach are two folds. First, the core function is usually separable, which makes the subproblems at each step decomposable and particularly attractive for parallel computing. Second, the choice of the core function is quite flexible. Consequently, by carefully specifying this function, APP may reduce to many basic optimization algorithms. In this paper, we pursue enhancing such flexibility by allowing the core function to be non-identical at each step of the algorithm, and name it varying auxiliary problem principle (VAPP). The convergence of VAPP for convex problem with coupling constraints is proved if the "difference" of the core functions associated with the two consecutive steps is bounded. Moreover, if this function is specialized to be quadratic, an o(1/k) convergence rate can be established. Interestingly, the new VAPP framework can cover several variants of Jacobian type ADMM as special cases. In the absence of the proximal terms, the condition used in our proof is weaker than that required by other literature for ADMM. Furthermore, our technique works for the convex problem with nonseparable objective and coupled linear constraints, which usually can not be handled by ADMM. Numerical results are presented to illustrate the efficiency of the proposed new framework.
Submission history
From: Lei Zhao [view email][v1] Mon, 14 Dec 2015 05:29:35 UTC (118 KB)
[v2] Sat, 26 Dec 2015 14:39:19 UTC (118 KB)
[v3] Tue, 5 Apr 2016 10:54:08 UTC (94 KB)
[v4] Mon, 16 Jan 2017 02:11:08 UTC (67 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.