Computer Science > Machine Learning
[Submitted on 24 Jan 2012 (this version), latest version 26 Jul 2013 (v5)]
Title:An Efficient Primal-Dual Prox Method for Non-Smooth Optimization
View PDFAbstract:We consider the non-smooth optimization problems in machine learning, where both the loss function and the regularizer are non-smooth functions. Previous studies on efficient empirical loss minimization assume either a smooth loss function or a strongly convex regularizer, making them unsuitable for non-smooth optimization. We develop an efficient method for a family of non-smooth optimization where the dual form of the loss function is bilinear in primal and dual variables. We cast a non-smooth optimization problem into a minimax optimization problem, and develop a primal dual prox method that solves the minimax optimization problem at a rate of $O(1/T)$, significantly faster than a standard gradient descent method ($O(1/\sqrt{T})$). Our empirical study verifies the efficiency of the proposed method for non-smooth optimization by comparing it to the state-of-the-art first order methods.
Submission history
From: Tianbao Yang [view email][v1] Tue, 24 Jan 2012 04:09:54 UTC (75 KB)
[v2] Fri, 27 Jan 2012 17:50:21 UTC (75 KB)
[v3] Fri, 10 Feb 2012 16:11:15 UTC (79 KB)
[v4] Mon, 2 Apr 2012 15:50:38 UTC (124 KB)
[v5] Fri, 26 Jul 2013 05:03:51 UTC (130 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
IArxiv Recommender
(What is IArxiv?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.