An Accelerated Linearly Convergent Stochastic L-BFGS Algorithm.
basic_science · Level V
Where this comes from
- Record sourced from PubMed, PMID 30703047.
- Also identified by DOI 10.1109/TNNLS.2019.2891088.
- No licence information is recorded for this record.
- Because redistribution is not established, this page shows the abstract only. Follow the links below for the full text.
Abstract
The limited memory version of the Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm is the most popular quasi-Newton algorithm in machine learning and optimization. Recently, it was shown that the stochastic L-BFGS (sL-BFGS) algorithm with the variance-reduced stochastic gradient converges linearly. In this paper, we propose a new sL-BFGS algorithm by importing a proper momentum. We prove an accelerated linear convergence rate under mild conditions. The experimental results on different data sets also verify this acceleration advantage.