An Accelerated Stochastic Trust Region Method for Stochastic Optimization (Journal of Computational Mathematics, 2025)
In this paper, we propose an accelerated stochastic variance reduction gradient methodwith a trust-region-like framework, referred as the NMSVRG-TR method. Based onNMSVRG, we incorporate a Katyusha-like acceleration step into the stochastic trust regionscheme, which improves the convergence rate of the SVRG methods. Under appropriateassumptions, the linear convergence of the algorithm is provided for strongly convex objective functions. Numerical experiment results show that our algorithm is generally superiorto some existing stochastic gradient methods.
Keywords: Stochastic optimization; Stochastic variance reduced gradient; Trust region; Gradient descent method; Machine learning.