A Robust Hybrid Heavy Ball Algorithm for Optimization with High Performance

Abstract

This paper proposes hybrid control algorithms for optimization of a convex objective function with fast convergence, reduced oscillations, and robustness. Developed using hybrid system tools, the algorithms feature a uniting control strategy, in which two standard heavy ball algorithms, one used globally and another used locally, with properly designed gravity and friction parameters, are employed. The proposed hybrid control strategy switches the parameters to converge quickly to the set of minimizers of the convex objective function without oscillations. A hybrid control algorithm implementing a switching strategy that measures the objective function and its gradient, and another algorithm that only measures its gradient, are designed. Key properties of the resulting closed-loop systems, including existence of solutions, asymptotic stability, and robustness, are analyzed. Numerical results validate the findings.