Hybrid control systems perspective to gradient based optimization

Abstract

Ordinary differential equations, and in general a dynamical system viewpoint, have seen a resurgence of interest in developing fast optimization methods, mainly thanks to
the availability of well-established analysis tools. In this talk, I will provide an overview of fast gradient-based algorithms and recent results from a dynamical systems perspective. I will then describe a hybrid control framework to design a class of fast gradient-based methods in continuous-time that, in comparison with the existing literature including Nesterov’s fast-gradient method, features a state-dependent, time-invariant damping term that acts as a feedback control input. The proposed design scheme allows for a user-defined, exponential rate of convergence for a class of nonconvex, unconstrained optimization problems. Finally, I will introduce a discretization method such that the resulting discrete dynamical system possesses an exponential rate of convergence.