Colloquium Mathematics - Bin Shi, Chinese Academy of Sciences

Title: Nesterov's accelerated gradient descent and high-resolution differential equation

Abstract:

For modern optimization and machine learning, Nesterov's accelerated gradient descent method is widely used, and revealing its acceleration mechanism is a core issue. In this report, we mainly describe how the theoretical framework of high-resolution differential equations uncover the acceleration phenomenon, and the specific mathematical techniques developed, such as phase space representation and construction of Lyapunov functions, etc. At the same time, we will also describe the latest research progress based on this perspective.