CompMath Seminar - E. Loevbak, University of Leuven

Title: Algorithmic differentiation and adjoints for stochastic solvers

Abstract:

In most scientific and engineering domains, one encounters optimization problems where the objective function contains the solution of a mathematical model. Due to model complexities, such as high-dimensionality or complex boundary conditions, there are various cases where a stochastic solver becomes the preferred simulation approach. As an example, we consider PDE constrained optimization, where the PDE is simulated using a particle method.

Computing gradients is an interesting challenge in this stochastic optimization setting. The classic cancellation-error pitfalls of finite differences become exacerbated by the simulation noise if applied naively and are generally expensive. We therefore consider two alternative approaches: algorithmic differentiation and adjoint simulations. In both approaches, one is able to compute the gradient through similar simulations to those used to evaluate the objective function. In this talk, we introduce both of these approaches and compare them. In doing so we highlight the close link between the adjoint approach and reverse-mode algorithmic differentiation. We then discuss the particularities that arise when applying these approaches to stochastic solvers. In particular, we address the challenges caused by needing the same random numbers used in the simulation for the objective function, but in reversed order.