CogniGron Seminar by Emre Neftci (Forschungszentrum Jülich): "Meta-learning for Memristive Devices"

Abstract

Memristive crossbar arrays show promise as non-von Neumann computing technologies, bringing sophisticated neural network processing to the edge and facilitating real-world online learning. However, their deployment for real-world learning problems faces challenges such as non-linearities in conductance updates, variations during operation, fabrication mismatch, conductance drift, and the realities of gradient descent training.

This talk will present methods to pre-train neural networks to be largely insensitive to these non-idealities during learning tasks. These methods rely on a phenomenological model of the device, obtainable experimentally, and bi-level optimization. We showcase this effect through meta-learning and a differentiable model of conductance updates on few-shot learning tasks. Since pre-training is a necessary procedure for any online learning scenario at the edge, our results may pave the way for real-world applications of memristive devices without significant adaptation overhead. Furthermore, by considering the programming of memristive devices as a learning problem in its own right, we demonstrate that the developed methods can accelerate existing write-verify techniques.

About Emre Neftci

Emre Neftci received his MSc degree in physics from EPFL in Switzerland, and his Ph.D. in 2010 at the Institute of Neuroinformatics at the University of Zurich and ETH Zurich. He is currently an institute director at the Jülich Research Centre and Professor at RWTH Aachen. His current research explores the bridges between neuroscience and machine learning, with a focus on the theoretical and computational modeling of learning algorithms that are best suited to neuromorphic hardware and non-von Neumann computing architectures.