CogniGron Seminar - Eleni Vasilaki

Title: Sparse Reservoir Computing (SpaRCe) for neuromorphic devices

Abstract:

I will present an overview of my research followed up by fundamental ideas about biological learning in frutflies, and how these are related to Machine Learning. Inspired by the architecture of small brains, and within the framework of Ecco State Networks, I will discuss the importance of neuron selectivity to specific stimuli. I will then introduce a threshold per reservoir neuron as an efficient mechanism to achieve sparseness in the neuronal representation. The threshold is adapted via a gradient rule on an error function structurally identical to threshold learning via backpropagation.  And yet, a simple mathematical analysis of its consequences for the specific architecture shows that it “removes” correlated neuronal input from the representation leading to neuronal selectivity. I will show in simulations that, within this context, our approach is advantageous in terms of performance versus imposing sparseness of weights via L1 norm and alleviates the problem of catastrophic forgetting. I will briefly discuss how such learning architectures can be exploited in the context of neuromorphic engineering.