Learning the time constants of the Time Difference Encoder for spatio-temporal pattern recognition

Type and duration:

Master project/thesis, flexible duration.

Context:

Humans are able to capture sound and spot words in it thanks to a very important organ in our ear: the cochlea. This element mechanically decomposes the sound into frequencies and communicates to the auditory nerve the amplitude of each frequency (like a fourier transform!) using spikes. However, how to extract meanings from such a transformation is still unclear. One possibility is that our brain uses the delays between different frequencies to decode the temporal pattern of the sound. For this purpose, we use the Time Difference Encoder (TDE) [1] to extract the delays between frequencies, then use a Spiking Neural Network (SNN) to classify the extracted spatio-temporal pattern. However, tuning the time constants of the TDE is difficult and time consuming. In this project, we want to learn these parameters using a recently proposed technique for learning membrane time constants in SNNs [2].

Objectives:

• Understand the impact of the TDE’s time constants based on the synthetic data as well as the N-TIDIGITS [3] dataset of spoken digits.
• Experiment in the SpikingJelly [4] simulator the training algorithm proposed in [2] that is capable to learn both the synaptic weights and the membrane time constants of a SNN.
• Add a TDE layer and apply the same training algorithm to learn the synaptic time constants of the TDEs.

Required skills:

Python programming, optimization and machine learning, strong interest in computational and cognitive neurosciences.

Contact person:

Hugh Greatorex

References:

1. M. B. Milde, O. J. N. Bertrand, H. Ramachandran, M. Egelhaaf, E. Chicca; Spiking Elementary Motion Detector in Neuromorphic Systems, Neural Computing, 2018.
2. Wei Fang, Zhaofei Yu, Yanqi Chen, Timothee Masquelier, Tiejun Huang, Yonghong Tian; Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks, 2020.
3. N-TIDIGITS Cochlea Spikes Dataset, https://docs.google.com/document/d/1Uxe7GsKKXcy6SlDUX4hoJVAC0-UkH-8kr5UXp0Ndi1M/edit
4. SpikingJelly, https://github.com/fangwei123456/spikingjelly