Multimodal unsupervised learning with self-organizing spiking neural networks
Type and duration:
Master project/thesis, flexible duration.
Cortical plasticity is one of the main features that enable our capability to learn and adapt in our environment. Indeed, the cerebral cortex has the ability to self-organize itself through two distinct forms of plasticity: the structural plasticity and the synaptic plasticity. These mechanisms are very likely at the basis of an extremely interesting characteristic of the human brain development: multimodal association. In fact, most processes and phenomena in the natural environment are expressed under different physical guises, which we refer to as different modalities. The brain uses spatio-temporal correlations between several modalities to structure the data and create sense from observations. Moreover, biological observations show that one modality can activate the internal representation of another modality when both are correlated. We have proposed a computational model for multimodal association based on Self-Organizing Maps (SOMs) , and successfully applied it to different multimodal classification tasks. The objective of this project is to replace the SOM by a Spiking Neural Network (SNN) which better fits the spatio-temporal sparsity of the event-based sensors used in neuromorphic computing, toward more energy-efficient implementations.
- Literature review of the previous works for unsupervised learning  and multimodal association  using SNNs;
- Improve the current model for multimodal (handwritten + spoken) digits classification on the BindsNET  simulator;
- Explore different synaptic plasticity rules for afferent and lateral learning.
Python programming, basic machine learning knowledge, strong interest in computational and cognitive neurosciences.
- Khacef, L.; Rodriguez, L.; Miramond, B. Brain-Inspired Self-Organization with Cellular Neuromorphic Computing for Multimodal Unsupervised Learning. Electronics, 2020.
- Diehl, P.; Cook, M. Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front. Comput. Neurosci. 2015.
- Rathi, N.; Roy, K. STDP-Based Unsupervised Multimodal Learning With Cross-Modal Processing in Spiking Neural Network. IEEE Trans. Emerg. Top. Comput. Intell. 2018.
- BindsNET, https://github.com/BindsNET/bindsnet
|Last modified:||16 April 2021 12.47 p.m.|