Skip to ContentSkip to Navigation
Research Zernike (ZIAM) Bio-inspired Circuits & Systems Chicca group

Keyword spotting with spiking neural networks using the Time Difference Encoder

Type and duration:

Master/Bachelor project/thesis, flexible duration.

Context:

Humans are able to capture sound and spot words in it thanks to a very important organ in our ear: the cochlea. This element mechanically decomposes the sound into frequencies and communicates to the auditory nerve the amplitude of each frequency (like a fourier transform!) using spikes. However, how to extract meanings from such a transformation is still unclear. One technique is to extract formants which are peaks in the frequency response of sounds caused by resonances in the vocal tract. These peaks are the characteristics that identify vowels, and the two first formants can be enough to disambiguate a vowel [1]. We want to use extracted formants from the cochlea and a layer of Time Difference Encoders (TDEs) [2] to classify the extracted spatio-temporal pattern for keyword spotting at minimum energy cost.

Objectives:

  • Understand the cohclea output encoding and review the different techniques used in the literature for this type of classification problems.
  • Extract the two first formants from the NTIDIGITS cochlea cataset [3] by using Spiking Neural Networks (SNNs) such as a Winner-Takes-All (WTA) on Nengo [4].
  • Add a TDE layer and use the Stimulus Specific Information (SSI) as well as the accuracy to quantify the classification performance.

Required skills:

Python programming, basic machine learning knowledge, strong interest in computational and cognitive neurosciences.

Contact person:

Lyes Khacef

References:

  1. Martin Coath, Sadique Sheik, Elisabetta Chicca, Giacomo Indiver, Susan L. Denham and Thomas Wennekers; A robust sound perception model suitable for neuromorphic implementation, 2014.
  2. M. B. Milde, O. J. N. Bertrand, H. Ramachandran, M. Egelhaaf, E. Chicca; Spiking Elementary Motion Detector in Neuromorphic Systems, Neural Computing, 2018.
  3. N-TIDIGITS Cochlea Spikes Dataset, https://docs.google.com/document/d/1Uxe7GsKKXcy6SlDUX4hoJVAC0-UkH-8kr5UXp0Ndi1M/edit
  4. Nengo, https://www.nengo.ai/
Last modified:25 January 2022 2.11 p.m.