Skip to ContentSkip to Navigation
Research Zernike (ZIAM) Bio-inspired Circuits & Systems Chicca group

Sound-Source-Localization with the Time-Difference-Encoder

Type and duration:

Master project, duration flexible.

Context:

Mammals such as human beings estimate the position of a sound source in space by combining different auditory cues, namely the Interaural Time Difference, the Interaural Level Difference and the Head Transfer Function. In this project you will extract the Interaural Time Difference which is the temporal delay with which an auditory wave reaches the two ears of a receiver. While traditional approaches use the Jeffress model, an array of coincidence detectors with delay lines, you will use a new type of encoder model, the Time Difference Encoder (TDE, previously called spiking elementary motion detector [sEMD]) [1, 2]. During your project you will model spiking neural networks consisting of Leaky Integrate and Fire (LIF) Neurons and the TDE to translate the sound source position into an auditory map. You will use the RIR generator Library to create a small dataset of sound source positions. You will input the extracted sound files into the Neuromorphic Auditory Sensor (NAS) [3] and further process the output data in spiking neural networks using the simulation platform nengo (NEST is also possible) [4]. The focus of this project lies on the design and characterization of TDE based spiking neural networks for sound source localization. As an extension of your thesis you can implement the working network on the Neuromorphic Pan-Tilt Unit (NEPTUn) [5] and test its performance in a real world scenario.

Objectives:

  • Understand and characterize how the Time Difference Encoder [2] model translates auditory time differences into spike rate.
  • Design and characterize Spiking Neural Networks including the TDE model.
  • Implement and test the model on the Neuromorphic Pan-Tilt Unit (optional)

Required skills:

python programming, strong interest in computational and cognitive neurosciences.

Helpful but optional: FPGA programming and SpiNNaker programming.

Contact person:

Thorben Schoepe

Neuromorphic Pan-Tilt-Unit including Event-Based Camera, Neuromorphic Auditory Sensor with 3DIO Binaural Microphone and SpiNNaker board. The focus of this project can be either on simulation or on using this hardware platform in dependency of your interest.
Neuromorphic Pan-Tilt-Unit including Event-Based Camera, Neuromorphic Auditory Sensor with 3DIO Binaural Microphone and SpiNNaker board. The focus of this project can be either on simulation or on using this hardware platform in dependency of your interest.

References:

  1. Schoepe, T., Gutierrez-Galan, D., Dominguez-Morales, J.P., Jiménez-Fernandez, A., Linares-Barranco, A., & Chicca, E. (2019). Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance. 2019 IEEE Biomedical Circuits and Systems Conference (BioCAS), 1-4.
  2. Milde, Moritz & Bertrand, Olivier & Ramachandran, H. & Egelhaaf, Martin & Chicca, Elisabetta. (2018). Spiking Elementary Motion Detector in Neuromorphic Systems. Neural Computation. 30. 1-34. 10.1162/neco_a_01112.
  3. Jimenez-Fernandez A, Cerezuela-Escudero E, Miro-Amarante L, Dominguez-Moralse MJ, de Asis Gomez-Rodriguez F, Linares-Barranco A, Jimenez-Moreno G. A Binaural Neuromorphic Auditory Sensor for FPGA: A Spike Signal Processing Approach. IEEE Trans Neural Netw Learn Syst. 2017 Apr;28(4):804-818. doi: 10.1109/TNNLS.2016.2583223. Epub 2016 Jul 27. PMID: 27479979.
  4. Bekolay, T., Bergstra, J., Hunsberger, E., Dewolf, T., Stewart, T. C., Rasmussen, D., Choo, X., Voelker, A. R., & Eliasmith, C. (2014). Nengo: a Python tool for building large-scale functional brain models. Frontiers in neuroinformatics, 7, 48. https://doi.org/10.3389/fninf.2013.00048
  5. T. Schoepe, D. Gutierrez-Galan, J. P. Dominguez-Morales, A. Jimenez-Fernandez, A. Linares-Barranco and E. Chicca, "Live Demonstration: Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance," 2020 IEEE International Symposium on Circuits and Systems (ISCAS), Seville, Spain, 2020, pp. 1-1, doi: 10.1109/ISCAS45731.2020.9181257.
Last modified:22 September 2022 12.22 p.m.