Speech Technology

This course will explain the basics of speech synthesis and recognition. It will briefly touch upon the history of speech recordings and the technology that comes along with speech, and the history of speech recognition and synthesis. You will become familiar with several voice technology applications, such as voice assistants, smart speakers, open-source speech recognizers and synthesizers, among others. The (speech) resources needed for creating speech technology applications will be addressed. You will acquire essential knowledge on data management requirements, licensing and privacy issues. This course will also show you how human and contextual factors affect the interaction between people and speech technology systems. Finally, you get acquainted with models that study the user acceptance of speech technology systems. During the course, you will work on an interesting speech tech project.

In this course, you will learn how to program in Python for voice technology. The code used by voice technology experts needs to be written so that it both achieves the purpose for which it is designed, but also is reusable and has replicable results. You will learn to adjust your code in response to reviews and be encouraged to reuse code of others.

This course is split evenly into two units. The first unit provides the essentials of programming. For example, you'll learn how to work with data organized into lists, dataframes and numpy.ndarrays and apply mathematical operations. The second unit explores the use of Python for data science in general and voice technology in particular. This unit builds on the content in unit 1. For instance, you'll learn how to execute mathematical operations on numpy.ndarrays as well as get first-hand experience using seaborn and matplotlib to visualize your data, data distributions and results. You will also do some hands-on work with speech and language data.

This course provides the fundamentals from phonetics and phonology. Therefore, we will address a few aspects of the phonetics and phonology of English and many other languages, from Aymara to Xhosa. We also cover aspects of anatomy and physiology of the vocal tract and ear, discuss how the International Phonetic Alphabet reflects the diversity of speech sounds, and consider applied issues relating to accented speech, speech perception, speech pathologies, and whispered speech, among other topics. Most importantly, we will leverage theoretical knowledge from phonetics and phonology and relate it to voice technologies.

You will develop a Lab Book in which completed speech analysis and processing assignments are organized. This Lab Book will be a useful resource not only for other courses and your thesis project but for your career in voice technology after the completion of this master's program. Additionally, together with peers, you will work on a group research project relating to speech production / perception analysis and processing.

This course teaches you to design computational models for specific tasks and problems in a data driven manner. One challenge is to ensure that any model you develop is replicable by peers. Although there is no fool-proof method, you'll learn how to reliably validate and adapt your model in a standard way that is widely accepted by data scientists. You'll work with Python to create classical machine learning models and more modern neural network architectures to process tabular data, images, text and, most prominently, sound. These will lay the foundation for the speech synthesis and recognition courses.

Since 1952 when the first speech recognizer Audrey was invented, Automatic Speech Recognition has developed in leaps and bounds. This advancement accelerated particularly after the 2010s when Deep Neural Networks (DNNs) were introduced in speech engineering. Consequently, many commercial products integrate speech recognition and some approach a human recognition level.

This course provides an introduction to such speech recognition technologies. To learn how speech recognition systems work, you'll make your own speech recognizer from scratch! At each step, you will gain experience with technologies in chronological order to achieve a deep understanding of the foundation upon which the state-of-the-art is built. You will simulate the product development process and make an ASR application using your hand-made speech recognizer, which you present in a demonstration session in the final week.

Speech synthesis has come a long way since its beginnings from a niche field with limited interest and high entry requirements. It is now a large field with people of widely varying expertise producing essential components of very successful commercial products. The success of voices like Alexa and Siri build on years of work on speech modelling and parametrization.

In this course you will learn the theoretical and practical foundations of speech synthesis. The course is divided into four units, in each you will be given an assignment and/or a quiz.

This course is dedicated to the design of your Master's thesis. We will focus on research design and experimental protocol. This is a highly interactive course and includes hands-on training, in-class group exercises, and individual reflection to help you pursue your interests in a rigorous, scientific way. To help streamline your educational experience, you will develop three deliverables in this course: 1) A paper based on independent research; 2) A software demonstrator prototype which demonstrates the outcomes of your research; and 3) A scientific poster related to the paper and demonstrator prototype which will be presented in a poster session.

In Speech Recognition II you will deepen your knowledge for practical speech recognition use cases. The course is organized into three units. In the first you will learn about the impact of Deep Neural Networks (DNNs) on the HMM-based framework.You will become (re)acquainted with the DNN and learn about the state-of-the-art speech recognition frameworks. You will also become familiar with speech recognition toolkits and interfaces. The second unit concerns building speech recognition systems for under-resourced languages and/or in multilingual contexts. The final unit concerns speech recognition technologies around you. In that unit, scholars and professionals will present on unique applications. At the end of the course, you will write a term paper and present on it. Through Speech Recognition I and II, you will acquire familiarity with many speech recognizers, will know the challenges that speech scientists face, have ideas of how to improve the speech recognition framework, and may come up with interesting ASR applications.

State of the art systems, based on advanced neural modelling techniques, are bridging the quality and naturalness gap while still offering flexibility and controllability. Such systems are capable of modelling challenging heterogeneous data, i.e. data that contains multiple sources of variation such as speakers and languages, non-ideal recording conditions, expressive and spontaneous speech.

In this course, you will learn how deep neural networks can generate speech from text, the advanced techniques that allow such systems to handle heterogeneous data and to be controllable and how they can be applied in different case scenarios. You will learn how to work with advanced tools for generating speech and consolidate knowledge by designing an experiment which answers a research question or showcases a new product.

The thesis forms the aptitude test for the Speech Technology MSc. In the course Thesis Design (block 3), students have written a research proposal and a related paper with a literature overview, research problem, research questions, appropriate methods for data collection and analysis and a planning for block 4. In this block students elaborate this further, based on the feedback they received from the instructor, and develop it into a thesis. Additionally, students will develop further their demonstrator prototype, modifying it from a proof-of-concept to a more polished demonstrator (it is also permitted that a student starts over with a completely new demonstrator in the event that the prototype from the Thesis Design course fell short of his/her expectations or if the student wants to tackle a different issue for other reasons). This demonstrator should be related to the experiment of the thesis study, or it can also be an application that is built based upon the outcomes of the thesis study.