Skip to ContentSkip to Navigation
About usNews and EventsNews articles

Applying dynamic Bayesian networks in transliteration detection and generation

02 December 2011

PhD ceremony: Mr. P. Nabende, 16.15 uur, Aula Academiegebouw, Broerstraat 5, Groningen

Dissertation: Applying dynamic Bayesian networks in transliteration detection and generation

Promotor(s): prof. J.A. Nerbonne

Faculty: Mathematics and Natural Sciences

Transliteration detection and generation are Natural Language Processing (NLP) tasks aimed at improving performance in NLP applications such as Machine Translation. There is a growing body of research on methods that can improve transliteration detection and generation quality. We propose applying two edit distance(ED)-based Dynamic Bayesian Network (DBN) modeling approaches that implement Pair Hidden Markov Models (Pair HMMs) and transduction-based DBN models for computing transliteration similarity.

We explore the effects of several factors represented by the DBN models on transliteration detection and generation quality. We specified and tested several definitions of Pair HMM emission and transition parameters while evaluating different scoring algorithms including the Forward and Viterbi algorithms and their log-odds versions obtained in combination with a random Pair HMM. From the transduction-based DBN modeling approach, we evaluated different generalizations of the basic techniques that define specific types of dependencies which we hypothesize to be important for computing transliteration similarity including edit operations and context dependencies.

We used standard transliteration datasets for eleven language pairs (English-Arabic, English-Bengali, English-Chinese, English-Dutch, English-French, English-German, English-Hindi, English-Kannada, English-Russian, English-Tamil, and English-Thai) to evaluate the performance of the DBN models. Transliteration detection and generation results underscore the importance of representing character context. Ensemble-based applications of the DBN models also resulted in improved transliteration detection quality. The ED-based DBN models considerably improved f-score values for mining English-Hindi and English-Tamil transliterations and posted competitive f-score values for other language pairs compared to best results from state-of-the-art approaches. The use of contextual transformation rules in post-processing steps after applying Pair HMMs also resulted in a large improvement in transliteration generation quality.

Last modified:15 September 2017 3.41 p.m.

More news

  • 17 April 2019

    Why lightning often strikes twice

    In contrast to popular belief, lightning often does strike twice, but the reason why a lightning channel is ‘reused’ has remained a mystery. Now, an international research team led by the University of Groningen has used the LOFAR radio telescope to...

  • 16 April 2019

    Still going strong after four decades

    On March 29th professor of Applied Physics Jeff de Hosson was offered a farewell symposium, a few months after his official retirement date near the close of 2018. ‘But 29 March was the 100th birthday of Jan Francken, my predecessor.’ Besides, De Hosson...

  • 11 April 2019

    Ben Feringa in orbit around the Sun

    Dozens of minor planets that used to orbit the Sun anonymously were named by the International Astronomical Union on 6 April 2019. The asteroid that used to be known as ‘minor planet 12655’ was named after Prof. Ben Feringa, winner of the 2016 Nobel...