Statistical physics of learning vector quantization

PhD ceremony: Mr. A.W. Witoelar, 11.00 uur, Academiegebouw, Broerstraat 5, Groningen

Thesis: Statistical physics of learning vector quantization

Promotor(s): prof. M. Biehl, prof. N. Petkov

Faculty: Mathematics and Natural Sciences

The field of machine learning concerns the design of algorithms to learn and recognize complex patterns from data. Learning Vector Quantization (LVQ) constitutes an important family of such algorithms using prototypes which serve as typical examples. Despite its wide range of applications, the theoretical understanding of LVQ in general remains very limited. In this thesis a theoretical framework is constructed using concepts from statistical physics which allows for an exact analysis of the typical learning behaviors of the system. Studies in this thesis compare the characteristics of various LVQ algorithms to demonstrate the robustness of Neural Gas (NG) schemes and the specific advantages of LVQ 2.1 and Robust Soft LVQ algorithms. Furthermore surprising non-trivial behaviors are revealed including learning plateaus and phase transitions in the training process. The results provide insights to general prototype-based learning prescriptions.