Maurice Mulder - Improving online image queries with image classification

afstudeercolloquium

Kalooga is a small internet company that specializes in providing photo galleries that are relevant to search queries or articles. These photo galleries are gathered from many publishers and social networks on the web. To provide appropriate images to the appropriate queries and articles, Kalooga need a measure of their relevance. A gallery's relevance to a user's query is completely deduced from text surrounding that image on the webpage. A problem with this information is that it may not always be present, and some images are left poorly or ambiguously described. However, there is still a source of information about the image left untapped: the image itself! This study tries to help Kalooga harness the information hidden between the pixels themselves. A range of methods from the fields of Object Recognition, Image Processing, Computer Vision and Machine Learning have been explored. These methods may improve the visual relevance of the images Kalooga serves and have been united in a single system. To measure their efficacy, they were tested on scientific datasets as well as a specific image domain created from Kalooga's data concerning soccer. Trying to automatically apply thousands of labels to even more images may be too much to ask. Instead a tool has been developed encompassing an entire image classification pipeline that allows a non-expert to easily experiment and test methods on any set of images. The tool is designed to help Kalooga in finding a solution to specific problems they may face. The system performs well. Although it does not improve on any important benchmarks, the Computer Vision tool can be easily used and extended to provide more functionality. Some visual descriptors that were developed have also been modified and show some interesting results. The system also shows that, although they will never faultlessly label Kalooga's entire database, the implemented methods may yet prove useful for Kalooga by focussing on solving problems their current system cannot handle.