Popke Altenburg - Content-Based Refinement of Keyword-Based Image Retrieval

afstudeercolloquium

This project is about keyword-based image search. One of the problems in this area is the retrieval of images that are unrelated to the verbal search query, for example due to noise in the keywords labeled to the images in the database. In this project, we try to improve image retrieval using the visual content of the retrieved images. Keypoints are extracted from the images using the Harris-Laplace keypoint detector. The SIFT keypoint descriptor is used for keypoint description. Based on these keypoints, pairwise similarity matching is applied to all images in the retrieved set to detect pairwise image similarities. For this we compare performance of the methods of `pyramid matching' and the method of `spectral matching and link analysis'. Based on these similarity values, two different ways of improving the image retrieval are applied. The first way is sorting the images based on the similarity values to improve the ranking order. The second way is using the pairwise similarity information to search for a cluster of highly similar images in the set of retrievals, so that images in the cluster can be labeled relevant. Results on artificially assembled image testsets show the problem is hard to solve, but in certain conditions results are positive. For image sorting, as well as cluster searching, `pyramid matching' gave the best results.