Horizon CDT Research Highlights

Research Highlights

The Study of Semantic Image Retrieval and Automatic Image Annotation

  Ke Sun (2014 cohort)   www.nottingham.ac.uk/~psxks7

1. Research Aim

My research interests are Computer Vision and Machine Learning, with an emphasis on Image Retrieval (IR) and Automatic Image Annotation (AIA). My research aims to build high-efficiency Image Retrieval systems for large-scale datasets, and bridge the gap between low-level image features and high-level semantic content of images.

2. Introduction

Compared with text information, digital images are more vivid and impressive, and have been widely used in people’s daily lives and industrial fields. In recent years, the fast popularization of electronic devices (digital cameras, smart phones, tablets, scanners, etc.) greatly helps generating thousands of digital images in every minute. For example, the famous social media website Facebook reveals we upload an astonishing 350 million photos to the network daily [1]. Although the huge amount of images brings us rich information, it also poses great challenges for searching: how could people quickly and correctly find expected images from massive image datasets? This is also the problem that Modern image retrieval techniques are trying to solve: keeping a high-precision search result while decreasing the searching time.

3. Research Gap and Motivation

Many contributions have been made to bridge the semantic gap for image retrieval in various directions, there however exists gap for designing and implementation of practical image retrieval systems:

A example of semantic gap problem: computers cannot distinguish a red flower from rising sun in terms of colour and texture

A example of semantic gap problem: computers cannot distinguish a red flower from rising sun in terms of colour and texture

(1) Personalisation of image search

In many real world applications, users do not clearly know what they want. They often just hold a general idea to explore some relevant images. As a result, building a recommendation system based on the user query is necessary.

(2) Curse of dimensionality

High dimensionality is one of the important challenges in image retrieval systems due to possible size of feature space and images.

(3) Limitation of automatic image annotation

Automatic image annotation is an effective approach the bridge the semantic gap for image retrieval, but it also has limitations in terms of potential relationships between concepts.

4. Application Fields

(1) Digital library

Digital libraries contain abundant image, audio and video resources, how to quickly and accurately find expected media information is the major problem need to be solved for building digital libraries.

(2) Intellectual property

Many intellectual properties appear as images, for example, trademarks and artwork. In order to avoid torts, newly proposed trademarks need to be compared with existing trademarks in the database to ensure no similar matches are found.

(3) Medical diagnoses

A medical image retrieval system allows doctors to access medical image database, search historical images to help diagnose illness for new patients.

(4) Tourism and shopping

With the popularisation of camera phones, many people want to obtain more information according to the photos they take by their mobile phones, and this could be solved with the help of IR techniques.

(5) Crime investigation

Some public security systems have face and fingerprint image databases used for crime investigation. By retrieving similar fingerprint or face images using query images that are collected from the crime scene, polices can quickly target the suspect.

References

[1] Jam Kotenko. (2013). Facebook reveals we upload a whopping 350 million photos to the network daily. Available: http://www.digitaltrends.com/social-media/according-to-facebook-there-are-350-million-photos-uploaded-on-the-social-network-daily-and-thats-just-crazy/. Digital Trends. Last accessed on 1 Mar 2015.

This work was carried out at the International Doctoral Innovation Centre (IDIC). The authors acknowledge the financial support from Ningbo Education Bureau, Ningbo Science and Technology Bureau, China's MOST, and the University of Nottingham. The work is also partially supported by EPSRC grant no EP/L015463/1.