Detail

Towards an Architecture for Efficient Distributed Search of Multimodal Information

The creation of very large-scale multimedia search engines, with more than one billion images and videos, is a pressing need of digital societies where data is generated by multiple connected devices.
Distributing search indexes in cloud environments is the inevitable solution to deal with the increasing scale of image and video collections.
This presentation will describe how sparse hash inverted indexing can be extended to a distributed setting.
The main goal is to ensure that indexes are uniformly distributed across computing nodes while keeping similar documents on the same nodes.
Load balancing is performed at both node and index level, to guarantee that the retrieval process is not delayed by nodes that have to inspect larger subsets of the index.
Experiments across multiple datasets show that sparse hashes can be used to distribute documents and queries in a balanced and redundant manner across nodes.