Blackthorn Pruning: High-Performance Interactive Multimodal Learning with Approximate High-Dimensional Indexing

Abstract

As the size of multimedia collections grows, so does the need for efficient and scalable search and exploration methods. In this thesis we present Blackthorn Pruning, a scalable, interactive multimodal learning approach that facilitates interactive analysis of vast multimedia collections. Blackthorn Pruning is created by combining Blackthorn, a state-of-the-art interactive multimodal learning approach, and eCP, a scalable, approximate high-dimensional indexing method. By pruning Blackthorn's search space, eCP reduces the number of data items scored by Blackthorn in each interaction round, leading to reduced time per interaction round, while maintaining the relevance of the items suggested. Experiments on the YFCC100M dataset, which consists of nearly 100 million images and metadata, show that compared to original Blackthorn, Blackthorn Pruning takes 14 times less time using 16 times less computational power. Experiments on a simulated collection of 1 billion images also further suggest the scalability potential of Blackthorn Pruning. Our proposed approach thus opens up interesting avenues for analytics on truly Web-scale collections and also unlocks the potential for such analytics to be performed on modest hardware configurations commonly found in consumer PCs and mobile devices.