Bags of Local Convolutional Features for Scalable Instance Search

	Best Poster Award at the ACM International Conference on Multimedia Retrieval (ICMR) 2016

Eva Mohedano	Amaia Salvador	Kevin McGuinness	Xavier Giro-i-Nieto	Noel O'Connor	Ferran Marques

A joint collaboration between:

Insight Centre for Data Analytics	Dublin City University (DCU)	Universitat Politecnica de Catalunya (UPC)	UPC ETSETB TelecomBCN	UPC Image Processing Group

Abstract

This work proposes a simple instance retrieval pipeline based on encoding the convolutional features of CNN using the bag of words aggregation scheme (BoW). Assigning each local array of activations in a convolutional layer to a visual word produces an assignment map, a compact representation that relates regions of an image with a visual word. We use the assignment map for fast spatial reranking, obtaining object localizations that are used for query expansion. We demonstrate the suitability of the BoW representation based on local CNN features for instance retrieval, achieving competitive performance on the Oxford and Paris buildings benchmarks. We show that our proposed system for CNN feature aggregation with BoW outperforms state-of-the-art techniques using sum pooling at a subset of the challenging TRECVid INS benchmark.

Publication

Find our paper at ACM Digital Library, arXiv and DCU Doras.

Please cite with the following Bibtex code:

@inproceedings{Mohedano:2016:BLC:2911996.2912061,
 author = {Mohedano, Eva and McGuinness, Kevin and O'Connor, Noel E. and Salvador, Amaia and Marques, Ferran and Giro-i-Nieto, Xavier},
 title = {Bags of Local Convolutional Features for Scalable Instance Search},
 booktitle = {Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval},
 series = {ICMR '16},
 year = {2016},
 isbn = {978-1-4503-4359-6},
 location = {New York, New York, USA},
 pages = {327--331},
 numpages = {5},
 url = {http://doi.acm.org/10.1145/2911996.2912061},
 doi = {10.1145/2911996.2912061},
 acmid = {2912061},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {bag of words, convolutional neural networks, instance retrieval},
} 

Best Poster Award at ICMR 2016

Talk on video

This talk also covers our paper "Faster R-CNN features for Instance Search" at CVPR 2016 Workshop on DeepVision.

Slides

These slides also cover our paper "Faster R-CNN features for Instance Search" at CVPR 2016 Workshop on DeepVision.

Related Work

• Mohedano, E., McGuinness, K., Giro-i-Nieto, X., & O'Connor, N. E. (2017). "Saliency Weighted Convolutional Features for Instance Search." arXiv preprint arXiv:1711.10795.
• Albert Jimenez, Jose M. Alvarez, and Xavier Giro-i-Nieto. “Class-Weighted Convolutional Features for Visual Instance Search.” In Proceedings of the 28th British Machine Vision Conference (BMVC). 2017.
• Reyes, C., Mohedano, E., McGuinness, K., O'Connor, N. E., & Giro-i-Nieto, X. (2016, October). Where is my phone?: Personal object retrieval from egocentric images. In Proceedings of the first Workshop on Lifelogging Tools and Applications (pp. 55-62). ACM.
• Amaia Salvador, Xavier Giro-i-Nieto, Ferran Marques and Shin'ichi Satoh. "Faster R-CNN Features for Instance Search." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2016.