Mining Mid-level Visual Patterns with Deep CNN Activations

详细信息    查看全文

• 作者：Yao Li ; Lingqiao Liu ; Chunhua Shen…
• 关键词：Mid ; level visual element discovery ; Pattern mining ; Convolutional neural networks
• 刊名：International Journal of Computer Vision
• 出版年：2017
• 出版时间：February 2017
• 年：2017
• 卷：121
• 期：3
• 页码：344-364
• 全文大小：
• 刊物类别：Computer Science
• 刊物主题：Computer Imaging, Vision, Pattern Recognition and Graphics; Artificial Intelligence (incl. Robotics); Image Processing and Computer Vision; Pattern Recognition;
• 出版者：Springer US
• ISSN：1573-1405
• 卷排序：121

文摘

The purpose of mid-level visual element discovery is to find clusters of image patches that are representative of, and which discriminate between, the contents of the relevant images. Here we propose a pattern-mining approach to the problem of identifying mid-level elements within images, motivated by the observation that such techniques have been very effective, and efficient, in achieving similar goals when applied to other data types. We show that Convolutional Neural Network (CNN) activations extracted from image patches typical possess two appealing properties that enable seamless integration with pattern mining techniques. The marriage between CNN activations and a pattern mining technique leads to fast and effective discovery of representative and discriminative patterns from a huge number of image patches, from which mid-level elements are retrieved. Given the patterns and retrieved mid-level visual elements, we propose two methods to generate image feature representations. The first encoding method uses the patterns as codewords in a dictionary in a manner similar to the Bag-of-Visual-Words model. We thus label this a Bag-of-Patterns representation. The second relies on mid-level visual elements to construct a Bag-of-Elements representation. We evaluate the two encoding methods on object and scene classification tasks, and demonstrate that our approach outperforms or matches the performance of the state-of-the-arts on these tasks.