Hollywood 3D: What are the Best 3D Features for Action Recognition?

详细信息    查看全文

• 作者：Simon Hadfield ; Karel Lebeda ; Richard Bowden
• 关键词：Action recognition ; In the wild ; 3D ; Structure ; Depth ; 3D motion ; Hollywood 3D ; Benchmark
• 刊名：International Journal of Computer Vision
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：121
• 期：1
• 页码：95-110
• 全文大小：1918KB
• 刊物类别：Computer Science
• 刊物主题：Computer Imaging, Vision, Pattern Recognition and Graphics; Artificial Intelligence (incl. Robotics); Image Processing and Computer Vision; Pattern Recognition;
• 出版者：Springer US
• ISSN：1573-1405
• 卷排序：121

文摘

Action recognition “in the wild” is extremely challenging, particularly when complex 3D actions are projected down to the image plane, losing a great deal of information. The recent growth of 3D data in broadcast content and commercial depth sensors, makes it possible to overcome this. However, there is little work examining the best way to exploit this new modality. In this paper we introduce the Hollywood 3D benchmark, which is the first dataset containing “in the wild” action footage including 3D data. This dataset consists of 650 stereo video clips across 14 action classes, taken from Hollywood movies. We provide stereo calibrations and depth reconstructions for each clip. We also provide an action recognition pipeline, and propose a number of specialised depth-aware techniques including five interest point detectors and three feature descriptors. Extensive tests allow evaluation of different appearance and depth encoding schemes. Our novel techniques exploiting this depth allow us to reach performance levels more than triple those of the best baseline algorithm using only appearance information. The benchmark data, code and calibrations are all made available to the community.