Weakly supervised activity analysis with spatio-temporal localisation

详细信息    查看全文

• 作者：Feng Gu^a ; ^b ; ^{fengy1982@gmail.com" class="auth_mail" title="E-mail the corresponding author}Author Vitae ; Muralikrishna Sridhar^aAuthor Vitae ; Anthony Cohn^aAuthor Vitae ; David Hogg^aAuthor Vitae ; Francisco Fló ; rez-Revuelta^bAuthor Vitae ; Dorothy Monekosso^cAuthor Vitae ; Paolo Remagnino^bAuthor Vitae
• 关键词：Human activity analysis ; Spatio-temporal localisation ; Weakly labelled video data ; Multi-instance multi-label learning
• 刊名：Neurocomputing
• 出版年：2016
• 出版时间：5 December 2016
• 年：2016
• 卷：216
• 期：Complete
• 页码：778-789
• 全文大小：2211 K

文摘

In computer vision, an increasing number of weakly annotated videos have become available, due to the fact it is often difficult and time consuming to annotate all the details in the videos collected. Learning methods that analyse human activities in weakly annotated video data have gained great interest in recent years. They are categorised as “weakly supervised learning”, and usually form a multi-instance multi-label (MIML) learning problem. In addition to the commonly known difficulties of MIML learning, i.e. ambiguities in instances and labels, a weakly supervised method also has to cope with large data size, high dimensionality, and a large proportion of noisy examples usually found in video data. In this work, we propose a novel learning framework that iteratively optimises over a scalable MIML model and an instance selection process incorporating pairwise spatio-temporal smoothing during training. Such learned knowledge is then generalised to testing via a noise removal process based on the support vector data description algorithm. According to the experiments on three challenging benchmark video datasets, the proposed framework yields a more discriminative MIML model and less noisy training and testing data, and thus improves the system performance. It outperforms the state-of-the-art weakly supervised and even fully supervised approaches in the literature, in terms of annotating and detecting actions of a single person and interactions between a pair of people.