Human activity prediction using temporally-weighted generalized time warping

详细信息    查看全文

• 作者：Haoran Wang^a ; ^{wanghaoran@ise.neu.edu.cn}Author Vitae ; Wankou Yang^b ; ^{youngwankou@yeah.net}Author Vitae ; Chunfeng Yuan^c ; ^{cfyuan@nlpr.ia.ac.cn}Author Vitae ; Haibin Ling^d ; ^{hbling@temple.edu}Author Vitae ; Weiming Hu^c ; ^{wmhu@nlpr.ia.ac.cn}Author Vitae
• 关键词：Activity prediction ; Time warping ; Alignment
• 刊名：Neurocomputing
• 出版年：2017
• 出版时间：15 February 2017
• 年：2017
• 卷：225
• 期：Complete
• 页码：139-147
• 全文大小：1309 K
• 卷排序：225

文摘

Different from traditional human activity recognition, human activity prediction aims to recognize an unfinished activity, typically in absence of explicit temporal progress status. In this paper, we propose a new human activity prediction approach by extending the recently proposed generalized time warping (GTW) [20], which allows an efficient and flexible alignment of two or more multi-dimensional time series. More specifically, for each activity video, either complete or incomplete, we first decompose it into a sequence of short video segments. Then, we represent each segment by the local spatial-temporal statistics using the classical bag-of-visual-words model. In this way, the comparison between a query sequence (i.e., containing an incomplete activity) and a reference sequence (i.e., containing a full activity) boils down to the problem of aligning their corresponding segment sequences. While GTW treats different portions of a sequence as equally important, our task is in favor of early portions since an incomplete activity video always aligns from the beginning of a complete one. Thus motivated, we develop a temporally-weighted GTW (TGTW) algorithm for the activity prediction problem by encouraging alignment in the early portion of an activity sequence. Finally, the similarity derived from TGTW is combined with the k-nearest neighbors algorithm for predicting the activity class of an input sequence. The proposed approach is evaluated on several publicly available datasets in comparison with state-of-the-art approaches. The experimental results and analysis clearly demonstrate the effectiveness of the proposed approach.