基于双特征融合与自适应提升机制的图像动作识别算法
|
摘要
针对复杂环境中动作识别易受到光照变化、目标旋转,遮挡等导致目标位置不精确,目标漂移以及识别错误等问题,提出了一种基于双特征融合与自适应提升的动作识别算法。首先,基于时空上下文(spatio-temporal context,STC)机制,通过时空上下文关系与视觉系统特性来提取图像序列特征,降低光照变化、遮挡对行为动作的影响;同时,利用卷积神经网络(convolution neural network,CNN)来处理图像序列,分别获得STC特征与CNN特征;其次,引入主成分分析算子,定义双特征融合规则,对获得STC特征与CNN特征进行组合,形成一种更准确、完整的特征表示;然后,通过得到的新特征,利用自适应提升算法(adaptive boosting algorithm,ABA)进行分类训练,完成对行为动作决策判断。在Weizmann、Hollywood数据集上测试表明,相对于当前常用的动作识别方法,所提算法对各种行为动作具有更高的识别精度与鲁棒性,更能适应复杂背景和动作变化。所提算法具有较高的人体动作识别精度,在视频监测、人机交互等领域具有一定的应用价值。
In order to solve the defects such as inaccurate target location,target drift and recognition error induced by influence of illumination change,target rotation,occlusion in complex environment,a motion recognition algorithm based on double feature fusion and adaptive boosting was proposed. Firstly,in order to reduce the influence of illumination variation and occlusion on behavior,spatiotemporal context was used to extract the image sequences feature based on spatiotemporal context and the visual system characteristics.At the same time,the convolution neural network was introduced to operate the image sequence's features for obtaining the STC and CNN features. Secondly,the principal component analysis operator was introduced to effectively combine the STC features and features to form a more accurate and complete feature representation. Then,by the new features,the adaptive boosting algorithm was used for classification training,the decision making of action was completed. The tests on the current popular data set show that,compared with the current commonly used behavior recognition methods,the proposed algorithm can recognize and understand all kinds of action,recognition rate is greatly improved,able to adapt for complex background and behavioral changes. This algorithm has higher accuracy and practical value in video surveillance and human-computer interaction.
In order to solve the defects such as inaccurate target location,target drift and recognition error induced by influence of illumination change,target rotation,occlusion in complex environment,a motion recognition algorithm based on double feature fusion and adaptive boosting was proposed. Firstly,in order to reduce the influence of illumination variation and occlusion on behavior,spatiotemporal context was used to extract the image sequences feature based on spatiotemporal context and the visual system characteristics.At the same time,the convolution neural network was introduced to operate the image sequence's features for obtaining the STC and CNN features. Secondly,the principal component analysis operator was introduced to effectively combine the STC features and features to form a more accurate and complete feature representation. Then,by the new features,the adaptive boosting algorithm was used for classification training,the decision making of action was completed. The tests on the current popular data set show that,compared with the current commonly used behavior recognition methods,the proposed algorithm can recognize and understand all kinds of action,recognition rate is greatly improved,able to adapt for complex background and behavioral changes. This algorithm has higher accuracy and practical value in video surveillance and human-computer interaction.
引文
[1]张飞燕,李俊峰,沈军民.基于梯度和光流统计特性的人体行为识别[J].光电子·激光,2015,26(8):1593-1601.ZHANG F Y,LI J F,SHEN J M.Human behavior recognition based on the statistical characteristics of gradient and optical flow[J].Journal of Optoelectronics Laser,2015,26(8):1593-1601.
[2]LI X X,CHEN SH X,ZHANG H SH.Towards reducing the impacts of unwanted movements on identification of motion intentions[J].Electromyography Kinesiol,2016,28(5):90-98.
[3]于宗泽,方勇,李兆元.基于压缩稀疏融合的动作识别方法[J].电子测量技术,2016,39(11):155-159.YU Z Z,FANG Y,LI ZH Y.Motion recognition method based on compressed sparse fusion[J].Electronic Measurement Technology,2016,39(11):155-159.
[4]秦磊,胡琼,黄庆明.基于特征点轨迹的动作识别[J].计算机学报,2014,37(6):1281-1288.QIN L,HU Q,HUANU Q M.Action recognition using trajectories of spatio-tempral feature points[J].Chinese Journal of Computers,2014,37(6):1281-1288.
[5]WANG H,KLASER A.Action recognition by dense trajectories[J].Computer Vision and Pattern Recognition,2013,62(11):3169-3176.
[6]林水强,吴亚东,余芳.姿势序列有限状态机动作识别方法[J].计算机辅助设计与图形学学报,2014,26(9):1403-1411.LIN SH Q,WU Y D,YU F.Posture sequence finitestate machine method for motion recognition[J].Journal of Computer-Aided Design&Computer Graphics,2014,26(9):1403-1411.
[7]刘万军,董帅含,曲海成.时空上下文抗遮挡视觉跟踪[J].中国图象图形学报,2016,21(8):1057-1062.LIU W J,DONG SH H,QU H C.Anti-bcclusion visual tracking algorithm based on spatial Temporal context learning[J].Journal of Image and Graphics,2016,21(8):1057-1062.
[8]徐建强,陆耀.一种基于加权时空上下文的鲁棒视觉跟踪算法[J].自动化学报,2015,41(11):1901-1912.XU J Q,LU Y.Robust visual tracking via weighted spatio-temporal context learning[J].Acta Automatica Sinica,2015,41(11):1901-1912.
[9]余永维,杜柳青,曾翠兰.基于深度学习特征匹配的铸件微小缺陷自动定位方法[J].仪器仪表学报,2016,37(6):1364-1370.YU Y W,DU L Q,ZENG C L.Automatic location method of small defects in castings based on depth learning feature matching[J].Chinese Journal of Instrument and Meters,2016,37(6):1364-1370.
[10]曲景影,孙显,高鑫.基于CNN模型的高分辨率遥感图像目标识别[J].国外电子测量技术,2016,12(8):45-50.QU J Y,SUN X,GAO X.Target recognition of high resolution remote sensing images based on CNN model[J].Foreign Electronic Measurement Technology,2016,12(8):45-50.
[11]霍雷刚,冯象初.基于主成分分析和字典学习的高光谱遥感图像去噪方法[J].电子与信息学报,2014,36(11):2723-2729.HUO L G,FENG X CH.Denoising of hyperspectral remote sensing image based on principal component analysis and dictionary learning[J].Journal of Electronics&Information Technology,2014,36(11):2723-2729.
[12]杨文文,毛建旭.基于分块LBP和分块PCA的指静脉识别方法[J].电子测量与仪器学报,2016,30(7):1000-1007.YANG W W,MAO J X.Finger vein recognition method based on block LBP and block PCA[J].Journal of Electronic Measurement and Instrumentation,2016,30(7):1000-1007.
[13]LIN ZH H,WANG L L,GAO J G.Interpretable functional principal component analysis[J].Biometrics,2016,72(3):846-854.
[14]付忠良.多标签代价敏感分类集成学习算法[J].自动化学报,2014,40(6):1075-1085.FU ZH L.Ensemble learning algorithm for multi label cost sensitive classification[J].Chinese Journal of Automation,2014,40(6):1075-1085.
[15]徐勤军,吴镇扬.视频序列中的行为识别研究进展[J].电子测量与仪器学报,2014,28(4):343-351.XU Q J,WU ZH Y.Research progress on activity recognition in video[J].Journal of Electronic Measurement and Instrumentation,2014,28(4):343-351.
[16]田国会,尹建芹,韩旭.一种基于关节点信息的人体行为识别新方法[J].机器人,2014,36(3):285-292.TIAN G H,YIN J Q,HAN X.A novel human activity recognition method using joint points information[J].Robot,2014,36(3):285-292.
[17]RAO L K,RAO D V.Local quantized extreme patterns for content based natural and texture image Retrieval[J].Human-Centric Computing and Information Sciences,2015,26(5):1-24.
[2]LI X X,CHEN SH X,ZHANG H SH.Towards reducing the impacts of unwanted movements on identification of motion intentions[J].Electromyography Kinesiol,2016,28(5):90-98.
[3]于宗泽,方勇,李兆元.基于压缩稀疏融合的动作识别方法[J].电子测量技术,2016,39(11):155-159.YU Z Z,FANG Y,LI ZH Y.Motion recognition method based on compressed sparse fusion[J].Electronic Measurement Technology,2016,39(11):155-159.
[4]秦磊,胡琼,黄庆明.基于特征点轨迹的动作识别[J].计算机学报,2014,37(6):1281-1288.QIN L,HU Q,HUANU Q M.Action recognition using trajectories of spatio-tempral feature points[J].Chinese Journal of Computers,2014,37(6):1281-1288.
[5]WANG H,KLASER A.Action recognition by dense trajectories[J].Computer Vision and Pattern Recognition,2013,62(11):3169-3176.
[6]林水强,吴亚东,余芳.姿势序列有限状态机动作识别方法[J].计算机辅助设计与图形学学报,2014,26(9):1403-1411.LIN SH Q,WU Y D,YU F.Posture sequence finitestate machine method for motion recognition[J].Journal of Computer-Aided Design&Computer Graphics,2014,26(9):1403-1411.
[7]刘万军,董帅含,曲海成.时空上下文抗遮挡视觉跟踪[J].中国图象图形学报,2016,21(8):1057-1062.LIU W J,DONG SH H,QU H C.Anti-bcclusion visual tracking algorithm based on spatial Temporal context learning[J].Journal of Image and Graphics,2016,21(8):1057-1062.
[8]徐建强,陆耀.一种基于加权时空上下文的鲁棒视觉跟踪算法[J].自动化学报,2015,41(11):1901-1912.XU J Q,LU Y.Robust visual tracking via weighted spatio-temporal context learning[J].Acta Automatica Sinica,2015,41(11):1901-1912.
[9]余永维,杜柳青,曾翠兰.基于深度学习特征匹配的铸件微小缺陷自动定位方法[J].仪器仪表学报,2016,37(6):1364-1370.YU Y W,DU L Q,ZENG C L.Automatic location method of small defects in castings based on depth learning feature matching[J].Chinese Journal of Instrument and Meters,2016,37(6):1364-1370.
[10]曲景影,孙显,高鑫.基于CNN模型的高分辨率遥感图像目标识别[J].国外电子测量技术,2016,12(8):45-50.QU J Y,SUN X,GAO X.Target recognition of high resolution remote sensing images based on CNN model[J].Foreign Electronic Measurement Technology,2016,12(8):45-50.
[11]霍雷刚,冯象初.基于主成分分析和字典学习的高光谱遥感图像去噪方法[J].电子与信息学报,2014,36(11):2723-2729.HUO L G,FENG X CH.Denoising of hyperspectral remote sensing image based on principal component analysis and dictionary learning[J].Journal of Electronics&Information Technology,2014,36(11):2723-2729.
[12]杨文文,毛建旭.基于分块LBP和分块PCA的指静脉识别方法[J].电子测量与仪器学报,2016,30(7):1000-1007.YANG W W,MAO J X.Finger vein recognition method based on block LBP and block PCA[J].Journal of Electronic Measurement and Instrumentation,2016,30(7):1000-1007.
[13]LIN ZH H,WANG L L,GAO J G.Interpretable functional principal component analysis[J].Biometrics,2016,72(3):846-854.
[14]付忠良.多标签代价敏感分类集成学习算法[J].自动化学报,2014,40(6):1075-1085.FU ZH L.Ensemble learning algorithm for multi label cost sensitive classification[J].Chinese Journal of Automation,2014,40(6):1075-1085.
[15]徐勤军,吴镇扬.视频序列中的行为识别研究进展[J].电子测量与仪器学报,2014,28(4):343-351.XU Q J,WU ZH Y.Research progress on activity recognition in video[J].Journal of Electronic Measurement and Instrumentation,2014,28(4):343-351.
[16]田国会,尹建芹,韩旭.一种基于关节点信息的人体行为识别新方法[J].机器人,2014,36(3):285-292.TIAN G H,YIN J Q,HAN X.A novel human activity recognition method using joint points information[J].Robot,2014,36(3):285-292.
[17]RAO L K,RAO D V.Local quantized extreme patterns for content based natural and texture image Retrieval[J].Human-Centric Computing and Information Sciences,2015,26(5):1-24.