基于骨架模型的人体行为分析
|
摘要
随着深度学习运用到图像领域,姿态估计、行为分析等算法的性能得到显著提升,希望在利用较好模型基础上进一步分析,在尽可能短的时间内得到更直观的结果。2016年提出的沙漏堆网络对人体关节点进行多尺度、多阶段的训练,在MPII数据集上回归了16对关节点坐标,在单个11 G显存的GPU上的平均准确率为87.6%;连接关节点构建人体骨架模型,然后根据骨架模型的加权角和倾斜角等几何特征,进一步推断人体的动作和行为状态,最后对人体行为进行分类和判断,包括站立、直坐、躺下等常见7类动作,平均准确率为82%,优势在于有效降低计算量和处理时间。
With the application of deep learning in the field of image, the performance of algorithm such as pose estimation and behavior analysis has been significantly improved. We hope to further analyze based on better models and get more intuitive results in the shortest time. The Stacked Hourglass network proposed in 2016 carried out multi-scale and multi-stage training on human keypoints, and regressed 16 pairs of coordinates of keypoints on the MPII dataset with 87.6% average accuracy rate on single GPU of 11 G video memory. These keypints were connected into a human body skeleton model. The motion and behavior of the human body are further inferred based on the geometric features of the skeleton model such as weighted angle and tilt angle. The result is classification on human behavior for seven common class actions including standing, sitting, lying and so on. The final average accuracy is 82% and the advantage is to effectively reduce the amount of calculation and processing time.
With the application of deep learning in the field of image, the performance of algorithm such as pose estimation and behavior analysis has been significantly improved. We hope to further analyze based on better models and get more intuitive results in the shortest time. The Stacked Hourglass network proposed in 2016 carried out multi-scale and multi-stage training on human keypoints, and regressed 16 pairs of coordinates of keypoints on the MPII dataset with 87.6% average accuracy rate on single GPU of 11 G video memory. These keypints were connected into a human body skeleton model. The motion and behavior of the human body are further inferred based on the geometric features of the skeleton model such as weighted angle and tilt angle. The result is classification on human behavior for seven common class actions including standing, sitting, lying and so on. The final average accuracy is 82% and the advantage is to effectively reduce the amount of calculation and processing time.
引文
[1] RAMAKRISHNA V,MUNOZ D,HEBERT M,et al.Pose machines:Articulated pose estimation via inference machines[C].Computer Vision-ECCV,2014.
[2] JAIN A,TOMPSON J,LECUN Y,et al.MoDeep:A deep learning framework using motion features for human pose estimation[C].Computer Vision-ACCV,2014:302-315.
[3] 赵勇,巨永锋.基于改进卷积神经网络的人体姿态估计[J].测控技术,2018,37(6):9-14.
[4] TOSHEV A,SZEGEDY C.Deeppose:Human pose estimation via deep neural networks[C].CVPR,IEEE,2014:1653-1660.
[5] JAIN M,JéGOU H,GROS P.Asymmetric hamming embedding:taking the best of our bits for large scale image search[C].ACM International Conference on Multimedia,2011:1441-1444.
[6] 马淼,李贻斌.基于多级动态模型的2维人体姿态估计[J].机器人,2016,38(5):578-587.
[7] 苏延超,艾海舟,劳世竑,图像和视频中基于部件检测器的人体姿态估计[J].电子与信息学报,2011,33(6):1413-1419.
[8] JéGOU H,CHUM O.Negative evidences and co-occurences in image retrieval:The benefit of PCA and whitening[C].European Conference on Computer Vision,2012:774-787.
[9] EIGEN D,PUHRSCH C,FERGUS R.Depth map prediction from a single image using a multi-scale deep network[J].Advances in Neural Information Processing Systems,2014:2366-2374.
[10] LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2014,39(4):640-651.
[11] 史东承,冯占君.视频中人体行为分析[J],吉林大学学报(信息科学版),2014,32(5):521-527.
[12] 黄永鑫.基于视觉的运动人体行为分析技术研究[J].黑龙江科技信息,2010(27):24.
[13] 李春霞,杨克俭,李波.人体骨架模型的建立及IK问题的一种解决方式[J].武汉理工大学学报(交通科学与工程版),2003,27(6):815-818.
[14] TOMPSON J,JAIN A,LECUN Y,et al.Joint training of a convolutional network and a graphical model for human pose estimation[J].Eprint Arxiv,2014:1799-1807.
[15] JOHNSON S,EVERINGHAM M.Learning effective human pose estimation from inaccurate annotation[C].Computer Vision and Pattern Recognition(CVPR),2011:1465-1472.
[16] 唐超,王文剑,李伟,等.基于多学习器协同训练模型的人体行为识别方法[J].软件学报,2015,26(11):2939-2950.
[17] 冉宪宇,刘凯,李光,等.自适应骨骼中心的人体行为识别算法[J].中国图象图形学报,2018,23(4):519-525.
[2] JAIN A,TOMPSON J,LECUN Y,et al.MoDeep:A deep learning framework using motion features for human pose estimation[C].Computer Vision-ACCV,2014:302-315.
[3] 赵勇,巨永锋.基于改进卷积神经网络的人体姿态估计[J].测控技术,2018,37(6):9-14.
[4] TOSHEV A,SZEGEDY C.Deeppose:Human pose estimation via deep neural networks[C].CVPR,IEEE,2014:1653-1660.
[5] JAIN M,JéGOU H,GROS P.Asymmetric hamming embedding:taking the best of our bits for large scale image search[C].ACM International Conference on Multimedia,2011:1441-1444.
[6] 马淼,李贻斌.基于多级动态模型的2维人体姿态估计[J].机器人,2016,38(5):578-587.
[7] 苏延超,艾海舟,劳世竑,图像和视频中基于部件检测器的人体姿态估计[J].电子与信息学报,2011,33(6):1413-1419.
[8] JéGOU H,CHUM O.Negative evidences and co-occurences in image retrieval:The benefit of PCA and whitening[C].European Conference on Computer Vision,2012:774-787.
[9] EIGEN D,PUHRSCH C,FERGUS R.Depth map prediction from a single image using a multi-scale deep network[J].Advances in Neural Information Processing Systems,2014:2366-2374.
[10] LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2014,39(4):640-651.
[11] 史东承,冯占君.视频中人体行为分析[J],吉林大学学报(信息科学版),2014,32(5):521-527.
[12] 黄永鑫.基于视觉的运动人体行为分析技术研究[J].黑龙江科技信息,2010(27):24.
[13] 李春霞,杨克俭,李波.人体骨架模型的建立及IK问题的一种解决方式[J].武汉理工大学学报(交通科学与工程版),2003,27(6):815-818.
[14] TOMPSON J,JAIN A,LECUN Y,et al.Joint training of a convolutional network and a graphical model for human pose estimation[J].Eprint Arxiv,2014:1799-1807.
[15] JOHNSON S,EVERINGHAM M.Learning effective human pose estimation from inaccurate annotation[C].Computer Vision and Pattern Recognition(CVPR),2011:1465-1472.
[16] 唐超,王文剑,李伟,等.基于多学习器协同训练模型的人体行为识别方法[J].软件学报,2015,26(11):2939-2950.
[17] 冉宪宇,刘凯,李光,等.自适应骨骼中心的人体行为识别算法[J].中国图象图形学报,2018,23(4):519-525.