基于全卷积神经网络的屏幕区域定位算法
|
摘要
很多情况下,人们需要记录屏幕、投影仪中出现的信息,但是在拍摄到屏幕的同时不可避免地会拍摄到屏幕外的背景。为了解决这个问题,提出一种在手机等便携设备上找到拍摄视频中出现的屏幕区域的算法。提取出视频中的每一帧;对每一帧用全卷积神经网络得到屏幕边缘图像和屏幕位置图像;在屏幕边缘图像上用直线检测算法检测直线;对屏幕位置图像进行分析,从检测到的直线中找到四条直线作为屏幕区域的边缘。由于全卷积神经网络的加入,该方法不需要设定复杂的参数,而且便于扩展到名片、文档等检测上。实验结果表明,该方法拥有很强的鲁棒性,较好的识别速度和准确率。
In many cases, people need to record the information appearing in the screen and projector, but when they shoot the screen, they inevitably shoot the background outside the screen. In order to solve this problem, we proposed an algorithm to find the screen area in the shooting video on mobile phones and other portable devices. We extracted each frame in the video and used full convolution neural network to get the edge image and position image of the screen for each frame. Then the straight line detection algorithm was used to detect the line on the screen edge image, and we analyzed the position image of the screen. Four lines were found from the detected line as the edge of the screen area. Due to the addition of full convolution neural network, this method does not need to set complex parameters, and is easy to extend to business card, document and other detection. The experimental results show that the method has strong robustness, good recognition speed and accuracy.
In many cases, people need to record the information appearing in the screen and projector, but when they shoot the screen, they inevitably shoot the background outside the screen. In order to solve this problem, we proposed an algorithm to find the screen area in the shooting video on mobile phones and other portable devices. We extracted each frame in the video and used full convolution neural network to get the edge image and position image of the screen for each frame. Then the straight line detection algorithm was used to detect the line on the screen edge image, and we analyzed the position image of the screen. Four lines were found from the detected line as the edge of the screen area. Due to the addition of full convolution neural network, this method does not need to set complex parameters, and is easy to extend to business card, document and other detection. The experimental results show that the method has strong robustness, good recognition speed and accuracy.
引文
[1] Kittler J.On the accuracy of the Sobel edge detector[J].Image & Vision Computing,1983,1(1):37-42.
[2] Canny J.A computational approach to edge detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.
[3] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition,2015.
[4] Arbelaez P,Maire M,Fowlkes C,et al.Contour Detection and Hierarchical Image Segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.
[5] Martin D R,Fowlkes C C,Malik J.Learning to detect natural image boundaries using local brightness,color,and texture cues[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(5):530-549.
[6] Dollar P,Tu Z,Belongie S.Supervised learning of edges and object boundaries[C]//IEEE Conference on Computer Vision and Pattern Recognition,2006.
[7] Ren X.Multi-scale improves boundary detection in natural images[C]//European Conference on Computer Vision,2008.
[8] Dollar P,Zitnick C L.Fast edge detection using struc-tured forests[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,13(7):120-135.
[9] Ganin Y,Lempitsky V.N4-fields:Neural network nearest neighbor fields for image transforms[C]// Asian Conference on Computer Vision,2014.
[10] Bertasius G,Shi J,Torresani L.DeepEdge:A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition,2015.
[11] Hu H,Lan S,Jiang Y,et al.FastMask:Segment Multi-scale Object Candidates in One Shot[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[12] Liu S,Jia J,Fidler S,et al.SGN:Sequential Grouping Networks for Instance Segmentation[C]// 2017 IEEE International Conference on Computer Vision(ICCV).IEEE,2017.
[13] Xie S,Tu Z.Holistically-Nested Edge Detection[C]//IEEE International Conference on Computer Vision,2015.
[14] Liu Y,Cheng M,Hu X,et al.Richer Convolutional Features for Edge Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[15] Yu Z,Feng C,Liu M,et al.CASENet:Deep Category-Aware Semantic Edge Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[16] Caelles S,Maninis K K,Pont-Tuset J,et al.One-Shot Video Object Segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2017.
[17] Bertasius G,Shi J,Torresani L.Semantic segmentation with boundary neural fields[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016.
[18] Cheng J,Liu S,Tsai Y,et al.Learning to Segment Instances in Videos with Spatial Propagation Network[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[19] Harley A W,Derpanis K G,Kokkinos I.Segmentation-Aware Convolutional Networks Using Local Attention Masks [C]//IEEE International Conference on Computer Vision,2017.
[20] Li K,Wang J,Wang H,et al.Structuring Lecture Videos by Automatic Projection Screen Localization and Analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(6):1233-1246.
[21] He K,Gkioxari G,Dollar P,et al.Mask r-cnn[C]//IEEE International Conference on Computer Vision,2017.
[22] Bradski G R.Computer vision face tracking for use in a perceptual user interface[C]// IEEE Workshop Applications of Computer Vision,1998:214-219.
[23] Grabner H,Grabner M,Bischof H.Real-time tracking via online boosting[C]//Proceedings of the British Machine Vision Conference,2006.
[24] Babenko B,Yang M,Belongie S.Visual tracking with online multiple instance learning[C]//IEEE Conference on Computer Vision and Pattern Recognition,2009.
[2] Canny J.A computational approach to edge detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.
[3] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition,2015.
[4] Arbelaez P,Maire M,Fowlkes C,et al.Contour Detection and Hierarchical Image Segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.
[5] Martin D R,Fowlkes C C,Malik J.Learning to detect natural image boundaries using local brightness,color,and texture cues[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(5):530-549.
[6] Dollar P,Tu Z,Belongie S.Supervised learning of edges and object boundaries[C]//IEEE Conference on Computer Vision and Pattern Recognition,2006.
[7] Ren X.Multi-scale improves boundary detection in natural images[C]//European Conference on Computer Vision,2008.
[8] Dollar P,Zitnick C L.Fast edge detection using struc-tured forests[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,13(7):120-135.
[9] Ganin Y,Lempitsky V.N4-fields:Neural network nearest neighbor fields for image transforms[C]// Asian Conference on Computer Vision,2014.
[10] Bertasius G,Shi J,Torresani L.DeepEdge:A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition,2015.
[11] Hu H,Lan S,Jiang Y,et al.FastMask:Segment Multi-scale Object Candidates in One Shot[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[12] Liu S,Jia J,Fidler S,et al.SGN:Sequential Grouping Networks for Instance Segmentation[C]// 2017 IEEE International Conference on Computer Vision(ICCV).IEEE,2017.
[13] Xie S,Tu Z.Holistically-Nested Edge Detection[C]//IEEE International Conference on Computer Vision,2015.
[14] Liu Y,Cheng M,Hu X,et al.Richer Convolutional Features for Edge Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[15] Yu Z,Feng C,Liu M,et al.CASENet:Deep Category-Aware Semantic Edge Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[16] Caelles S,Maninis K K,Pont-Tuset J,et al.One-Shot Video Object Segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2017.
[17] Bertasius G,Shi J,Torresani L.Semantic segmentation with boundary neural fields[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016.
[18] Cheng J,Liu S,Tsai Y,et al.Learning to Segment Instances in Videos with Spatial Propagation Network[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017.
[19] Harley A W,Derpanis K G,Kokkinos I.Segmentation-Aware Convolutional Networks Using Local Attention Masks [C]//IEEE International Conference on Computer Vision,2017.
[20] Li K,Wang J,Wang H,et al.Structuring Lecture Videos by Automatic Projection Screen Localization and Analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(6):1233-1246.
[21] He K,Gkioxari G,Dollar P,et al.Mask r-cnn[C]//IEEE International Conference on Computer Vision,2017.
[22] Bradski G R.Computer vision face tracking for use in a perceptual user interface[C]// IEEE Workshop Applications of Computer Vision,1998:214-219.
[23] Grabner H,Grabner M,Bischof H.Real-time tracking via online boosting[C]//Proceedings of the British Machine Vision Conference,2006.
[24] Babenko B,Yang M,Belongie S.Visual tracking with online multiple instance learning[C]//IEEE Conference on Computer Vision and Pattern Recognition,2009.