视频帧率上转换检测技术综述
|
摘要
视频帧率上转换检测技术是视频取证技术的一种。为了系统阐述视频上转换检测领域现状,合理导向后续研究,对相关技术进行了综述。首先对相关研究历史和发展进程进行阐述,总结上转换概念及技术框架。然后根据检测技术的目的,对现有算法分类阐述。最后汇总介绍了视频帧率上转换检测领域的主要研究团队及其研究成果。从算法框架、检测结果等方面对比现有检测技术,提出了两点展望。视频帧率上转换作为视频后处理技术的重要组成部分,目前仍需进一步研究。
Video frame rate up-conversion detection technology is a kind of video forensics technology. In order to systematically elaborate the research progress in the field of video up-conversion detection and guide the follow-up research of video up-conversion detection algorithms, the main algorithms of up-conversion detection technology are summarized. First of all, elaborate the relevant research history and development process and summarize the up-conversion concept and technical framework. According to the purpose of the detection technology, classify and explain the existing algorithms. Finally, the main research teams in the field and their research results were summarized. From the aspects of algorithm framework and test results, compare the existing detection techniques and propose two prospects. Video frame rate up-conversion is an important part of video post-processing technology, and still need further research.
Video frame rate up-conversion detection technology is a kind of video forensics technology. In order to systematically elaborate the research progress in the field of video up-conversion detection and guide the follow-up research of video up-conversion detection algorithms, the main algorithms of up-conversion detection technology are summarized. First of all, elaborate the relevant research history and development process and summarize the up-conversion concept and technical framework. According to the purpose of the detection technology, classify and explain the existing algorithms. Finally, the main research teams in the field and their research results were summarized. From the aspects of algorithm framework and test results, compare the existing detection techniques and propose two prospects. Video frame rate up-conversion is an important part of video post-processing technology, and still need further research.
引文
[1]GIRONI A,FONTANI M,BIANCHI T.A video forensic technique for detecting frame deletion and insertion[J].IEEE International Conference on Acoustics,Speech and Signal Processing.2014:6226-6230.
[2]LUO W,QU Z,PAN F,et al.A survey of passive technology for digital image forensics[J].Frontiers of Computer Science in China.2008,1(2):166-179.
[3]黄昆仑,白蔚.视频帧率上转换技术综述[J].数字通信世界,2011(5):82-84.HUANG K L,BAI W.Overview of video frame rate up-conversion technology[J].Digital communication world,2011(5):82-84.
[4]LUO W,WU M,HUANG J.MPEG recompression detection based on block artifacts[J].Proceedings of SPIE-The International Society for Optical.2008,6819:68190X-68190X-12.
[5]DING X,YANG G,Li R.Identification of motion-compensated frame rate up-conversion based on residual signal[J].IEEE Transactions on Circuits&Systems for Video Technology,2017,PP(99):1-1.
[6]BESTAGINI P,BATTAGLIA S,MILANI S,et al.Detection of temporal interpolation in video sequences[C]//IEEE International Conference on Acoustics,Speech and Signal Processing.2013:3033-3037.
[7]BIAN S,LUO W,HUANG J.Detecting video frame-rate up-conversion based on periodic properties of inter-frame similarity[J].Multimedia Tools&Applications,2014,72(1):437-451.
[8]夏明,杨高波,黄勤珍.使用PSNR周期特性检测视频帧率上转伪造[J].计算机应用研究,2016,33(8):2409-2412.XIA M,YANG G B,HUANG Q Z.Detecting frame-rate up-conversion forgery in videos using periodicity of PSNR change[J].Application Research of Computers,2016,33(8):2409-2412.
[9]YAO Y,YANG G,SUN X.Detecting video frame-rate up-conversion based on periodic properties of edge-intensity[J].Journal of Information Security&Applications,2016,26(C):39-50.
[10]XIA M,YANG G,LI L.Detecting video frame rate up-conversion based on frame-level analysis of average texture variation[J].Multimedia Tools&Applications,2017,76(6):8399-8421.
[11]JUNG D J,LEE H K.Frame-rate conversion detection based on periodicity of motion artifact[J].Multimedia Tools&Applications,2017(1):1-22.
[12]CHAO J,JIANG X,SUN T.A novel video inter-frame forgery model detection scheme based on optical flow consistency[M].Digital Forensics and Watermarking.Springer Berlin Heidelberg,2013:267-281.
[13]WU Y,JIANG X,SUN T.Exposing video inter-frame forgery based on velocity field consistency[C]//IEEE International Conference on Acoustics,Speech and Signal Processing.IEEE,2014:2674-2678.
[14]LIU Y,HUANG T.Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis[J].Multimedia Systems,2017,23(2):223-238.
[15]ORCHARD M T,SULLIVAN G J.Overlapped block motion compensation:an estimation-theoretic approach.[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,1994,3(5):693-9.
[16]KANG S J,CHO K R,KIM Y H.Motion compensated frame rate up-conversion using extended bilateral motion estimation[J].IEEETransactions on Consumer Electronics,2008,53(4):1759-1767.
[17]CHOI B D,HAN J W,KIM C S.Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation[J].IEEE Transactions on Circuits&Systems for Video Technology,2007,17(4):407-416.
[18]KANG S J,YOO S,KIM Y H.Dual motion estimation for frame rate up-conversion[J].IEEE Transactions on Circuits&Systems for Video Technology,2011,20(12):1909-1914.
[19]LI R,GAN Z,CUI Z.Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual-weighted overlapped block motion compensation[J].Journal of Display Technology,2014,10(12):1010-1023.
[20]YOO D G,KANG S J,KIM Y H.Direction-select motion estimation for motion-compensated frame rate up-conversion[J].IEEE/OSA Journal of Display Technology,2013,9(10):840-850.
[21]LIN H Y,TSAI T H.High visual quality particle based frame rate up conversion with acceleration assisted motion trajectory calibration[J].Journal of Display Technology,2012,8(6):341-351.
[22]LIU H,XIONG R,ZHAO D.Multiple hypotheses bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(8):1188-1198.
[23]陈伟.基于运动估计和运动补偿的帧率上转换算法研究[D].哈尔滨工业大学,2016.CHEN W.Research on frame rate up-conversion algorithm based on motion estimation and motion compensation[D].Harbin Institute of Technology,2016.
[24]KANG S J,CHO K R,KIM Y H.Motion compensated frame rate up-conversion using extended bilateral motion estimation[J].IEEETransactions on Consumer Electronics,2008,53(4):1759-1767.
[25]ORCHARD M T,SULLIVAN G J.Overlapped block motion compensation:an estimation-theoretic approach[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,1994,3(5):693-9.
[26]CHOI B D,HAN J W,Kim C S,et al.Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation[J].IEEE Transactions on Circuits&Systems for Video Technology,2007,17(4):407-416.
[27]KANG S J,YOO S,KIM Y H.Dual motion estimation for frame rate up-conversion[J].IEEE Transactions on Circuits&Systems for Video Technology,2011,20(12):1909-1914.
[28]胡圆圆.基于视觉显著性的视频关键帧提取与帧速率上转换[D].南京邮电大学,2016.HU Y Y.Video key frame extraction and frame rate up conversion based on vision saliency[D].Nanjing University of Posts and Telecommunications,2016.
[29]LI R,GAN Z,CUI Z.Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual-weighted overlapped block motion compensation[J].Journal of Display Technology,2014,10(12):1010-1023.
[30]YOO D G,KANG S J,KIM Y H.Direction-select motion estimation for motion-compensated frame rate up-conversion[J].IEEE/OSA Journal of Display Technology,2013,9(10):840-850.
[31]MAHDIAN B,SAIC S.Blind authentication using periodic properties of interpolation[J].IEEE Transactions on Information Forensics&Security,2008,3(3):529-538.
[32]DIETTERICH T G,BAKIRI G.Solving multiclass learning problems via error correcting output codes[J].Journal of Artificial Intelligence Research,1995,2(1):263--286.
[33]POPESCU A C,FARID H.Exposing digital forgeries by detecting traces of resampling[J].IEEE Transactions on Signal Processing,2005,53(2):758-767.
[34]VAZQUEZ-PADIN D,COMESANA P.ML estimation of the resampling factor[C]//IEEE International Workshop on Information Forensics and Security.IEEE,2012:205-210.
[35]KIRCHNER M.Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue[C]//The Workshop on Multimedia&Security.ACM,2008:11-20.
[36]HUANG H C,FANG W C.Metadata-based image watermarking for copyright protection[J].Simulation Modelling Practice&Theory,2010,18(4):436-445.
[37]KOBAYASHI M,OKABE T,SATO Y.Detecting forgery from static-scene video based on inconsistency in noise level functions[J].IEEETransactions on Information Forensics&Security,2010,5(4):883-892.
[38]LIAO D,YANG R,LIU H.Double H.264/AVC compression detection using quantized nonzero AC coefficients[J].Proceedings of SPIE-The International Society for Optical Engineering,2011,7880(2):78800Q-78800Q-10.
[39]TEKALP A M.Digital video processing[M].Englewood Cliffs,NJ:Prentice Hall;1995.
[40]KAUFMAN P J.Smarter Trading:Improving Performance in Changing Markets[M].McGraw-Hill;1995:p.138-51.
[41]KIM U S,SUNWOO M H.New frame rate up-conversion algorithms with low computational complexity[J].IEEE Transactions on Circuits&Systems for Video Technology,2014,24(3):384-393.
[42]TSAI T H,Lin H Y.High visual quality particle based frame rate up conversion with acceleration assisted motion trajectory calibration[J].Journal of Display Technology,2012,8(6):341-351.
[43]LIU H,XIONG R,Zhao D.Multiple hypotheses Bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(8):1188-1198.
[44]DAR Y,BRUCKSTEIN A M.Motion-compensated coding and frame rate up-conversion:models and analysis[J].IEEE Transactions on Image Processing,2015,24(7):2051-2066.
[2]LUO W,QU Z,PAN F,et al.A survey of passive technology for digital image forensics[J].Frontiers of Computer Science in China.2008,1(2):166-179.
[3]黄昆仑,白蔚.视频帧率上转换技术综述[J].数字通信世界,2011(5):82-84.HUANG K L,BAI W.Overview of video frame rate up-conversion technology[J].Digital communication world,2011(5):82-84.
[4]LUO W,WU M,HUANG J.MPEG recompression detection based on block artifacts[J].Proceedings of SPIE-The International Society for Optical.2008,6819:68190X-68190X-12.
[5]DING X,YANG G,Li R.Identification of motion-compensated frame rate up-conversion based on residual signal[J].IEEE Transactions on Circuits&Systems for Video Technology,2017,PP(99):1-1.
[6]BESTAGINI P,BATTAGLIA S,MILANI S,et al.Detection of temporal interpolation in video sequences[C]//IEEE International Conference on Acoustics,Speech and Signal Processing.2013:3033-3037.
[7]BIAN S,LUO W,HUANG J.Detecting video frame-rate up-conversion based on periodic properties of inter-frame similarity[J].Multimedia Tools&Applications,2014,72(1):437-451.
[8]夏明,杨高波,黄勤珍.使用PSNR周期特性检测视频帧率上转伪造[J].计算机应用研究,2016,33(8):2409-2412.XIA M,YANG G B,HUANG Q Z.Detecting frame-rate up-conversion forgery in videos using periodicity of PSNR change[J].Application Research of Computers,2016,33(8):2409-2412.
[9]YAO Y,YANG G,SUN X.Detecting video frame-rate up-conversion based on periodic properties of edge-intensity[J].Journal of Information Security&Applications,2016,26(C):39-50.
[10]XIA M,YANG G,LI L.Detecting video frame rate up-conversion based on frame-level analysis of average texture variation[J].Multimedia Tools&Applications,2017,76(6):8399-8421.
[11]JUNG D J,LEE H K.Frame-rate conversion detection based on periodicity of motion artifact[J].Multimedia Tools&Applications,2017(1):1-22.
[12]CHAO J,JIANG X,SUN T.A novel video inter-frame forgery model detection scheme based on optical flow consistency[M].Digital Forensics and Watermarking.Springer Berlin Heidelberg,2013:267-281.
[13]WU Y,JIANG X,SUN T.Exposing video inter-frame forgery based on velocity field consistency[C]//IEEE International Conference on Acoustics,Speech and Signal Processing.IEEE,2014:2674-2678.
[14]LIU Y,HUANG T.Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis[J].Multimedia Systems,2017,23(2):223-238.
[15]ORCHARD M T,SULLIVAN G J.Overlapped block motion compensation:an estimation-theoretic approach.[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,1994,3(5):693-9.
[16]KANG S J,CHO K R,KIM Y H.Motion compensated frame rate up-conversion using extended bilateral motion estimation[J].IEEETransactions on Consumer Electronics,2008,53(4):1759-1767.
[17]CHOI B D,HAN J W,KIM C S.Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation[J].IEEE Transactions on Circuits&Systems for Video Technology,2007,17(4):407-416.
[18]KANG S J,YOO S,KIM Y H.Dual motion estimation for frame rate up-conversion[J].IEEE Transactions on Circuits&Systems for Video Technology,2011,20(12):1909-1914.
[19]LI R,GAN Z,CUI Z.Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual-weighted overlapped block motion compensation[J].Journal of Display Technology,2014,10(12):1010-1023.
[20]YOO D G,KANG S J,KIM Y H.Direction-select motion estimation for motion-compensated frame rate up-conversion[J].IEEE/OSA Journal of Display Technology,2013,9(10):840-850.
[21]LIN H Y,TSAI T H.High visual quality particle based frame rate up conversion with acceleration assisted motion trajectory calibration[J].Journal of Display Technology,2012,8(6):341-351.
[22]LIU H,XIONG R,ZHAO D.Multiple hypotheses bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(8):1188-1198.
[23]陈伟.基于运动估计和运动补偿的帧率上转换算法研究[D].哈尔滨工业大学,2016.CHEN W.Research on frame rate up-conversion algorithm based on motion estimation and motion compensation[D].Harbin Institute of Technology,2016.
[24]KANG S J,CHO K R,KIM Y H.Motion compensated frame rate up-conversion using extended bilateral motion estimation[J].IEEETransactions on Consumer Electronics,2008,53(4):1759-1767.
[25]ORCHARD M T,SULLIVAN G J.Overlapped block motion compensation:an estimation-theoretic approach[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,1994,3(5):693-9.
[26]CHOI B D,HAN J W,Kim C S,et al.Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation[J].IEEE Transactions on Circuits&Systems for Video Technology,2007,17(4):407-416.
[27]KANG S J,YOO S,KIM Y H.Dual motion estimation for frame rate up-conversion[J].IEEE Transactions on Circuits&Systems for Video Technology,2011,20(12):1909-1914.
[28]胡圆圆.基于视觉显著性的视频关键帧提取与帧速率上转换[D].南京邮电大学,2016.HU Y Y.Video key frame extraction and frame rate up conversion based on vision saliency[D].Nanjing University of Posts and Telecommunications,2016.
[29]LI R,GAN Z,CUI Z.Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual-weighted overlapped block motion compensation[J].Journal of Display Technology,2014,10(12):1010-1023.
[30]YOO D G,KANG S J,KIM Y H.Direction-select motion estimation for motion-compensated frame rate up-conversion[J].IEEE/OSA Journal of Display Technology,2013,9(10):840-850.
[31]MAHDIAN B,SAIC S.Blind authentication using periodic properties of interpolation[J].IEEE Transactions on Information Forensics&Security,2008,3(3):529-538.
[32]DIETTERICH T G,BAKIRI G.Solving multiclass learning problems via error correcting output codes[J].Journal of Artificial Intelligence Research,1995,2(1):263--286.
[33]POPESCU A C,FARID H.Exposing digital forgeries by detecting traces of resampling[J].IEEE Transactions on Signal Processing,2005,53(2):758-767.
[34]VAZQUEZ-PADIN D,COMESANA P.ML estimation of the resampling factor[C]//IEEE International Workshop on Information Forensics and Security.IEEE,2012:205-210.
[35]KIRCHNER M.Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue[C]//The Workshop on Multimedia&Security.ACM,2008:11-20.
[36]HUANG H C,FANG W C.Metadata-based image watermarking for copyright protection[J].Simulation Modelling Practice&Theory,2010,18(4):436-445.
[37]KOBAYASHI M,OKABE T,SATO Y.Detecting forgery from static-scene video based on inconsistency in noise level functions[J].IEEETransactions on Information Forensics&Security,2010,5(4):883-892.
[38]LIAO D,YANG R,LIU H.Double H.264/AVC compression detection using quantized nonzero AC coefficients[J].Proceedings of SPIE-The International Society for Optical Engineering,2011,7880(2):78800Q-78800Q-10.
[39]TEKALP A M.Digital video processing[M].Englewood Cliffs,NJ:Prentice Hall;1995.
[40]KAUFMAN P J.Smarter Trading:Improving Performance in Changing Markets[M].McGraw-Hill;1995:p.138-51.
[41]KIM U S,SUNWOO M H.New frame rate up-conversion algorithms with low computational complexity[J].IEEE Transactions on Circuits&Systems for Video Technology,2014,24(3):384-393.
[42]TSAI T H,Lin H Y.High visual quality particle based frame rate up conversion with acceleration assisted motion trajectory calibration[J].Journal of Display Technology,2012,8(6):341-351.
[43]LIU H,XIONG R,Zhao D.Multiple hypotheses Bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(8):1188-1198.
[44]DAR Y,BRUCKSTEIN A M.Motion-compensated coding and frame rate up-conversion:models and analysis[J].IEEE Transactions on Image Processing,2015,24(7):2051-2066.