基于压缩感知的视频编码技术研究
摘要
压缩感知是最近出现的信号处理理论。它指出对于一个可压缩的信号,只采集少量的线性投影,就包含了重构信号所需的足够的信息。该理论的出现缓解了人们对信息的高需求所带来的信号采样、传输以及存储的巨大压力。压缩感知是一种简单而且有效的信号获取方式,它可以先进行低速采样,之后再花费一定的计算量就能够从看起来似乎不完整的观测值中重构出原信号。该理论已经展开了许多的应用,尤其是在视频采集及编码方面。
传统的图像或视频信息获取过程是先进行高速采样,然后再进行信号压缩。而在压缩的过程中,通常的做法是将信号在变换域进行分解,舍弃大多数的数据而仅保留少量重要的数据,于是,大量的采样资源被浪费掉了。而基于压缩感知理论的视频采集方法避免了这一点,它能够在采样的同时进行压缩,而且所需的采样点数远小于传统方法的要求,因此能够大大节省采样资源。另外,这种方法还具有对噪声的鲁棒性、可延展性、加密型等优势,具有很高的应用价值。
本文提出了一种新的基于压缩感知的视频采集及编码方法。本方法中,视频序列中划分参考帧,而在参考帧之后的若干非参考帧用与其前一帧之间的差值信号来表示。在重构时利用视频序列帧间相关性强的特征,根据帧间差值信号的稀疏性,自适应地选择在空域或小波域进行重构。本文用模拟视频序列和真实视频序列分别进行了仿真分析,结果表明本文的方法能够获得很好的重构效果。
在此方法基础之上,本文又提出了一种改进的重构算法,该算法首先估计出图像中运动物体的大致位置,然后以此为先验信息,使用加权的l1范数重构算法重构信号。仿真结果表明这种改进的算法能够有效地改进重构效果。
Compressive sensing (CS) is a recently emerged signal processing method. It shows that a small number of linear projections of a compressible signal contain enough information for reconstruction. This theory provides an opportunity for solving the problem that the existing systems are very difficult to meet the challenges of high speed sampling, large volume data transmission and storage. CS is a very simple and efficient signal acquisition protocol which samples at a low rate and later uses computational power for reconstruction from what appears to be an incomplete set of measurements. This theory has many applications, especially in the field of video sampling and coding.
In the conventional image or video signal acquisition progress, we need to sample at a high rate first, and then compress the signal. The usual compressing method is that we express the signal in another domain, and then set all but the largest several coefficients to zero. So a large number of sampling resources are wasted. Video sampling based on compressive sensing can sample and compress signal at the same time, and the samples needed are far fewer than that of the conventional method. As a result, it can greatly save up sampling resources. What's more, this method also has advantages in robustness to noise, scalability, encryption, and so on. In brief, it has great value in application.
A new video acquisition and coding scheme based on compressive sensing is presented. In this scheme, the video sequence is represented by a reference frame followed by the differences of measurement results between each subsequent pair of neighboring frames. This method takes advantage of the spacial redundancy of the video sequence, and adaptively reconstruct the video signal in spacial domain or wavelet domain, according to the sparsity of the differencing signal of neighboring frames. Results using both simulated and real video sequences show that this scheme provides good recovery results.
A new signal recovery algorithm is presented based on this scheme. This algorithm firstly estimate the location of the moving object in the image, and then takes this as a priori knowledge to improve the reconstruction performance by a weighted l1 norm reconstruction method. Simulation results shows that this method can efficiently improve the reconstruction performance.
传统的图像或视频信息获取过程是先进行高速采样,然后再进行信号压缩。而在压缩的过程中,通常的做法是将信号在变换域进行分解,舍弃大多数的数据而仅保留少量重要的数据,于是,大量的采样资源被浪费掉了。而基于压缩感知理论的视频采集方法避免了这一点,它能够在采样的同时进行压缩,而且所需的采样点数远小于传统方法的要求,因此能够大大节省采样资源。另外,这种方法还具有对噪声的鲁棒性、可延展性、加密型等优势,具有很高的应用价值。
本文提出了一种新的基于压缩感知的视频采集及编码方法。本方法中,视频序列中划分参考帧,而在参考帧之后的若干非参考帧用与其前一帧之间的差值信号来表示。在重构时利用视频序列帧间相关性强的特征,根据帧间差值信号的稀疏性,自适应地选择在空域或小波域进行重构。本文用模拟视频序列和真实视频序列分别进行了仿真分析,结果表明本文的方法能够获得很好的重构效果。
在此方法基础之上,本文又提出了一种改进的重构算法,该算法首先估计出图像中运动物体的大致位置,然后以此为先验信息,使用加权的l1范数重构算法重构信号。仿真结果表明这种改进的算法能够有效地改进重构效果。
Compressive sensing (CS) is a recently emerged signal processing method. It shows that a small number of linear projections of a compressible signal contain enough information for reconstruction. This theory provides an opportunity for solving the problem that the existing systems are very difficult to meet the challenges of high speed sampling, large volume data transmission and storage. CS is a very simple and efficient signal acquisition protocol which samples at a low rate and later uses computational power for reconstruction from what appears to be an incomplete set of measurements. This theory has many applications, especially in the field of video sampling and coding.
In the conventional image or video signal acquisition progress, we need to sample at a high rate first, and then compress the signal. The usual compressing method is that we express the signal in another domain, and then set all but the largest several coefficients to zero. So a large number of sampling resources are wasted. Video sampling based on compressive sensing can sample and compress signal at the same time, and the samples needed are far fewer than that of the conventional method. As a result, it can greatly save up sampling resources. What's more, this method also has advantages in robustness to noise, scalability, encryption, and so on. In brief, it has great value in application.
A new video acquisition and coding scheme based on compressive sensing is presented. In this scheme, the video sequence is represented by a reference frame followed by the differences of measurement results between each subsequent pair of neighboring frames. This method takes advantage of the spacial redundancy of the video sequence, and adaptively reconstruct the video signal in spacial domain or wavelet domain, according to the sparsity of the differencing signal of neighboring frames. Results using both simulated and real video sequences show that this scheme provides good recovery results.
A new signal recovery algorithm is presented based on this scheme. This algorithm firstly estimate the location of the moving object in the image, and then takes this as a priori knowledge to improve the reconstruction performance by a weighted l1 norm reconstruction method. Simulation results shows that this method can efficiently improve the reconstruction performance.
引文
[1]E Candes. Compressive sampling[C]. Proceedings of the International Congress of Mathematicians. Madrid, Spain,2006,3:1433-1452.
[2]E Candes, J Romberg, Terence Tao. Robust uncertainty principles:Exact signal reconstruction from highly incomplete frequency information[J]. IEEE Trans. on Information Theory,2006,52(2):489-509.
[3]E Cande s and J Romberg. Quantitative robust uncertainty principles and optimally sparse decompositions[J]. Foundations of Comput Math,2006,6(2):227-254.
[4]D L Donoho. Compressed sensing[J]. IEEE Trans. on Information Theory.2006, 52(4):1289-1306.
[5]D L Donoho, Y Tsaig. Extensions of compressed sensing[J]. Signal Processing, 2006,86 (3):533-548.
[6]E. Candes and T. Tao. Near optimal signal recovery from random projections:Universal encoding strategies?[J]. IEEE Trans. Inform. Theory,2006,52(12):5406-5425.
[7]高文.媒体数据压缩技术[M].北京：电子工业出版社,1994.
[8]欧阳合,韩军.H.264和MPEG-4视频压缩—新一代多媒体的视频编码技术[M].长沙：国防科技大学出版社,2004.
[9]Tomes L, Kunt M. Video Coding:The Second Generation Approach[M]. Boston:Kuwer Academic Publishers,1996.
[10]Iain E. G. Richardson. H.264/MPEG-4 Part 10 White Paper[M]. http://www. vcodex. com /h264. html.
[11]Iain E.G.Richardson. H.264 and MPEG-4 Video Commpression Video Coding for Next Generation Multimedia[M].2003.
[12]Atul Puri, Xuemin Chen, Ajay Luthra. Video Coding Using the H.264/MPEG-4 AVC Compression standard[J]. Singnal Processing:Image Communication,2004,19: 793-849.
[13]E. Candes, B. Wakin. An Introduction To Compressive Sampling[J]. IEEE signal processing magazine,2008,3:21-30.
[14]G Peyre. Best Basis compressed sensing[J]. Lecture Notes in Computer Science, 2007,4485:80-91.
[15]D. L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition[J]. IEEE Trans. Inform. Theory,2001,47(7):2845-2862.
[16]E. Candes. The restricted isometry property and its implications for compressed sensing[C]. Computational Mathematics, California Institute of Technology, 2008,346 (9):589-592.
[17]R Baraniuk. A lecture on compressive sensing[J]. IEEE Signal Processing Magazine, 2007,24(4):118-121.
[18]R. G. Baraniuk, M. Davenport, R. DeVore, and M. B. Wakin. A simple proof of the restricted isometry principle for random matrices(aka the Johnson-Lindenstrauss lemma meets compressed sensing)[CP]. Constructive Approximation, http://dsp.rice.edu/cs/jlcs-v03.pdf,2007.
[19]R G Baraniuk, M Davenport, R DeVore etc. The Johnson-Lin-denstrauss lemma meets compressed sensing[CP]. http://dsp.rice.edu/cs/jlcs2v03.pdf.
[20]S S Chen, D L Donoho, and M A Saunders. Atomic decomposition by basis pursuit[J]. SIAM Review,2001,43(1):129-159.
[21]Richard G. Baraniuk. Compressive Sensing[J]. IEEE signal processing magazine, 2007,24(4):118-121.
[22]S S Chen, D L Donoho, and M A Saunders. Atomic decomposition by basis pursuit [J]. SIAM Review,2001,43(1):129-159.
[23]J A Tropp and A C Gilbert. Signal Recovery from Partial Information by Orthogonal Matching Pursuit[CP]. April 2005, www.personal.umich.edu/jtropp/papers/ TG05-Signal-Recovery.pdf.
[24]D L Donoho, Y Tsaig, I Drori etc. Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit[R]. Technical Report,2006.
[25]E. Candes and J. Romberg. Sparsity and incoherence in compressive sampling [J]. Inverse Prob.,2007,23(3):969-985.
[26]D. Baron, M. B. Wakin, M. F. Duarte, S. Sarvotham, and R. G. Baraniuk. Distributed compressed sensing[R].2005, Preprint.
[27]E. Candes and T. Tao. Decoding by linear programming[J]. IEEE Trans. Inform. Theory, 2005,51(12):4203-4215.
[28]M Lustig, D L Donoho, J M Pauly. Rapid MR Imaging with Compressed Sensing and Randomly Under Sampled 3DFT Trajectories[C]. Proceedings of the 14th. Annual Meeting of ISMRM. Seattle, WA,2006.
[29]M. B. Wakin, J. N. Laska, M. F. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. F. Kelly, and R. G. Baraniuk. An architecture for compressive imaging[C]. in Proc. IEEE Int. Conf. Image Processing,2006,1273-1276.
[30]D Takhar, J Laska,M Wakin,etc. A new compressive imaging camera architecture using optical-domain compression[C]. Proceedings of SPIE. Bellingham WA: International Society for Optical Engineering.2006:43-52.
[31]R. Marcia and R. Willett. Compressive coded aperture video reconstruct ion [C]. in European Signal Processing Conf. (EUSIPCO), Lausanne, Switzerland, August 2008:918-923.
[32]M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk. Compressive imaging for video representation and coding[C]. in Proc. Picture Coding Symp,2006:2046-2051.
[33]I. Drori. Compressed video sensing[R]. in Proc. BMVA Symp.3D Video-Analysis, Display, and Applications,2008.
[34]Abhijit Mahalanobis, Robert Muise. Object specific image reconstruction using a compressive sensing architecture for application in surveillance systems[J]. in Aerospace and Electronic Systems,2009,45(3):1289-1306.
[35]Jing Zheng, Eddie L. Jacobs. Video compressive sensing using spatial domain sparsity[J]. Optical Engineering,2009,48(8):1-10.
[36]Jae Young Park, Wakin, M. B. A multiscale framework for Compressive Sensing of video[C]. Picture Coding Symposium,2009:1-4.
[37]V. Stankovic, L. Stankovic, and S. Cheng. Compressive video sampling[C]. European Signal Processing Conf. (EUSIPCO), Lausanne, Switzerland, August 2008:3563-3567.
[38]Xie Xiaochun, Guan Lixin, Lu Zhenhui, Lai Zhaoshen. Fast encoding of video based on compressive sensing[C]. Information, Computing and Telecommunication, 2009,20:114-117.
[39]练秋生,陈书贞.基于混合基稀疏图像表示的压缩传感图像重构[J].自动化学报,2010,26(3)：385-391.
[40]Abhijit Mahalanobis, Robert Muise. Object specific image reconstruction using a compressive sensing architecture for application in surveillance systems[J]. in Aerospace and Electronic Systems,2009,45(3):1289-1306.
[41]Baig, Y. Lai, E.M.K. Lewis, J. P. Quantization effects on Compressed Sensing Video[C]. Telecommunications(ICT),2010 IEEE 17th International,2010,930-940.
[42]Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity [J]. IEEE Trans. Image Process,2004, 13(4):600-612.
[43]E. Candes, M. B. Wakin, and S. P. Boyd. Enhancing sparsity by reweighted minimization[J]. J. Fourier Anal. Appl., Special Issue on Sparsity, 2008,14 (5):877-905.
[2]E Candes, J Romberg, Terence Tao. Robust uncertainty principles:Exact signal reconstruction from highly incomplete frequency information[J]. IEEE Trans. on Information Theory,2006,52(2):489-509.
[3]E Cande s and J Romberg. Quantitative robust uncertainty principles and optimally sparse decompositions[J]. Foundations of Comput Math,2006,6(2):227-254.
[4]D L Donoho. Compressed sensing[J]. IEEE Trans. on Information Theory.2006, 52(4):1289-1306.
[5]D L Donoho, Y Tsaig. Extensions of compressed sensing[J]. Signal Processing, 2006,86 (3):533-548.
[6]E. Candes and T. Tao. Near optimal signal recovery from random projections:Universal encoding strategies?[J]. IEEE Trans. Inform. Theory,2006,52(12):5406-5425.
[7]高文.媒体数据压缩技术[M].北京：电子工业出版社,1994.
[8]欧阳合,韩军.H.264和MPEG-4视频压缩—新一代多媒体的视频编码技术[M].长沙：国防科技大学出版社,2004.
[9]Tomes L, Kunt M. Video Coding:The Second Generation Approach[M]. Boston:Kuwer Academic Publishers,1996.
[10]Iain E. G. Richardson. H.264/MPEG-4 Part 10 White Paper[M]. http://www. vcodex. com /h264. html.
[11]Iain E.G.Richardson. H.264 and MPEG-4 Video Commpression Video Coding for Next Generation Multimedia[M].2003.
[12]Atul Puri, Xuemin Chen, Ajay Luthra. Video Coding Using the H.264/MPEG-4 AVC Compression standard[J]. Singnal Processing:Image Communication,2004,19: 793-849.
[13]E. Candes, B. Wakin. An Introduction To Compressive Sampling[J]. IEEE signal processing magazine,2008,3:21-30.
[14]G Peyre. Best Basis compressed sensing[J]. Lecture Notes in Computer Science, 2007,4485:80-91.
[15]D. L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition[J]. IEEE Trans. Inform. Theory,2001,47(7):2845-2862.
[16]E. Candes. The restricted isometry property and its implications for compressed sensing[C]. Computational Mathematics, California Institute of Technology, 2008,346 (9):589-592.
[17]R Baraniuk. A lecture on compressive sensing[J]. IEEE Signal Processing Magazine, 2007,24(4):118-121.
[18]R. G. Baraniuk, M. Davenport, R. DeVore, and M. B. Wakin. A simple proof of the restricted isometry principle for random matrices(aka the Johnson-Lindenstrauss lemma meets compressed sensing)[CP]. Constructive Approximation, http://dsp.rice.edu/cs/jlcs-v03.pdf,2007.
[19]R G Baraniuk, M Davenport, R DeVore etc. The Johnson-Lin-denstrauss lemma meets compressed sensing[CP]. http://dsp.rice.edu/cs/jlcs2v03.pdf.
[20]S S Chen, D L Donoho, and M A Saunders. Atomic decomposition by basis pursuit[J]. SIAM Review,2001,43(1):129-159.
[21]Richard G. Baraniuk. Compressive Sensing[J]. IEEE signal processing magazine, 2007,24(4):118-121.
[22]S S Chen, D L Donoho, and M A Saunders. Atomic decomposition by basis pursuit [J]. SIAM Review,2001,43(1):129-159.
[23]J A Tropp and A C Gilbert. Signal Recovery from Partial Information by Orthogonal Matching Pursuit[CP]. April 2005, www.personal.umich.edu/jtropp/papers/ TG05-Signal-Recovery.pdf.
[24]D L Donoho, Y Tsaig, I Drori etc. Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit[R]. Technical Report,2006.
[25]E. Candes and J. Romberg. Sparsity and incoherence in compressive sampling [J]. Inverse Prob.,2007,23(3):969-985.
[26]D. Baron, M. B. Wakin, M. F. Duarte, S. Sarvotham, and R. G. Baraniuk. Distributed compressed sensing[R].2005, Preprint.
[27]E. Candes and T. Tao. Decoding by linear programming[J]. IEEE Trans. Inform. Theory, 2005,51(12):4203-4215.
[28]M Lustig, D L Donoho, J M Pauly. Rapid MR Imaging with Compressed Sensing and Randomly Under Sampled 3DFT Trajectories[C]. Proceedings of the 14th. Annual Meeting of ISMRM. Seattle, WA,2006.
[29]M. B. Wakin, J. N. Laska, M. F. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. F. Kelly, and R. G. Baraniuk. An architecture for compressive imaging[C]. in Proc. IEEE Int. Conf. Image Processing,2006,1273-1276.
[30]D Takhar, J Laska,M Wakin,etc. A new compressive imaging camera architecture using optical-domain compression[C]. Proceedings of SPIE. Bellingham WA: International Society for Optical Engineering.2006:43-52.
[31]R. Marcia and R. Willett. Compressive coded aperture video reconstruct ion [C]. in European Signal Processing Conf. (EUSIPCO), Lausanne, Switzerland, August 2008:918-923.
[32]M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk. Compressive imaging for video representation and coding[C]. in Proc. Picture Coding Symp,2006:2046-2051.
[33]I. Drori. Compressed video sensing[R]. in Proc. BMVA Symp.3D Video-Analysis, Display, and Applications,2008.
[34]Abhijit Mahalanobis, Robert Muise. Object specific image reconstruction using a compressive sensing architecture for application in surveillance systems[J]. in Aerospace and Electronic Systems,2009,45(3):1289-1306.
[35]Jing Zheng, Eddie L. Jacobs. Video compressive sensing using spatial domain sparsity[J]. Optical Engineering,2009,48(8):1-10.
[36]Jae Young Park, Wakin, M. B. A multiscale framework for Compressive Sensing of video[C]. Picture Coding Symposium,2009:1-4.
[37]V. Stankovic, L. Stankovic, and S. Cheng. Compressive video sampling[C]. European Signal Processing Conf. (EUSIPCO), Lausanne, Switzerland, August 2008:3563-3567.
[38]Xie Xiaochun, Guan Lixin, Lu Zhenhui, Lai Zhaoshen. Fast encoding of video based on compressive sensing[C]. Information, Computing and Telecommunication, 2009,20:114-117.
[39]练秋生,陈书贞.基于混合基稀疏图像表示的压缩传感图像重构[J].自动化学报,2010,26(3)：385-391.
[40]Abhijit Mahalanobis, Robert Muise. Object specific image reconstruction using a compressive sensing architecture for application in surveillance systems[J]. in Aerospace and Electronic Systems,2009,45(3):1289-1306.
[41]Baig, Y. Lai, E.M.K. Lewis, J. P. Quantization effects on Compressed Sensing Video[C]. Telecommunications(ICT),2010 IEEE 17th International,2010,930-940.
[42]Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity [J]. IEEE Trans. Image Process,2004, 13(4):600-612.
[43]E. Candes, M. B. Wakin, and S. P. Boyd. Enhancing sparsity by reweighted minimization[J]. J. Fourier Anal. Appl., Special Issue on Sparsity, 2008,14 (5):877-905.