视频压缩感知中分级多假设预测算法
|
摘要
目的多假设预测是视频压缩感知多假设预测残差重构算法的关键技术之一,现有的视频压缩感知多假设预测算法中预测分块固定,这种方法存在两点不足:1)对于视频帧中运动形式复杂的图像块预测效果不佳; 2)对于运动平缓区域,相邻图像块的运动矢量非常相近,每块单独通过运动估计寻找最佳匹配块,导致算法复杂度较大。针对这些问题,提出了分级多假设预测思路(Hi-MH),即对运动复杂程度不同的区域采取不同的块匹配预测方法。方法对于平缓运动区域的图像块,利用邻域图像块的运动矢量预测当前块的运动矢量,从而降低运动估计的算法复杂度;对于运动较复杂的图像块,用更小的块寻找最佳匹配;对于运动特别复杂的图像块利用自回归模型对单个像素点进行预测,提高预测精度。结果 Hi-MH算法与现有的快速搜索预测算法相比,每帧预测时间至少缩短了1.4s,与现有最优的视频压缩感知重构算法相比,对于运动较为复杂的视频序列,峰值信噪比(PSNR)提升幅度达到1dB。结论 Hi-MH算法对于运动形式简单的视频序列或区域降低了计算复杂度,对于运动形式较为复杂的视频序列或区域提高了预测精度。
Objective In traditional video acquisition,a video signal is sampled based on Nyquist sampling theory with a sampling frequency greater than or equal to twice the maximum frequency of the signal. The spatial and temporal redundancy information in the video signal is removed by the conventional encoding method. As people's requirements on the quality of multimedia content are increasing,the burden on the video encoder is becoming heavier. However,the traditional videocoding method is unsuitable for the application environments with limits in power consumption,storage capacity,and computing power( e. g.,wireless video surveillance). Compressed sensing( CS) conducts sampling and compression simultaneously,thereby saving enormous sampling resources while reducing the sampling complexity significantly. Thus,this technique is suitable for application scenarios with a resource-deprived sampling side. CS-based distributed video coding attracts considerable attention,in which utilizing the correlation among frames to reconstruct video efficiently has become a main research area. Multi-hypothesis( MH) prediction is a key technique in predicting residual reconstruction algorithm for compressed video sensing. In the existing MH prediction algorithm,the block size usually remains unchanged during the prediction process. The scheme accuracy depends on the similarity between the hypothetical and current blocks; hence,high similarity of the block group is assumed to lead to a good prediction result. Nevertheless,the content motion type is complicated for some image blocks in a video frame. The invariable-size block prediction scheme consequently leads to inconsiderably similar matching blocks and poor prediction results. Simulations indicate that the motion vectors of the image block in the motion gradual region are close,and therefore,searching the best match for each single block produces an unnecessary computing burden. The existing MH prediction algorithm generally has two disadvantages. First,the prediction accuracy for video frames with complex movement is poor. Second,for the smooth motion region or frames,the motion vectors of adjacent image blocks are highly similar,and searching the best matching block for each one separately leads to high algorithm complexity. Method For these problems,we propose a hierarchical MH prediction method( Hi-MH) that adopts different block-matching prediction methods for regions with different motion complexities and then introduce an implementation method. For the image block in smooth motion regions,the motion vector of the current block is predicted by that of the neighboring image block to decrease the motion estimation complexity( Motion estimation starts from a large block with a size four times of the observing block,and the motion estimation process from large block to small block is controlled by a suitable threshold to ensure the accuracy of each motion estimation until the block size is smaller than the observing block size,which means that this image block does not belong to a flat motion area). For the image blocks with complex movement,smaller blocks are used to find the best match and then adopt the MH prediction in pixel domain to obtain the prediction block. For the image blocks with a considerably complex movement,the autoregressive model is used to predict every individual pixel in the blocks. The reconstruction superiority of the regression model improves the prediction accuracy.Result A comparison of the result of Hi-MH and that of an MH prediction scheme based on fast diamond search with two matching regions( MH-DS) shows that the prediction time for each frame decreases by 1. 43 s and 1. 73 s for the Foreman and Coastguard sequences, respectively. The reconstruction accuracy of Hi-MH is higher than those of 2 sMHR( Gw_2 sMHR,Fw_2 sMHR) and MH-DS. At the sample rate from 0. 1 to 0. 5 for non-key frames,the average PSNR of Hi-MH is 1. 3 d B better than that of Fw_ 2 sMHR,1. 1 d B better than that of Gw_ 2 sMHR,and 0. 34 d B better than that of MH-DS. Compared with the PBCR algorithm which currently has the best reconstruction accuracy,the Hi-MH improves the reconstruction accuracy by 1 d B for some complex motion sequences. Conclusion 1) The Hi-MH algorithm is improved based on the MH-DS algorithm. For some image blocks with complex motion,the hierarchical motion estimation scheme in Hi-MH can find more accurate matching regions and obtain high-quality hypothesis block groups to improve the prediction accuracy of those blocks. The block classification prediction scheme in Hi-MH improves the prediction accuracy for some severely deformed image blocks; therefore,the overall reconstruction quality is enhanced. 2) For fast-moving video sequences,the Hi-MH algorithm has a significant improvement in reconstruction result over the PBCR-DCVS algorithm which currently has the best reconstruction quality. Local correlation in the videos is fully utilized because the Hi-MH algorithm proposed in this study can obtain higher accuracy image block-matching regions through the fast diamond search method and hierarchical motion estimation. Thus,the video reconstruction result is better. For slow-moving video sequences,such as Mother-daughter and Coastguard,the Hi-MH algorithm remains superior to the PBCR-DCVS algorithm at low sampling rates. As the sampling rate increases,the advantage gradually disappears. The reason is that at low sampling rates,the PBCR-DCVS algorithm cannot find more high-quality hypothetical block groups but Hi-MH can better solve this problem,thereby greatly improving the reconstruction quality. As the sampling rate increases,numerous observations are transmitted to the decoder,and PBCR-DCVS can find a good matching block group that helps in high-quality reconstruction.However,the neighborhood motion vector prediction technique used in Hi-MH to reduce the motion estimation complexity decreases the quality of the matching block group and the reconstruction quality. In general,the Hi-MH algorithm reduces the computational complexity for video sequences or regions with simple movement and improves the prediction accuracy for video sequences or regions with complex motion patterns.
Objective In traditional video acquisition,a video signal is sampled based on Nyquist sampling theory with a sampling frequency greater than or equal to twice the maximum frequency of the signal. The spatial and temporal redundancy information in the video signal is removed by the conventional encoding method. As people's requirements on the quality of multimedia content are increasing,the burden on the video encoder is becoming heavier. However,the traditional videocoding method is unsuitable for the application environments with limits in power consumption,storage capacity,and computing power( e. g.,wireless video surveillance). Compressed sensing( CS) conducts sampling and compression simultaneously,thereby saving enormous sampling resources while reducing the sampling complexity significantly. Thus,this technique is suitable for application scenarios with a resource-deprived sampling side. CS-based distributed video coding attracts considerable attention,in which utilizing the correlation among frames to reconstruct video efficiently has become a main research area. Multi-hypothesis( MH) prediction is a key technique in predicting residual reconstruction algorithm for compressed video sensing. In the existing MH prediction algorithm,the block size usually remains unchanged during the prediction process. The scheme accuracy depends on the similarity between the hypothetical and current blocks; hence,high similarity of the block group is assumed to lead to a good prediction result. Nevertheless,the content motion type is complicated for some image blocks in a video frame. The invariable-size block prediction scheme consequently leads to inconsiderably similar matching blocks and poor prediction results. Simulations indicate that the motion vectors of the image block in the motion gradual region are close,and therefore,searching the best match for each single block produces an unnecessary computing burden. The existing MH prediction algorithm generally has two disadvantages. First,the prediction accuracy for video frames with complex movement is poor. Second,for the smooth motion region or frames,the motion vectors of adjacent image blocks are highly similar,and searching the best matching block for each one separately leads to high algorithm complexity. Method For these problems,we propose a hierarchical MH prediction method( Hi-MH) that adopts different block-matching prediction methods for regions with different motion complexities and then introduce an implementation method. For the image block in smooth motion regions,the motion vector of the current block is predicted by that of the neighboring image block to decrease the motion estimation complexity( Motion estimation starts from a large block with a size four times of the observing block,and the motion estimation process from large block to small block is controlled by a suitable threshold to ensure the accuracy of each motion estimation until the block size is smaller than the observing block size,which means that this image block does not belong to a flat motion area). For the image blocks with complex movement,smaller blocks are used to find the best match and then adopt the MH prediction in pixel domain to obtain the prediction block. For the image blocks with a considerably complex movement,the autoregressive model is used to predict every individual pixel in the blocks. The reconstruction superiority of the regression model improves the prediction accuracy.Result A comparison of the result of Hi-MH and that of an MH prediction scheme based on fast diamond search with two matching regions( MH-DS) shows that the prediction time for each frame decreases by 1. 43 s and 1. 73 s for the Foreman and Coastguard sequences, respectively. The reconstruction accuracy of Hi-MH is higher than those of 2 sMHR( Gw_2 sMHR,Fw_2 sMHR) and MH-DS. At the sample rate from 0. 1 to 0. 5 for non-key frames,the average PSNR of Hi-MH is 1. 3 d B better than that of Fw_ 2 sMHR,1. 1 d B better than that of Gw_ 2 sMHR,and 0. 34 d B better than that of MH-DS. Compared with the PBCR algorithm which currently has the best reconstruction accuracy,the Hi-MH improves the reconstruction accuracy by 1 d B for some complex motion sequences. Conclusion 1) The Hi-MH algorithm is improved based on the MH-DS algorithm. For some image blocks with complex motion,the hierarchical motion estimation scheme in Hi-MH can find more accurate matching regions and obtain high-quality hypothesis block groups to improve the prediction accuracy of those blocks. The block classification prediction scheme in Hi-MH improves the prediction accuracy for some severely deformed image blocks; therefore,the overall reconstruction quality is enhanced. 2) For fast-moving video sequences,the Hi-MH algorithm has a significant improvement in reconstruction result over the PBCR-DCVS algorithm which currently has the best reconstruction quality. Local correlation in the videos is fully utilized because the Hi-MH algorithm proposed in this study can obtain higher accuracy image block-matching regions through the fast diamond search method and hierarchical motion estimation. Thus,the video reconstruction result is better. For slow-moving video sequences,such as Mother-daughter and Coastguard,the Hi-MH algorithm remains superior to the PBCR-DCVS algorithm at low sampling rates. As the sampling rate increases,the advantage gradually disappears. The reason is that at low sampling rates,the PBCR-DCVS algorithm cannot find more high-quality hypothetical block groups but Hi-MH can better solve this problem,thereby greatly improving the reconstruction quality. As the sampling rate increases,numerous observations are transmitted to the decoder,and PBCR-DCVS can find a good matching block group that helps in high-quality reconstruction.However,the neighborhood motion vector prediction technique used in Hi-MH to reduce the motion estimation complexity decreases the quality of the matching block group and the reconstruction quality. In general,the Hi-MH algorithm reduces the computational complexity for video sequences or regions with simple movement and improves the prediction accuracy for video sequences or regions with complex motion patterns.
引文
[1]Donoho D L.Compressed sensing[J].IEEE Transactions on Information Theory,2006,52(4):1289-1306.[DOI:10.1109/TIT.2006.871582]
[2]Candes E J,Romberg J,Tao T.Robust uncertainty principles:exact signal reconstruction from highly incomplete frequency information[J].IEEE Transactions on Information Theory,2006,52(2):489-509.[DOI:10.1109/TIT.2005.862083]
[3]Fowler J E,Mun S,Tramel E W.Block-based compressed sensing of images and video[J].Foundations and Trends in Signal Processing,2012,4(4):297-416.[DOI:10.1561/2000000033]
[4]Mun S,Fowler J E.Block compressed sensing of images using directional transforms[C]//Proceedings of the 16th IEEE International Conference on Image Processing.Cairo,Egypt:IEEE,2009:3021-3024.[DOI:10.1109/ICIP.2009.5414429]
[5]Zhang J,Zhao D B,Jiang F,et al.Structural group sparse representation for image compressive sensing recovery[C]//Proceedings of 2013 Data Compression Conference.Snowbird,Utah,USA:IEEE,2013:331-340.[DOI:10.1109/DCC.2013.41]
[6]Zhang J,Zhao D B,Gao W.Group-based sparse representation for image restoration[J].IEEE Transactions on Image Processing,2014,23(8):3336-3351.[DOI:10.1109/TIP.2014.2323127]
[7]He Z J,Yang C L,Tang R D.Research on structural similarity based inter-frame group sparse representation for compressed video sensing[J].Acta Electronica Sinica,2018,46(3):544-553.[和志杰,杨春玲,汤瑞东.视频压缩感知中基于结构相似的帧间组稀疏表示重构算法研究[J].电子学报,2018,46(3):544-553.][DOI:10.3969/j.issn.0372-2112.2018.03.005]
[8]Chen C,Tramel E W,Fowler J E.Compressed-sensing recovery of images and video using multihypothesis predictions[C]//Proceedings of 2011 Conference Record of the 45th Asilomar Conference on Signals,Systems and Computers.Pacific Grove,CA,USA:IEEE,2011:1193-1198.[DOI:10.1109/ACSSC.2011.6190204]
[9]Azghani M,Karimi M,Marvasti F.Multihypothesis compressed video sensing technique[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(4):627-635.[DOI:10.1109/TCSVT.2015.2418586]
[10]Chen J,Chen Y Z,Qin D,et al.An elastic net-based hybrid hypothesis method for compressed video sensing[J].Multimedia Tools and Applications,2015,74(6):2085-2108.[DOI:10.1007/s11042-013-1743-y]
[11]Zheng S,Chen J,Kuo Y H.An improved distributed compressed video sensing scheme in reconstruction algorithm[J].Multimedia Tools and Applications,2018,77(7):8711-8728.[DOI:10.1007/s11042-017-4765-z]
[12]Chen J,Wang N,Xue F,et al.Distributed compressed video sensing based on the optimization of hypothesis set update technique[J].Multimedia Tools and Applications,2017,76(14):15735-15754.[DOI:10.1007/s11042-016-3866-4]
[13]Zhao C,Ma S W,Zhang J,et al.Video compressive sensing reconstruction via reweighted residual sparsity[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,27(6):1182-1195.[DOI:10.1109/TCSVT.2016.2527181]
[14]Ou W F,Yang C L,Li W H,et al.A two-stage multi-hypothesis reconstruction scheme in compressed video sensing[C]//Proceedings of 2016 IEEE International Conference on Image Processing.Phoenix,AZ,USA:IEEE,2016:2494-2498.[DOI:10.1109/ICIP.2016.7532808]
[15]Ou W F,Yang C L,Dai C.A two-stage multi-hypothesis reconstruction and two implementation schemes for compressed video sensing[J].Journal of Electronics&Information Technology,2017,39(7):1688-1696.[欧伟枫,杨春玲,戴超.一种视频压缩感知中两级多假设重构及实现方法[J].电子与信息学报,2017,39(7):1688-1696.][DOI:10.11999/JEIT161142]
[16]Yang C L,Dai C.A prediction scheme based on fast diamond search and two match regions in compressed video sensing[J].Journal of South China University of Technology:Natural Science Edition,2018,46(3):49-57.[杨春玲,戴超.视频压缩感知中基于菱形快速搜索的双匹配区域预测[J].华南理工大学学报:自然科学版,2018,46(3):49-57.]
[2]Candes E J,Romberg J,Tao T.Robust uncertainty principles:exact signal reconstruction from highly incomplete frequency information[J].IEEE Transactions on Information Theory,2006,52(2):489-509.[DOI:10.1109/TIT.2005.862083]
[3]Fowler J E,Mun S,Tramel E W.Block-based compressed sensing of images and video[J].Foundations and Trends in Signal Processing,2012,4(4):297-416.[DOI:10.1561/2000000033]
[4]Mun S,Fowler J E.Block compressed sensing of images using directional transforms[C]//Proceedings of the 16th IEEE International Conference on Image Processing.Cairo,Egypt:IEEE,2009:3021-3024.[DOI:10.1109/ICIP.2009.5414429]
[5]Zhang J,Zhao D B,Jiang F,et al.Structural group sparse representation for image compressive sensing recovery[C]//Proceedings of 2013 Data Compression Conference.Snowbird,Utah,USA:IEEE,2013:331-340.[DOI:10.1109/DCC.2013.41]
[6]Zhang J,Zhao D B,Gao W.Group-based sparse representation for image restoration[J].IEEE Transactions on Image Processing,2014,23(8):3336-3351.[DOI:10.1109/TIP.2014.2323127]
[7]He Z J,Yang C L,Tang R D.Research on structural similarity based inter-frame group sparse representation for compressed video sensing[J].Acta Electronica Sinica,2018,46(3):544-553.[和志杰,杨春玲,汤瑞东.视频压缩感知中基于结构相似的帧间组稀疏表示重构算法研究[J].电子学报,2018,46(3):544-553.][DOI:10.3969/j.issn.0372-2112.2018.03.005]
[8]Chen C,Tramel E W,Fowler J E.Compressed-sensing recovery of images and video using multihypothesis predictions[C]//Proceedings of 2011 Conference Record of the 45th Asilomar Conference on Signals,Systems and Computers.Pacific Grove,CA,USA:IEEE,2011:1193-1198.[DOI:10.1109/ACSSC.2011.6190204]
[9]Azghani M,Karimi M,Marvasti F.Multihypothesis compressed video sensing technique[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(4):627-635.[DOI:10.1109/TCSVT.2015.2418586]
[10]Chen J,Chen Y Z,Qin D,et al.An elastic net-based hybrid hypothesis method for compressed video sensing[J].Multimedia Tools and Applications,2015,74(6):2085-2108.[DOI:10.1007/s11042-013-1743-y]
[11]Zheng S,Chen J,Kuo Y H.An improved distributed compressed video sensing scheme in reconstruction algorithm[J].Multimedia Tools and Applications,2018,77(7):8711-8728.[DOI:10.1007/s11042-017-4765-z]
[12]Chen J,Wang N,Xue F,et al.Distributed compressed video sensing based on the optimization of hypothesis set update technique[J].Multimedia Tools and Applications,2017,76(14):15735-15754.[DOI:10.1007/s11042-016-3866-4]
[13]Zhao C,Ma S W,Zhang J,et al.Video compressive sensing reconstruction via reweighted residual sparsity[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,27(6):1182-1195.[DOI:10.1109/TCSVT.2016.2527181]
[14]Ou W F,Yang C L,Li W H,et al.A two-stage multi-hypothesis reconstruction scheme in compressed video sensing[C]//Proceedings of 2016 IEEE International Conference on Image Processing.Phoenix,AZ,USA:IEEE,2016:2494-2498.[DOI:10.1109/ICIP.2016.7532808]
[15]Ou W F,Yang C L,Dai C.A two-stage multi-hypothesis reconstruction and two implementation schemes for compressed video sensing[J].Journal of Electronics&Information Technology,2017,39(7):1688-1696.[欧伟枫,杨春玲,戴超.一种视频压缩感知中两级多假设重构及实现方法[J].电子与信息学报,2017,39(7):1688-1696.][DOI:10.11999/JEIT161142]
[16]Yang C L,Dai C.A prediction scheme based on fast diamond search and two match regions in compressed video sensing[J].Journal of South China University of Technology:Natural Science Edition,2018,46(3):49-57.[杨春玲,戴超.视频压缩感知中基于菱形快速搜索的双匹配区域预测[J].华南理工大学学报:自然科学版,2018,46(3):49-57.]