Multi-modal and Multi-spectral Registration for Natural Images

详细信息    查看全文

• 作者：Xiaoyong Shen (19)
  Li Xu (20)
  Qi Zhang (19)
  Jiaya Jia (19)
  
• 关键词：multi ; modal ; multi ; spectral ; dense matching ; variational model
• 刊名：Lecture Notes in Computer Science
• 出版年：2014
• 出版时间：2014
• 年：2014
• 卷：8692
• 期：1
• 页码：309-324
• 全文大小：3,762 KB
• 参考文献：1. Agrawal, A.K., Raskar, R., Nayar, S.K., Li, Y.: Removing photography artifacts using gradient projection and flash-exposure sampling. ToG?24(3), 828-35 (2005) CrossRef
  2. Andronache, A., von Siebenthal, M., Székely, G., Cattin, P.C.: Non-rigid registration of multi-modal images using both mutual information and cross-correlation. Medical Image Analysis?12(1), 3-5 (2008) CrossRef
  3. Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU?63(1), 75-04 (1996)
  4. Brown, M., Susstrunk, S.: Multi-spectral sift for scene category recognition. In: CVPR, pp. 177-84 (2011)
  5. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J.(G.) (eds.) ECCV 2004. LNCS, vol.?3024, pp. 25-6. Springer, Heidelberg (2004) CrossRef
  6. Bruhn, A., Weickert, J.: Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: ICCV, pp. 749-55 (2005)
  7. Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding?89(2), 114-41 (2003) CrossRef
  8. Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. ToG?21(3), 249-56 (2002) CrossRef
  9. Firmenichy, D., Brown, M., Süsstrunk, S.: Multispectral interest points for rgb-nir image registration. In: ICIP, pp. 181-84 (2011)
  10. Han, J., Pauwels, E.J., de Zeeuw, P.M.: Visible and infrared image registration in man-made environments employing hybrid visual features. Pattern Recognition Letters?34(1), 42-1 (2013) CrossRef
  11. Heo, Y.S., Lee, K.M., Lee, S.U.: Robust stereo matching using adaptive normalized cross-correlation. PAMI?33(4), 807-22 (2011) CrossRef
  12. Hermosillo, G., Chefd’Hotel, C., Faugeras, O.D.: Variational methods for multimodal image matching. IJCV?50(3), 329-43 (2002) CrossRef
  13. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell.?17(1-3), 185-03 (1981) CrossRef
  14. Hrka?, T., Kalafati?, Z., Krapac, J.: Infrared-visual image registration based on corners and hausdorff distance. In: Ersb?ll, B.K., Pedersen, K.S. (eds.) SCIA 2007. LNCS, vol.?4522, pp. 383-92. Springer, Heidelberg (2007) CrossRef
  15. Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: ICCV, pp. 959-66 (1998)
  16. Jian, B., Vemuri, B.C.: Robust point set registration using gaussian mixture models. PAMI?33(8), 1633-645 (2011) CrossRef
  17. Kolár, R., Kubecka, L., Jan, J.: Registration and fusion of the autofluorescent and infrared retinal images. International Journal of Biomedical Imaging (2008)
  18. Krishnan, D., Fergus, R.: Dark flash photography. ToG?28(3) (2009)
  19. Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: Sift flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol.?5304, pp. 28-2. Springer, Heidelberg (2008) CrossRef
  20. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV?60(2), 91-10 (2004) CrossRef
  21. Palos, G., Betrouni, N., Coulanges, M., Vermandel, M., Devlaminck, V., Rousseau, J.: Multimodal matching by maximisation of mutual information and optical flow technique. In: IEEE International Conference on Engineering in Medicine and Biology Society, pp. 1679-682 (2004)
  22. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M.F., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ToG?23(3), 664-72 (2004) CrossRef
  23. Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Mutual information based registration of medical images: A survey. IEEE Transaction on Medical Imaging?22(8), 986-004 (2003) CrossRef
  24. Sen, P., Kalantari, N.K., Yaesoubi, M., Darabi, S., Goldman, D.B., Shechtman, E.: Robust patch-based hdr reconstruction of dynamic scenes. ToG?31(6), 203 (2012) CrossRef
  25. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432-439 (2010)
  26. Sun, J., Kang, S.B., Xu, Z., Tang, X., Shum, H.Y.: Flash cut: Foreground extraction with flash and no-flash image pairs. In: CVPR (2007)
  27. Sun, J., Zheng, N., Shum, H.Y.: Stereo matching using belief propagation. PAMI?25(7), 787-00 (2003) CrossRef
  28. Szeliski, R.: Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics Vision?2(1), 1-04 (2006) CrossRef
  29. Tsin, Y., Kanade, T.: A correlation-based approach to robust point set registration. In: Pajdla, T., Matas, J.(G.) (eds.) ECCV 2004. LNCS, vol.?3023, pp. 558-69. Springer, Heidelberg (2004) CrossRef
  30. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences?1(3), 248-72 (2008) CrossRef
  31. Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol.?5302, pp. 739-51. Springer, Heidelberg (2008) CrossRef
  32. Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, pp. 2464-471 (2010)
  33. Xiong, Z., Zhang, Y.: A critical review of image registration methods. International Journal of Image and Data Fusion?1(2), 137-58 (2010) CrossRef
  34. Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. PAMI?34(9), 1744-757 (2012) CrossRef
  35. Yan, Q., Shen, X., Xu, L., Zhuo, S., Zhang, X., Shen, L., Jia, J.: Cross-field joint image restoration via scale map. In: ICCV (2013)
  36. Yang, J., Blum, R.S., Williams, J.P., Sun, Y., Xu, C.: Non-rigid image registration using geometric features and local salient region features. In: CVPR, pp. 825-32 (2006)
  37. Yi, Z., Soatto, S.: Nonrigid registration combining global and local statistics. In: CVPR (2009)
  38. Yuan, L., Sun, J., Quan, L., Shum, H.Y.: Image deblurring with blurred/noisy image pairs. ToG?26(3) (2007)
  39. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. Pattern Recognition, 214-23 (2007)
  40. Zhang, Z., Jiang, Y., Tsui, H.: Consistent multi-modal non-rigid registration based on a variational approach. Pattern Recognition Letters?27(7), 715-25 (2006) CrossRef
  41. Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., Seidel, H.-P.: Complementary optic flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol.?5681, pp. 207-20. Springer, Heidelberg (2009) CrossRef
  42. Zitová, B., Flusser, J.: Image registration methods: a survey. Image and Vision Computing?21(11), 977-000 (2003) CrossRef
  
• 作者单位：Xiaoyong Shen (19)
  Li Xu (20)
  Qi Zhang (19)
  Jiaya Jia (19)
  
  19. The Chinese University of Hong Kong, China
  20. Image & Visual Computing Lab, Lenovo R&T, Project Website, Hong Kong, China
  
• ISSN：1611-3349

文摘

Images now come in different forms -color, near-infrared, depth, etc. -due to the development of special and powerful cameras in computer vision and computational photography. Their cross-modal correspondence establishment is however left behind. We address this challenging dense matching problem considering structure variation possibly existing in these image sets and introduce new model and solution. Our main contribution includes designing the descriptor named robust selective normalized cross correlation (RSNCC) to establish dense pixel correspondence in input images and proposing its mathematical parameterization to make optimization tractable. A computationally robust framework including global and local matching phases is also established. We build a multi-modal dataset including natural images with labeled sparse correspondence. Our method will benefit image and vision applications that require accurate image alignment.