基于彩色条纹结构光的三维重建方法研究
|
摘要
三维重建是计算机视觉的主要目的和计算机辅助设计的核心问题。在各种建模和重建技术中,结构光技术以其简单、非接触、测量范围大、精度高的特点,获得了广泛的应用。
本文研究了一种基于彩色条纹结构光的三维物体重建方法。主要工作包括彩色条纹的编码、颜色区分算法、边缘提取和解码算法、摄像机和投影仪标定方法、三维信息计算和三维重建方法。
首先,研究了彩色条纹的编码。提出了利用RGB颜色立方体八个顶点值对彩色条纹进行编码的方法,所采用的特定编码顺序保证了各条纹编码的唯一性,有利于后续的解码处理。设计了一种新的边缘提取方法,对彩色图像中R、G、B三个通道分别提取边缘,再将结果进行合并处理获得图像的边缘信息。实验效果表明此方法可以得到较完整准确的边缘。
第二,研究了将颜色信息从RGB空间转换到HSI空间中,实现对颜色区分的算法,以去除亮度不均的影响。提出了一种结合阈值和HSI空间的分色算法,利用阈值完成对于黑色和白色的识别,在HSI空间对彩色信息进行区分,实现对于彩色编码中使用的八种颜色的区分。提出了一种基于条纹颜色的解码算法,解决了物体表面不连续时的解码问题。实验结果表明,分色和解码算法可以实现较好的分色效果和准确的解码。
第三,研究了使用标准板对摄像机进行标定的方法。利用标准板上标志点中心的空间坐标和图像坐标获得最优的摄像机投影矩阵。基于摄像机的投影矩阵和标准板平面方程完成了投影仪的标定。
最后,研究了通过摄像机和投影仪的标定参数计算物体表面三维信息的方法。研究样条插值运算的算法并利用样条插值实现对物体表面的三维重建。通过实验证明了文中提出方法的有效性与先进性。
Three-dimensional reconstruction is the main purpose of machine vision and key issue in computer-aided design. Among the various modeling and reconstruction technologies, structured light is being widely used because of its advantages such as simplicity, no touch, wide detection range and high accuracy.
The research of three-dimensional reconstruction based on colorful stripe structured light is addressed in this thesis. The main work of the thesis includes coding and decoding strategy of the colorful stripe, color identification strategy, edge detection method, camera and projector calibration, three-dimensional position calculation and reconstruction.
Firstly, a coding strategy for colorful stripe is proposed in the thesis. To meet the uniqueness requirement of code word, which can simplify the decoding process, a new type of projection pattern based on the colors of the eight corners on the RGB cube is proposed in the thesis. In order to detect the complete and accurate edges in the colorful image, an edge detection method is proposed which firstly extracts the edges in R, G, B channels separately and then combines these three results together. The credibility of this edge detection method has been shown by the experimental results.
Secondly, in order to minimize the interference effect of lightness, the color data is transformed from RGB space to HSI space. A color identification algorithm combining channel threshold and HSI space is proposed, which identifies black and white colors by threshold and the other six colors in HSI space to accomplish the color identification of the eight colors used in coding. A decoding strategy using the stripe color is addressed to overcome the discontinuity problem. The results indicate that the proposed methods are effective.
Thirdly, in the calibration process the world and image coordinates of the marks' centers on the calibration plate are applied to optimize the camera projection matrix. The calibration of the projector is accomplished on the basis of camera projection matrix and plane equation of the calibration plate.
Lastly, the three-dimensional position calculation is carried out by using the parameters of the camera and the projector. The three-dimensional reconstruction is implemented based on spline interpolation. The effectiveness and advantages of the proposed method in the thesis are proved by the experimental results.
本文研究了一种基于彩色条纹结构光的三维物体重建方法。主要工作包括彩色条纹的编码、颜色区分算法、边缘提取和解码算法、摄像机和投影仪标定方法、三维信息计算和三维重建方法。
首先,研究了彩色条纹的编码。提出了利用RGB颜色立方体八个顶点值对彩色条纹进行编码的方法,所采用的特定编码顺序保证了各条纹编码的唯一性,有利于后续的解码处理。设计了一种新的边缘提取方法,对彩色图像中R、G、B三个通道分别提取边缘,再将结果进行合并处理获得图像的边缘信息。实验效果表明此方法可以得到较完整准确的边缘。
第二,研究了将颜色信息从RGB空间转换到HSI空间中,实现对颜色区分的算法,以去除亮度不均的影响。提出了一种结合阈值和HSI空间的分色算法,利用阈值完成对于黑色和白色的识别,在HSI空间对彩色信息进行区分,实现对于彩色编码中使用的八种颜色的区分。提出了一种基于条纹颜色的解码算法,解决了物体表面不连续时的解码问题。实验结果表明,分色和解码算法可以实现较好的分色效果和准确的解码。
第三,研究了使用标准板对摄像机进行标定的方法。利用标准板上标志点中心的空间坐标和图像坐标获得最优的摄像机投影矩阵。基于摄像机的投影矩阵和标准板平面方程完成了投影仪的标定。
最后,研究了通过摄像机和投影仪的标定参数计算物体表面三维信息的方法。研究样条插值运算的算法并利用样条插值实现对物体表面的三维重建。通过实验证明了文中提出方法的有效性与先进性。
Three-dimensional reconstruction is the main purpose of machine vision and key issue in computer-aided design. Among the various modeling and reconstruction technologies, structured light is being widely used because of its advantages such as simplicity, no touch, wide detection range and high accuracy.
The research of three-dimensional reconstruction based on colorful stripe structured light is addressed in this thesis. The main work of the thesis includes coding and decoding strategy of the colorful stripe, color identification strategy, edge detection method, camera and projector calibration, three-dimensional position calculation and reconstruction.
Firstly, a coding strategy for colorful stripe is proposed in the thesis. To meet the uniqueness requirement of code word, which can simplify the decoding process, a new type of projection pattern based on the colors of the eight corners on the RGB cube is proposed in the thesis. In order to detect the complete and accurate edges in the colorful image, an edge detection method is proposed which firstly extracts the edges in R, G, B channels separately and then combines these three results together. The credibility of this edge detection method has been shown by the experimental results.
Secondly, in order to minimize the interference effect of lightness, the color data is transformed from RGB space to HSI space. A color identification algorithm combining channel threshold and HSI space is proposed, which identifies black and white colors by threshold and the other six colors in HSI space to accomplish the color identification of the eight colors used in coding. A decoding strategy using the stripe color is addressed to overcome the discontinuity problem. The results indicate that the proposed methods are effective.
Thirdly, in the calibration process the world and image coordinates of the marks' centers on the calibration plate are applied to optimize the camera projection matrix. The calibration of the projector is accomplished on the basis of camera projection matrix and plane equation of the calibration plate.
Lastly, the three-dimensional position calculation is carried out by using the parameters of the camera and the projector. The three-dimensional reconstruction is implemented based on spline interpolation. The effectiveness and advantages of the proposed method in the thesis are proved by the experimental results.
引文
1. Filareti Tsalakanidou, Frank Forster, Sotiris Malassiotis. Real-time acquisition of depth and color images using structured light and its application to 3D face recognition [J], Real-Time Imaging,2005,11:358-369.
2. Frode Grytten, Egil Fagerholt, Trond Auestad. Out-of-plane deformation measurements of an aluminium plate during quasi-static perforation using structured light and close-range photogrammetry [J], International Journal of Solids and Structures,2007, doi:10.1016.
3. Zhenzhong Wei, Guangjun Zhang. Inspecting verticality of cylindrical work pieces via multi-vision sensors based on structured light [J], Optics and Lasers in Engineering,2005, 43:1167-1178.
4. Christophe Doignon, Dominique Knittel. A structured light vision system for out-of-plane vibration frequencies location of a moving web [J], Machine Vision and Applications, 2005,16(5):289-297.
5. Boulbaba Ben Amor, Mohsen Ardabilian, Liming Chen.3D face modeling based on structured-light assisted stereo sensor [C], ICIAP,2005,842-849.
6. Jiyoung Park, Cheolhwon Kim, Juneho Yi, Matthew Turk. Efficient depth edge detection using structured light [C], ISVC,2005,737-744.
7. S. Boverie, M. Devy, F. Lerasle. Comparison of structured light and stereovision sensors for new airbag generations [J], Control Engineering Practice,2003,11:1413-1421.
8. D.Q. Huynh. Calibrating a structured light stripe system:a novel approach [J], International Journal of Computer Vision,1999,33(1):73-86.
9. Ming-June Tsai, Chuan-Cheng Hung. Development of a high-precision surface metrology system using structured light projection [J], Measurement,2005,38:236-247.
10. Changsoo Je, Sang Wook Lee, Rae-Hong Park. High-contrast color-stripe pattern for rapid structured-light range imaging [C], ECCV,2004,95-107.
11. Peter Lindsey, Andrew Blake. Real-time tracking of surfaces with structured light [J], Image and Vision Computing,1995,13(7):585-591.
12. Jordi Pag'es, Joaquim Salvi, Carles Matabosch. Robust segmentation and decoding of a grid pattern for structured light [C], IBPRIA,2003,689-697.
13. A. Dipanda, S.Woo. Towards a real-time 3D shape reconstruction using a structured light system [J], Pattern Recognition,2005,38:1632-1650.
14. David Fofi, Joaquim Salvi, ElMustapha Mouaddib. Uncalibrated reconstruction:an adaptation to structured light vision [J], Pattern Recognition,2003,36:1631-1644.
15. Oleksandr A. Skydan, Michael J. Lalor, David R. Burton. Using colored structured light in 3-D surface measurement [J], Optics and Lasers in Engineering,2005,43:801-814.
16. Qingcang Yu, Xiaojun Jia, Jian Tao, Yun Zhao. An encoded mini-grid structured light pattern for dynamic scenes [C], ICIC,2005,126-135.
17. Hui-Min Yue, Xian-Yu Su, Yong-Zhi Liu. Fourier transform profilometry based on composite structured light pattern [J], Optics & Laser Technology,2007,39:1170-1175.
18. Zhenzhong Wei, Fuqiang Zhou, Guangjun Zhang.3D coordinates measurement based on structured light sensor [J], Sensors and Actuators,2005, A 120:527-535.
19. Guanghui Wang, Zhanyi Hu, Fuchao Wu, Hung-Tat Tsui. Implementation and experimental study on fast object modeling based on multiple structured stripes [J], Optics and Lasers in Engineering,2004,42:627-638.
20. B. Zhang, Y.F. Li, Y. H. Wu. Self-recalibration of a structured light system via plane-based homography [J], Pattern Recognition,2007,40:1368-1377.
21.万波,张大朴.利用交比不变性的摄像机内参数标定方法[J],计算机工程,2008,34(6)：261-262.
22.张洪波,李元宗.摄像机标定的一种方法[J],机械管理开发,2007,2：78-79.
23.张颖康,李雅轩,孟军英.一种多摄像机视觉系统的标定方法[J],河北科技大学学报,2008,29(1)：44-47.
24.王以忠,李琳,黄华芳.一种基于双目线结构光视觉系统的简单标定方法[J],长沙交通学院学报,2008,24(1)：58-62.
25.刘维一,王肇圻,母国光,方志良.彩色编码投影光栅三维轮廓术中分色问题的研究[J],光学学报,2001,21(4)：454-458.
26. Rameshi Jain, Rangachar Kasturi, Brian G. Schunck. Machine Vision [M], Beijing:China Machine Press,2003,289-364.
27.刘维一,王肇圻,母国光,方志良.彩色数字编码投影光栅三维轮廓术的研究[J],光 学学报,21(6)：687-690.
28. Joaquim Salvi, Jordi Pag'es, Joan Batlle. Pattern codification strategies in structured light systems [J], Pattern Recognition,2004,37:827-849.
29.于泓.摄像机标定算法研究[D],济南：山东大学,2006.
30.朱清溢.彩色编码结构光三维测量的研究[D],成都：四川大学,2006.
31.李宏伟.结构光三维视觉检测关键技术研究[D],哈尔滨：哈尔滨工程大学,2006.
32.谭晓波.摄像机标定及相关技术研究[D],长沙：国防科学技术大学,2004.
33. Rafael C. Gonzales, Richard E. Woods, Steven L. Eddins. Digital Image Processing [M], Beijing:Publishing House of Electronics Industry,2006,285-293.
34. J. Batlle, E. Mouaddib, J. Salvi. Recent progress in coded structured light as a technique to solve the correspondence problem:a survey [J], Pattern Recognition,1998,31(7): 963-982.
35.魏秀宁.基于结构光的三维测量与目标重建方法的研究[D],哈尔滨：哈尔滨理工大学,2006.
36.薛定宇.高等应用数学问题的MATLAB求解[M],北京：清华大学出版社,2005,260-275.
37.黄红强,冯华君,徐之海,李奇,戴顺林.彩色结构光三维成像技术[J],浙江大学学报,2001,35(6)：587-591.
38.杨萍,唐亚哲.结构光三维曲面重构[J],科学技术与工程,2006,6(19)：3057-3060.
39.周宝国.颜色编码三维测量系统及标定方法的研究[D],哈尔滨：哈尔滨理工大学,2006.
40. Dongjoe Shin, Jaihie Kim. Point to point calibration method of structured light for facial data reconstruction [C], ICBA,2004,200-206.
41. Xie Zexiao, Zhang Chengguo, Zhang Qiumei. A simplified method for the extrinsic calibration of structured light sensors using a single ball target [J], International Journal of Machine Tools & Manufacture,2004,44:1197-1203.
42. Fuqiang Zhou, Guangjun Zhang, Jie Jiang. Constructing feature points for calibrating a structured light vision sensor by viewing a plane from unknown orientations [J], Optics and Lasers in Engineering,2005,43:1056-1070.
43. Carlos Hinojosa, Alfonso Serrano-Heredia, Juan G. Ibarra. Recovery of three-dimensional shapes by using defocused structured light [J], Optics & Laser Technology,1998, 30:281-290.
2. Frode Grytten, Egil Fagerholt, Trond Auestad. Out-of-plane deformation measurements of an aluminium plate during quasi-static perforation using structured light and close-range photogrammetry [J], International Journal of Solids and Structures,2007, doi:10.1016.
3. Zhenzhong Wei, Guangjun Zhang. Inspecting verticality of cylindrical work pieces via multi-vision sensors based on structured light [J], Optics and Lasers in Engineering,2005, 43:1167-1178.
4. Christophe Doignon, Dominique Knittel. A structured light vision system for out-of-plane vibration frequencies location of a moving web [J], Machine Vision and Applications, 2005,16(5):289-297.
5. Boulbaba Ben Amor, Mohsen Ardabilian, Liming Chen.3D face modeling based on structured-light assisted stereo sensor [C], ICIAP,2005,842-849.
6. Jiyoung Park, Cheolhwon Kim, Juneho Yi, Matthew Turk. Efficient depth edge detection using structured light [C], ISVC,2005,737-744.
7. S. Boverie, M. Devy, F. Lerasle. Comparison of structured light and stereovision sensors for new airbag generations [J], Control Engineering Practice,2003,11:1413-1421.
8. D.Q. Huynh. Calibrating a structured light stripe system:a novel approach [J], International Journal of Computer Vision,1999,33(1):73-86.
9. Ming-June Tsai, Chuan-Cheng Hung. Development of a high-precision surface metrology system using structured light projection [J], Measurement,2005,38:236-247.
10. Changsoo Je, Sang Wook Lee, Rae-Hong Park. High-contrast color-stripe pattern for rapid structured-light range imaging [C], ECCV,2004,95-107.
11. Peter Lindsey, Andrew Blake. Real-time tracking of surfaces with structured light [J], Image and Vision Computing,1995,13(7):585-591.
12. Jordi Pag'es, Joaquim Salvi, Carles Matabosch. Robust segmentation and decoding of a grid pattern for structured light [C], IBPRIA,2003,689-697.
13. A. Dipanda, S.Woo. Towards a real-time 3D shape reconstruction using a structured light system [J], Pattern Recognition,2005,38:1632-1650.
14. David Fofi, Joaquim Salvi, ElMustapha Mouaddib. Uncalibrated reconstruction:an adaptation to structured light vision [J], Pattern Recognition,2003,36:1631-1644.
15. Oleksandr A. Skydan, Michael J. Lalor, David R. Burton. Using colored structured light in 3-D surface measurement [J], Optics and Lasers in Engineering,2005,43:801-814.
16. Qingcang Yu, Xiaojun Jia, Jian Tao, Yun Zhao. An encoded mini-grid structured light pattern for dynamic scenes [C], ICIC,2005,126-135.
17. Hui-Min Yue, Xian-Yu Su, Yong-Zhi Liu. Fourier transform profilometry based on composite structured light pattern [J], Optics & Laser Technology,2007,39:1170-1175.
18. Zhenzhong Wei, Fuqiang Zhou, Guangjun Zhang.3D coordinates measurement based on structured light sensor [J], Sensors and Actuators,2005, A 120:527-535.
19. Guanghui Wang, Zhanyi Hu, Fuchao Wu, Hung-Tat Tsui. Implementation and experimental study on fast object modeling based on multiple structured stripes [J], Optics and Lasers in Engineering,2004,42:627-638.
20. B. Zhang, Y.F. Li, Y. H. Wu. Self-recalibration of a structured light system via plane-based homography [J], Pattern Recognition,2007,40:1368-1377.
21.万波,张大朴.利用交比不变性的摄像机内参数标定方法[J],计算机工程,2008,34(6)：261-262.
22.张洪波,李元宗.摄像机标定的一种方法[J],机械管理开发,2007,2：78-79.
23.张颖康,李雅轩,孟军英.一种多摄像机视觉系统的标定方法[J],河北科技大学学报,2008,29(1)：44-47.
24.王以忠,李琳,黄华芳.一种基于双目线结构光视觉系统的简单标定方法[J],长沙交通学院学报,2008,24(1)：58-62.
25.刘维一,王肇圻,母国光,方志良.彩色编码投影光栅三维轮廓术中分色问题的研究[J],光学学报,2001,21(4)：454-458.
26. Rameshi Jain, Rangachar Kasturi, Brian G. Schunck. Machine Vision [M], Beijing:China Machine Press,2003,289-364.
27.刘维一,王肇圻,母国光,方志良.彩色数字编码投影光栅三维轮廓术的研究[J],光 学学报,21(6)：687-690.
28. Joaquim Salvi, Jordi Pag'es, Joan Batlle. Pattern codification strategies in structured light systems [J], Pattern Recognition,2004,37:827-849.
29.于泓.摄像机标定算法研究[D],济南：山东大学,2006.
30.朱清溢.彩色编码结构光三维测量的研究[D],成都：四川大学,2006.
31.李宏伟.结构光三维视觉检测关键技术研究[D],哈尔滨：哈尔滨工程大学,2006.
32.谭晓波.摄像机标定及相关技术研究[D],长沙：国防科学技术大学,2004.
33. Rafael C. Gonzales, Richard E. Woods, Steven L. Eddins. Digital Image Processing [M], Beijing:Publishing House of Electronics Industry,2006,285-293.
34. J. Batlle, E. Mouaddib, J. Salvi. Recent progress in coded structured light as a technique to solve the correspondence problem:a survey [J], Pattern Recognition,1998,31(7): 963-982.
35.魏秀宁.基于结构光的三维测量与目标重建方法的研究[D],哈尔滨：哈尔滨理工大学,2006.
36.薛定宇.高等应用数学问题的MATLAB求解[M],北京：清华大学出版社,2005,260-275.
37.黄红强,冯华君,徐之海,李奇,戴顺林.彩色结构光三维成像技术[J],浙江大学学报,2001,35(6)：587-591.
38.杨萍,唐亚哲.结构光三维曲面重构[J],科学技术与工程,2006,6(19)：3057-3060.
39.周宝国.颜色编码三维测量系统及标定方法的研究[D],哈尔滨：哈尔滨理工大学,2006.
40. Dongjoe Shin, Jaihie Kim. Point to point calibration method of structured light for facial data reconstruction [C], ICBA,2004,200-206.
41. Xie Zexiao, Zhang Chengguo, Zhang Qiumei. A simplified method for the extrinsic calibration of structured light sensors using a single ball target [J], International Journal of Machine Tools & Manufacture,2004,44:1197-1203.
42. Fuqiang Zhou, Guangjun Zhang, Jie Jiang. Constructing feature points for calibrating a structured light vision sensor by viewing a plane from unknown orientations [J], Optics and Lasers in Engineering,2005,43:1056-1070.
43. Carlos Hinojosa, Alfonso Serrano-Heredia, Juan G. Ibarra. Recovery of three-dimensional shapes by using defocused structured light [J], Optics & Laser Technology,1998, 30:281-290.