面向微操作的显微视觉深度恢复方法研究
|
摘要
显微视觉在微操作和微装配中起着极其重要的作用,显微视觉可以通过非接触的视觉反馈提供微操作目标的显微几何、空间关系以及更高级别的信息。但是,考虑到显微光学的独特属性,显微视觉与宏观视觉有很大的区别,其中显微镜景深有限是一个在微操作和微装配中应用时必须考虑的最为突出的特点。在微操作过程中存在非平面目标,同时需要聚焦于不同的高度。如何聚焦于不同的高度以及得到目标的三维信息已经成为将显微视觉应用于微操作和微装配的一个主要障碍。
本文首先分析了深度信息恢复的研究现状,其中两种技术-基于聚焦和基于离焦的方法是适用于微操作和微装配的。然后通过分析显微镜的成像原理和模糊图像的形成过程发现图像的模糊程度可以用来恢复目标的三维信息,这种方法只需要单目普通光学显微镜就可以完成。
在微操作过程中需要聚焦于不同高度的平面,因此自动调焦是必不可少的,本文提出了两种适用于微操作的基于离焦的快速自动调焦方法。为了达到自动调焦效率与精度兼顾的目的提出了另一种将离焦与聚焦结合的方法。实验证明这种方法在效率与精度之间做了很好的平衡。
在典型的三维微装配过程中,动态记录目标的位置信息和空间关系非常重要。微操作目标在三维空间的位置应该被实时的跟踪,本文提出了一种基于单目视觉的新颖的三维跟踪技术。一种被广泛应用于宏观视觉的二维跟踪方法camshift被应用于显微视觉对模糊目标进行二维跟踪,同时基于离焦(DFD)方法被用来得到微操作目标的相对高度信息,最后将camshift与DFD相结合成功的恢复了微操作目的运动轨迹。
最后,通过跟踪调焦和模糊操作两个综合实验的研究进一步证明了各种技术综合应用于微操作和微装配领域的重要意义。
Microscopic computer vision is of fundamental importance to micromanipulation and microassembly in providing noncontact feedback of microscale geometry, spatial relations and high-level task understanding .However, because of the unique properties of microscope optics, microscopic computer vision differs significantly from macroscale computer vision. The small depth-of-field of microscope optics is a fundamental constraint that must be considered in developing microscopic computer vision techniques for micromanipulation and microassembly. In a typical microassembly situation there are non-planar objects and several different focal planes. How to focus on different planes and get the 3D information has become a major roadblock to the development of practical visual applications to micromanipulation and microassembly.
In this paper, firstly we analysis the state of depth information recovery and two technichs Depth-from-Focus and Depth-from-Defocus are found to be feasible for micromanipulation and microassembly,then the principle of microscope and the forming process of blurred images are analysied.we found that the level of defocus of images can be used for 3D information recovery for micromanipulation and microassembly using a single optical microscope.
In a typical microassembly situation there are several different focal planes. Accordingly, autofocusing is an essential operation to focus on different depth. Two new effective fast autofocusing methods based on image defocus is presented for microscopy images in micromanipulation. A unified approach to DFF and DFD is constructed and experimentally demonstrated to be efficient and effective for common optical zoom microscope, it made a good trade-off between accuracy and efficiency.
Dynamic measurement of poses of the objects of interest and their spatial relations is important in a typical 3D microassembly situation; the micro object should be tracked in the 3D space in real time. In this paper, a novel imaging technique is proposed.This technique tracks 3D coordinates of micro-object for micromanipulation and microassembly using a single optical microscope. A widely used macroscale 2D tracking method called camshaft is used for microscopy images to track defocusing miro-object. And a depth from defocus based method is used to obtain the depth information in z direction. The integration of these two techniques is implemented successfully for obtaining 3D trajectories of micro-object.
At last, conpretentive experiments are designed to validate the feasible of our theory for manipulation and microassembly.
本文首先分析了深度信息恢复的研究现状,其中两种技术-基于聚焦和基于离焦的方法是适用于微操作和微装配的。然后通过分析显微镜的成像原理和模糊图像的形成过程发现图像的模糊程度可以用来恢复目标的三维信息,这种方法只需要单目普通光学显微镜就可以完成。
在微操作过程中需要聚焦于不同高度的平面,因此自动调焦是必不可少的,本文提出了两种适用于微操作的基于离焦的快速自动调焦方法。为了达到自动调焦效率与精度兼顾的目的提出了另一种将离焦与聚焦结合的方法。实验证明这种方法在效率与精度之间做了很好的平衡。
在典型的三维微装配过程中,动态记录目标的位置信息和空间关系非常重要。微操作目标在三维空间的位置应该被实时的跟踪,本文提出了一种基于单目视觉的新颖的三维跟踪技术。一种被广泛应用于宏观视觉的二维跟踪方法camshift被应用于显微视觉对模糊目标进行二维跟踪,同时基于离焦(DFD)方法被用来得到微操作目标的相对高度信息,最后将camshift与DFD相结合成功的恢复了微操作目的运动轨迹。
最后,通过跟踪调焦和模糊操作两个综合实验的研究进一步证明了各种技术综合应用于微操作和微装配领域的重要意义。
Microscopic computer vision is of fundamental importance to micromanipulation and microassembly in providing noncontact feedback of microscale geometry, spatial relations and high-level task understanding .However, because of the unique properties of microscope optics, microscopic computer vision differs significantly from macroscale computer vision. The small depth-of-field of microscope optics is a fundamental constraint that must be considered in developing microscopic computer vision techniques for micromanipulation and microassembly. In a typical microassembly situation there are non-planar objects and several different focal planes. How to focus on different planes and get the 3D information has become a major roadblock to the development of practical visual applications to micromanipulation and microassembly.
In this paper, firstly we analysis the state of depth information recovery and two technichs Depth-from-Focus and Depth-from-Defocus are found to be feasible for micromanipulation and microassembly,then the principle of microscope and the forming process of blurred images are analysied.we found that the level of defocus of images can be used for 3D information recovery for micromanipulation and microassembly using a single optical microscope.
In a typical microassembly situation there are several different focal planes. Accordingly, autofocusing is an essential operation to focus on different depth. Two new effective fast autofocusing methods based on image defocus is presented for microscopy images in micromanipulation. A unified approach to DFF and DFD is constructed and experimentally demonstrated to be efficient and effective for common optical zoom microscope, it made a good trade-off between accuracy and efficiency.
Dynamic measurement of poses of the objects of interest and their spatial relations is important in a typical 3D microassembly situation; the micro object should be tracked in the 3D space in real time. In this paper, a novel imaging technique is proposed.This technique tracks 3D coordinates of micro-object for micromanipulation and microassembly using a single optical microscope. A widely used macroscale 2D tracking method called camshaft is used for microscopy images to track defocusing miro-object. And a depth from defocus based method is used to obtain the depth information in z direction. The integration of these two techniques is implemented successfully for obtaining 3D trajectories of micro-object.
At last, conpretentive experiments are designed to validate the feasible of our theory for manipulation and microassembly.
引文
1赵新,孙明竹等.基于离焦状态模糊显微图像反馈的微操作方法.高技术通讯. 2006,(04):381-386
2吕遐东,黄心汉,王敏等.基于显微图像散焦特征的微操作机器人深度信息提取.机器人. 2003 , 25 (4) : 322-326
3赵新,余斌等.基于系统辨识的显微镜点扩散参数提取方法及应用.计算机学报. 2004 , 27(1) : 140-144
4张建勋,薛大庆,卢桂章等.通过显微图像特征抽取获得微操作目标纵向信息.机器人. 2003 , 25 (4):73-77
5王华,苏显渝.光学离焦三维传感技术的现状与进展.激光杂志,2003, 24(3):1-4
6黄大刚,张建勋,赵新.从平面显微图像中提取三维位置反馈信息方法的研究.仪器仪表学报. 2002, 23(3):230-231
7龚俊锋,徐西鹏.基于聚焦合成的砂轮表面三维重构方法.金刚石与磨料磨具工程, 2006,154(4):14-16
8孙玉秋,田金文等.基于图像金字塔的分维融合算法.计算机应用, 2005,25(5):1064-1075
9谢少荣,彭商贤,赵新等.基于虚拟显微镜技术的微操作工具Z方向定位方法研究.高技术通讯. 2001, 20(2):72-75
10李敏,赵新,卢桂章等.微操作机器人系统拟实环境的实现.机器人. 2001, 23(4):305-310
11王跃宗,刘冲等.微操作系统中微观对像三维立体形状重构研究.计算机辅助设计与图形学学报. 2003,15(6):701-705
12刘莉,姜志国等.光学体视显微图像立体测量系统研究与开发.中国体视学与图像分析,2003,12(8)
13夏明革,何友等.像素级图像融合方法分类与比较.火力与指挥控制. 27(3):161-164
14 G. Ligthart, and F.C.A.Groen. A Comparison of Different Autofocus Algorithms. Proceedings on International Conference on Pattern Recognition, 1982:597-600
15 M Boissenin, J Wedekind, AN Selvan, BP Amavasai. Focus set based reconstruction of micro-objects. Image and Vision Computing, 2004:1-4
16 T.Darrel, K. Wohn. Pyramid based depth from focus. Proc.of IEEE Conference on Computer Vision and Pattern Recognition, June 1988:504-509
17 Shree K. Nayar, Yasuo Nakagawa. Shape from Focus. IEEETransactions on pattern analysis and machine intelligence,Vol. 16, No 8, Aug, 1992: 824-830
18 T.Choi. Shape and Image Reconstruction from Focus. Ph.D.thesis, Dept.of lectrical Engg, SUNY at Stony Brook,1993
19 M. Subbarao and T.Choi. A New Method for Shape from Focus. Proceedings of SPIE Conference, Vol.2046,September 1993
20穆静,杜亚勤等.小波包变换的图像融合技术的研究.西安工业学院学报,2005,25(4):356-358
21熊四昌,杨涌等.基于图像小波分析的显微序列图像合成系统.光学仪器,2006,21(23):10467-10471
22 Chipman, L.J, T.M, Lewis, L.N. Wavelets and image fusion. IEEE Transactions on Image Processing, 1995:248-251
23 Foresti G L, Regazzoni C S. Statistical Mo rpho logical Skeleton for Representing and Coding Noisy Shapes. IEEE P roc. 2V ision Image Signal Process, 1999, 146 (2) : 85～92
24 B. Forster, D. Van De Ville, J. Berent, D. Sage, M. Unser. Complex Wavelets for Extended Depth-of-Field: A New Method for the Fusion of Multichannel Microscopy Images. Microscopy Research and Technique, vol. 65, 2004:1-2
25 Forster, B. Van De Ville, D. Berent, J. Extended depth-of-focus for ulti-channel microscopy images: a complex wavelet approach. International Symposium on Biomedical Imaging IEEE, April 2004:660-663
26 JIANG Zhi-guo, SHI Wen-hua, HAN Dong-bing. A wavelet based algorithm for multi-focus micro-image fusion [A]. Third Intemational Conference on Image and graphics(ICIG04)[C], 2004:176-179
27 A.P.Pentladn.Depth of Scene from Depth of Field. Proc.Image Understanding Workshop, 1982:253-259
28 M.Subbarao, G.Natarajan. Depth recovery from blurred edges. Proceedings of the IEEE Computer Socierty Conference on Computer Vision and Pattern Recognition, Ann Arbor,Michigan, June 1988:498-503
29 A.P.Pentladn. A New Sense for Depth of Field. IEEE Transactions on Pattern Analysis and Machine Intellegence,Vol.PAMI-9,NO.4,July 1987:523-531
30 Subbarao. Direct recovery of depth-map. Tech. Report 87-02, Computer Vision and Graphics Laboratory, Dept. of Electrical Engineering, SUNY at Stony Brook, 1987
31 J.Ens, P.Lawrence. A Matrix Based Method for Determining Depth from Focus. Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition, June 1991:600-609
32 J.Ens, P.Lawrence, An Investigation of Methods for Determining Depth from Focus. Proceedings of the IEEE Computer Society Conference on pattern analysis and machine intelligence, February 1993
33 M. Subbarao, T.Wei. Depth from Defocus and Rapid Autofocusing:A practical Approach. Proceedings of the IEEE Computer Socierty Conference on Computer Vision and Pattern Recognition, Champaign, Illinois, June 1992 :773-776
34 T.Wei. Three-dimensional machine vision using image defocus. Ph.D.thesis, Dept.of Electrical Engg, SUNY at Stony Brook, 1994
35 M. Subbarao, G. Surya.Depth from Defocus: A Spatial Domain Approach. International Journal of Computer Vision, Vol. 13, No. 3, 1994:271-294
36 M. Watanabe, S.K. Nayar. Rational Filters for Passive Depth from Defocus. International Journal on Computer Vision. Vol.27, No.3, May 1997:203-225
37 O. Ghita, P. F. Whelan. Real time 3-D estimation using detph from defocus. Proc. Irish Machine Vision and Image Processing Conference, Dublin, 1999:167-181
38 T. Xian, M. Subbarao. Performance evaluation of different depth from defocus(dfd) techniques. In Proc. of SPIE: Two- and Three-Dimensional Methodsfor Inspection and Metrology III (Optics East 2005), October 2005
39 T.Xian. Three-Dimensional Modeling and Autofocusing technology for New Generation Digital Cameras. Ph.D.thesis,Dept.of Electrical Engg, SUNY at Stony Brook,2006
40 Chu, Hongxia Ye, Shujiang. Object Tracking Algorithm Based on CAMSHIFT Algorithm Combinating with Difference in Frame Automation and Logistics. 2007 IEEE International Conference. 2007: 51-55
41 Bradski, G.R. Computer vision face tracking for use in a perceptual user interface. Intel Technol, 1998:1-15.
2吕遐东,黄心汉,王敏等.基于显微图像散焦特征的微操作机器人深度信息提取.机器人. 2003 , 25 (4) : 322-326
3赵新,余斌等.基于系统辨识的显微镜点扩散参数提取方法及应用.计算机学报. 2004 , 27(1) : 140-144
4张建勋,薛大庆,卢桂章等.通过显微图像特征抽取获得微操作目标纵向信息.机器人. 2003 , 25 (4):73-77
5王华,苏显渝.光学离焦三维传感技术的现状与进展.激光杂志,2003, 24(3):1-4
6黄大刚,张建勋,赵新.从平面显微图像中提取三维位置反馈信息方法的研究.仪器仪表学报. 2002, 23(3):230-231
7龚俊锋,徐西鹏.基于聚焦合成的砂轮表面三维重构方法.金刚石与磨料磨具工程, 2006,154(4):14-16
8孙玉秋,田金文等.基于图像金字塔的分维融合算法.计算机应用, 2005,25(5):1064-1075
9谢少荣,彭商贤,赵新等.基于虚拟显微镜技术的微操作工具Z方向定位方法研究.高技术通讯. 2001, 20(2):72-75
10李敏,赵新,卢桂章等.微操作机器人系统拟实环境的实现.机器人. 2001, 23(4):305-310
11王跃宗,刘冲等.微操作系统中微观对像三维立体形状重构研究.计算机辅助设计与图形学学报. 2003,15(6):701-705
12刘莉,姜志国等.光学体视显微图像立体测量系统研究与开发.中国体视学与图像分析,2003,12(8)
13夏明革,何友等.像素级图像融合方法分类与比较.火力与指挥控制. 27(3):161-164
14 G. Ligthart, and F.C.A.Groen. A Comparison of Different Autofocus Algorithms. Proceedings on International Conference on Pattern Recognition, 1982:597-600
15 M Boissenin, J Wedekind, AN Selvan, BP Amavasai. Focus set based reconstruction of micro-objects. Image and Vision Computing, 2004:1-4
16 T.Darrel, K. Wohn. Pyramid based depth from focus. Proc.of IEEE Conference on Computer Vision and Pattern Recognition, June 1988:504-509
17 Shree K. Nayar, Yasuo Nakagawa. Shape from Focus. IEEETransactions on pattern analysis and machine intelligence,Vol. 16, No 8, Aug, 1992: 824-830
18 T.Choi. Shape and Image Reconstruction from Focus. Ph.D.thesis, Dept.of lectrical Engg, SUNY at Stony Brook,1993
19 M. Subbarao and T.Choi. A New Method for Shape from Focus. Proceedings of SPIE Conference, Vol.2046,September 1993
20穆静,杜亚勤等.小波包变换的图像融合技术的研究.西安工业学院学报,2005,25(4):356-358
21熊四昌,杨涌等.基于图像小波分析的显微序列图像合成系统.光学仪器,2006,21(23):10467-10471
22 Chipman, L.J, T.M, Lewis, L.N. Wavelets and image fusion. IEEE Transactions on Image Processing, 1995:248-251
23 Foresti G L, Regazzoni C S. Statistical Mo rpho logical Skeleton for Representing and Coding Noisy Shapes. IEEE P roc. 2V ision Image Signal Process, 1999, 146 (2) : 85～92
24 B. Forster, D. Van De Ville, J. Berent, D. Sage, M. Unser. Complex Wavelets for Extended Depth-of-Field: A New Method for the Fusion of Multichannel Microscopy Images. Microscopy Research and Technique, vol. 65, 2004:1-2
25 Forster, B. Van De Ville, D. Berent, J. Extended depth-of-focus for ulti-channel microscopy images: a complex wavelet approach. International Symposium on Biomedical Imaging IEEE, April 2004:660-663
26 JIANG Zhi-guo, SHI Wen-hua, HAN Dong-bing. A wavelet based algorithm for multi-focus micro-image fusion [A]. Third Intemational Conference on Image and graphics(ICIG04)[C], 2004:176-179
27 A.P.Pentladn.Depth of Scene from Depth of Field. Proc.Image Understanding Workshop, 1982:253-259
28 M.Subbarao, G.Natarajan. Depth recovery from blurred edges. Proceedings of the IEEE Computer Socierty Conference on Computer Vision and Pattern Recognition, Ann Arbor,Michigan, June 1988:498-503
29 A.P.Pentladn. A New Sense for Depth of Field. IEEE Transactions on Pattern Analysis and Machine Intellegence,Vol.PAMI-9,NO.4,July 1987:523-531
30 Subbarao. Direct recovery of depth-map. Tech. Report 87-02, Computer Vision and Graphics Laboratory, Dept. of Electrical Engineering, SUNY at Stony Brook, 1987
31 J.Ens, P.Lawrence. A Matrix Based Method for Determining Depth from Focus. Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition, June 1991:600-609
32 J.Ens, P.Lawrence, An Investigation of Methods for Determining Depth from Focus. Proceedings of the IEEE Computer Society Conference on pattern analysis and machine intelligence, February 1993
33 M. Subbarao, T.Wei. Depth from Defocus and Rapid Autofocusing:A practical Approach. Proceedings of the IEEE Computer Socierty Conference on Computer Vision and Pattern Recognition, Champaign, Illinois, June 1992 :773-776
34 T.Wei. Three-dimensional machine vision using image defocus. Ph.D.thesis, Dept.of Electrical Engg, SUNY at Stony Brook, 1994
35 M. Subbarao, G. Surya.Depth from Defocus: A Spatial Domain Approach. International Journal of Computer Vision, Vol. 13, No. 3, 1994:271-294
36 M. Watanabe, S.K. Nayar. Rational Filters for Passive Depth from Defocus. International Journal on Computer Vision. Vol.27, No.3, May 1997:203-225
37 O. Ghita, P. F. Whelan. Real time 3-D estimation using detph from defocus. Proc. Irish Machine Vision and Image Processing Conference, Dublin, 1999:167-181
38 T. Xian, M. Subbarao. Performance evaluation of different depth from defocus(dfd) techniques. In Proc. of SPIE: Two- and Three-Dimensional Methodsfor Inspection and Metrology III (Optics East 2005), October 2005
39 T.Xian. Three-Dimensional Modeling and Autofocusing technology for New Generation Digital Cameras. Ph.D.thesis,Dept.of Electrical Engg, SUNY at Stony Brook,2006
40 Chu, Hongxia Ye, Shujiang. Object Tracking Algorithm Based on CAMSHIFT Algorithm Combinating with Difference in Frame Automation and Logistics. 2007 IEEE International Conference. 2007: 51-55
41 Bradski, G.R. Computer vision face tracking for use in a perceptual user interface. Intel Technol, 1998:1-15.