数字化细胞微注射机器人显微视觉系统设计及实验研究
|
摘要
由于在微操作器上安装传感器存在困难,在微操作机器人系统中,显微图像被视为唯一的现场信息获取的来源。因此,显微视觉系统的研究成为微操作系统研究的一个重要内容。本文围绕数字化细胞微注射机器人系统提高精度和实现自动化的主题,对其视觉系统进行深入研究,设计并实现了一个显微视觉系统。主要研究内容如下:
在分析数字化细胞微注射机器人系统功能和需求的基础上,选择视觉系统的软件平台和工具,进行硬件结构设计和选择;搭建数字化细胞微注射机器人系统。
视觉系统硬件的程序设计和开发:利用采集卡中二次开发用API函数实现系统的图像采集模块;利用控制箱提供的通讯协议实现微位移驱动机构的自动定位,通过实验测得精度为1.5μm,定位误差在1-2 pixel内。
对含有噪声的细胞微注射过程中的图像进行显微图像预处理,验证了中值滤波、直方图均衡、图像二值化、Canny边缘检测算法的有效性。实验证明:预处理能很好地滤除噪声,突出目标,识别出静态的细胞、微注射针、吸持针三个目标物体。
数字化细胞微注射机器人显微视觉系统的实现,包括微操作工具的自动搜索,细胞圆拟合和质心检测,注射针针尖位置检测,注射针运动检测和跟踪。最后在视觉系统的引导下进行细胞微注射实验,并分析系统中由于图像处理速度而引起的误差。实验证明:视觉系统可实现细胞微注射中快速进针、刺膜、注射及快速退针操作的自动控制。
In the micromanipulation robot system, micro-image is regarded as the only source of obtaining information because it is difficult to install sensors in the micromanipulator. Therefore the study of micro-vision system has become an important topic of the micromanipulation robot system. In order to improve the level of precision and automatization of the cell micro-injection robot system based on the digitalization of microfluids technology, micro-vision system is discussed, designed and realized. The following topics are mainly discussed in this paper.
On the basis of analyzing the function and requirement of the cell micro-injection robot system, the software and hardware of the micro-vision system are selected and presented. A cell micro-injection robot system based on the digitalization of microfluids technology is built.
Programmes of the micro-vision system are designed and developed. First the image capture is realized using API function for secondly development. Second the automatic positioning of the micro-displacement drive system is realized using communication protocol. The experimental results show the system accuracy is 1.5 micron, and position error is in the 1 -2 pixel.
Based on the characteristics of the micro-image, some processings are done, including median arithmetic, histogram equalization arithmetic, two-value arithmetic, Canny edge arithmetic. The experimental results indicate that the yawp is sieved, also the cell, the holding needles and the micropipette are recognized well.
Micro-vision system is achieved. It consists of searching the micromanipulator automatically, gaining the centroid pixel coordinate of the cell, detecting the point of micropipette, detecting and tracking the moving micropipette. Lastly, experiments are carried out with micro-vision system. Also, the error is analyzed. The experimental results indicate that with micro-vision system it can automatically complete the piercing, injecting and withdrawing manipulation of cell micro-injection.
在分析数字化细胞微注射机器人系统功能和需求的基础上,选择视觉系统的软件平台和工具,进行硬件结构设计和选择;搭建数字化细胞微注射机器人系统。
视觉系统硬件的程序设计和开发:利用采集卡中二次开发用API函数实现系统的图像采集模块;利用控制箱提供的通讯协议实现微位移驱动机构的自动定位,通过实验测得精度为1.5μm,定位误差在1-2 pixel内。
对含有噪声的细胞微注射过程中的图像进行显微图像预处理,验证了中值滤波、直方图均衡、图像二值化、Canny边缘检测算法的有效性。实验证明:预处理能很好地滤除噪声,突出目标,识别出静态的细胞、微注射针、吸持针三个目标物体。
数字化细胞微注射机器人显微视觉系统的实现,包括微操作工具的自动搜索,细胞圆拟合和质心检测,注射针针尖位置检测,注射针运动检测和跟踪。最后在视觉系统的引导下进行细胞微注射实验,并分析系统中由于图像处理速度而引起的误差。实验证明:视觉系统可实现细胞微注射中快速进针、刺膜、注射及快速退针操作的自动控制。
In the micromanipulation robot system, micro-image is regarded as the only source of obtaining information because it is difficult to install sensors in the micromanipulator. Therefore the study of micro-vision system has become an important topic of the micromanipulation robot system. In order to improve the level of precision and automatization of the cell micro-injection robot system based on the digitalization of microfluids technology, micro-vision system is discussed, designed and realized. The following topics are mainly discussed in this paper.
On the basis of analyzing the function and requirement of the cell micro-injection robot system, the software and hardware of the micro-vision system are selected and presented. A cell micro-injection robot system based on the digitalization of microfluids technology is built.
Programmes of the micro-vision system are designed and developed. First the image capture is realized using API function for secondly development. Second the automatic positioning of the micro-displacement drive system is realized using communication protocol. The experimental results show the system accuracy is 1.5 micron, and position error is in the 1 -2 pixel.
Based on the characteristics of the micro-image, some processings are done, including median arithmetic, histogram equalization arithmetic, two-value arithmetic, Canny edge arithmetic. The experimental results indicate that the yawp is sieved, also the cell, the holding needles and the micropipette are recognized well.
Micro-vision system is achieved. It consists of searching the micromanipulator automatically, gaining the centroid pixel coordinate of the cell, detecting the point of micropipette, detecting and tracking the moving micropipette. Lastly, experiments are carried out with micro-vision system. Also, the error is analyzed. The experimental results indicate that with micro-vision system it can automatically complete the piercing, injecting and withdrawing manipulation of cell micro-injection.
引文
[1]Koyano K.,Sato T..Micro object handling system with concentrated visual fields and handling skills[C].Proceedings of the 1996 IEEE International Conference on Robotics and Automation.1996,2541-2548
[2]Kallio P.,Zhou Q.,Lind M.,et al.A 3 DOF Piezohydraulic Parallel Micromanipulator[C].Proceedings of the 1998 IEEE International Conference on Robotics&Automation.Leuven,Belgium,May,1998,2:1823-1828
[3]Sano T.,Nagahata H.,Endo H.,et al.A Visual Feedback System for Micromanipulation with Stereoscopic Microscope[C].IEEE Instrumentation and Measurement Technology Conference St.Paul,Minnesota,USA,May 18-21,1998,2,1127-1132
[4]Kasaya T.,Miyazaki H.,Saito S.,Sato T..Micro Object Handling under SEM by Vision-based Automatic Control[C].Proceedings of the 1999 IEEE International Conference on Robotics & Automation,Michigan,May 1999,3,2189-2196
[5]Yu S.,Nelson B.J..Microrobotic Cell Injection[C].Proceedings of the 2001 IEEE International Conference on Robotics & Automation,May,2001:620-625
[6]Jea-Hong Shim,Sung-Yong Cho and Dong-Hyuk Cha.Vision-guided micromanipulation system for biomedical application[C].Proc.of SPIE,2004,5604:98-107
[7]Satoru Sakai,Sachihiro Youoku,Yoshinori Suto,Moritoshi Ando,and Akio Ito.Automated high-throughput microinjection system for floating cells[C].Proc.of SPIE,2005,5699:59-66
[8]Sachihiro Youoku,Yoshinori Suto,Moritoshi Ando,AkioIto.Automated microinjection system for adherent cells[C].Proc.of SPIE-OSA Biomedical Optics,2007,6633:66330S-1-66330S-9
[9]L.Mattos,E.Grant,and R.Thresher.Semi-automated blastocyst microinjection [C].Proceedings of the 2006 IEEE International Conference on Robotics and Automation,Orlando,USA,May 2006:1780-1785
[10]L.Mattos,E.Grant,and R.Thresher.Speeding-up video processing for blastocyst microinjection[C].Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems,Beijing,China,October 2006:5825-5830
[11]L.Mattos,E.Grant,and R.Thresher.New Developments Towards Automated Blastocyst Microinjections[C].Proceedings of the 2007 IEEE International Conference on Robotics and Automation,Roma,Italy,10-14 April 2007:1924-1929
[12]卢桂章,张建勋,赵新.面向生物工程实验的微操作机器人[J].南开大学学报(自然科学),1999,32(3):42-46
[13]谢少荣,彭商贤,赵新,卢桂章.基于虚拟显微镜技术的微操作工具Z方向定位方法研究[J].高技术通讯,2001,11(6):72-75
[14]张建勋,薛大庆,卢桂章,李彬.通过显微图像特征抽取获得微操作目标纵向信息[J].机器人,2001,23(1):73-77
[15]王守杰,宗光华,毕树生.微操作机器人显微视觉系统的研究[J].机器人,1998,20(2):138-142
[16]李旭东,宗光华,毕树生,赵玮.生物工程微操作机器人视觉系统的研究[J].北京航空航天大学学报,2002,28(3):249-252
[17]于波,王立鼎.一种双目分时显微立体成像系统的研制[J].光学精密工程,1997,5(4):49-52
[18]刘冲,徐征,王立鼎,王跃宗.微操作系统的实时显微立体成像研究[J].中国机械工程,2002,13(18):1567-1569
[19]郭阿全,钟辉,洪小雅,席文明,孙道恒.机械手细胞微注射的深度提取及图像数据处理[J].MEMS器件与技术,2007,11:1012-1016
[20]科技导报编辑部.2004年中国重大科学、技术与工程进展[J].科技导报,2005,23(2):58-62
[21]章维一,侯丽雅.微流体数字化的科学与技术问题(Ⅰ):概念、方法和效果[J].科技导报,2005,23(8):4-9
[22]章维一,侯丽雅.微流体数字化的科学与技术问题(Ⅱ):物质数字化及物质能量信息统一数字化概念研究[J].科技导报,2006,24(3):41-47
[23]章维一,侯丽雅.影响流体流动的方法及其装置和应用[P].中国:ZL03152948.8,2006-05-24
[24]章维一,侯丽雅,刘天军.细胞显微注射仪[P].中国:200410065016.7,2006-04-26
[25]陈立国.基于显微视觉的微操作机器人系统的研究[D].哈尔滨:哈尔滨工业大学,2003
[26]安连生.应用光学[M].北京:北京理工大学出版社,2000
[27]张树霖.近场光学显微镜及其应用[M].北京:科学出版社,2000
[28]http://www.sipat.com/index-c.htm
[29]周严,侯丽雅,章维一.波形和参数在线可编程的功率电源及其应用[J].电测与仪表.2004,41(457):17-20
[30]段怡安.图像处理在人脸识别系统中的应用[D].南京:南京理工大学,2002
[31]刘连臣,卢桂章,翁春华等.微操作机器人及其显微视觉伺服控制系统的研究[J].高技术通讯,2001,6:56-58
[32]龚声蓉,刘纯平,王强等.数字图像处理与分析[M].北京:清华大学出版社,2006
[33]姚敏等.数字图像处理[M].北京:机械工业出版社,2006
[34]陈天华.数字图像处理[M].北京:清华大学出版社,2007
[35]何斌等.Visual C++数字图像处理[M].北京:人民邮电出版社,2001
[36]夏良正.数字图像处理[M].南京:东南大学出版社,1999
[37](日)谷口庆治.数字图像处理(基础篇)[M].北京:科学出版社,2001
[38]李俊山,李旭辉.数字图像处理[M].北京:清华大学出版处,2007
[39]邓菁,郑永果.基于形态学的图像二值化方法[J].计算机工程,2002,28(11):205-206
[40]沈庭芝,方子文.数字图像处理及模式识别[M].北京:北京理工大学,1998
[41]J.Canny.A computational approach to edge detection[J].IEEE Trans.on Pattern Analysis and Machine Intelligence.1986,8(6):679-698
[42]Henri Maitre.现代数字图像处理[M].北京:电子工业出版社.2006
[43]Milan Sonka,Vaclav Hlavac,Roger Boyle.图像处理、分析与机器视觉(第二版)[M].北京:人民邮电出版社.2003
[44]温遇华.微操作机器人自动作业的几项关键技术研究[D].天津:南开大学,2004
[45]李彩花,王听,王学影.边缘检测技术在工件中心位置识别中的应用[J].传感器技术,2005,24(4):73-78
[46]郑南宁.计算机视觉和模式识别[M].北京:国防工业出版社.1998
[47]David A.Forsyth,Jean Ponce等.计算机视觉:一种现代方法[M].北京:电子工业出版社.2004
[48]毕树生,宗光华.微操作机器人视觉伺服控制系统的误差分析[J].光电工程,2004,31(2):31-33,43
[2]Kallio P.,Zhou Q.,Lind M.,et al.A 3 DOF Piezohydraulic Parallel Micromanipulator[C].Proceedings of the 1998 IEEE International Conference on Robotics&Automation.Leuven,Belgium,May,1998,2:1823-1828
[3]Sano T.,Nagahata H.,Endo H.,et al.A Visual Feedback System for Micromanipulation with Stereoscopic Microscope[C].IEEE Instrumentation and Measurement Technology Conference St.Paul,Minnesota,USA,May 18-21,1998,2,1127-1132
[4]Kasaya T.,Miyazaki H.,Saito S.,Sato T..Micro Object Handling under SEM by Vision-based Automatic Control[C].Proceedings of the 1999 IEEE International Conference on Robotics & Automation,Michigan,May 1999,3,2189-2196
[5]Yu S.,Nelson B.J..Microrobotic Cell Injection[C].Proceedings of the 2001 IEEE International Conference on Robotics & Automation,May,2001:620-625
[6]Jea-Hong Shim,Sung-Yong Cho and Dong-Hyuk Cha.Vision-guided micromanipulation system for biomedical application[C].Proc.of SPIE,2004,5604:98-107
[7]Satoru Sakai,Sachihiro Youoku,Yoshinori Suto,Moritoshi Ando,and Akio Ito.Automated high-throughput microinjection system for floating cells[C].Proc.of SPIE,2005,5699:59-66
[8]Sachihiro Youoku,Yoshinori Suto,Moritoshi Ando,AkioIto.Automated microinjection system for adherent cells[C].Proc.of SPIE-OSA Biomedical Optics,2007,6633:66330S-1-66330S-9
[9]L.Mattos,E.Grant,and R.Thresher.Semi-automated blastocyst microinjection [C].Proceedings of the 2006 IEEE International Conference on Robotics and Automation,Orlando,USA,May 2006:1780-1785
[10]L.Mattos,E.Grant,and R.Thresher.Speeding-up video processing for blastocyst microinjection[C].Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems,Beijing,China,October 2006:5825-5830
[11]L.Mattos,E.Grant,and R.Thresher.New Developments Towards Automated Blastocyst Microinjections[C].Proceedings of the 2007 IEEE International Conference on Robotics and Automation,Roma,Italy,10-14 April 2007:1924-1929
[12]卢桂章,张建勋,赵新.面向生物工程实验的微操作机器人[J].南开大学学报(自然科学),1999,32(3):42-46
[13]谢少荣,彭商贤,赵新,卢桂章.基于虚拟显微镜技术的微操作工具Z方向定位方法研究[J].高技术通讯,2001,11(6):72-75
[14]张建勋,薛大庆,卢桂章,李彬.通过显微图像特征抽取获得微操作目标纵向信息[J].机器人,2001,23(1):73-77
[15]王守杰,宗光华,毕树生.微操作机器人显微视觉系统的研究[J].机器人,1998,20(2):138-142
[16]李旭东,宗光华,毕树生,赵玮.生物工程微操作机器人视觉系统的研究[J].北京航空航天大学学报,2002,28(3):249-252
[17]于波,王立鼎.一种双目分时显微立体成像系统的研制[J].光学精密工程,1997,5(4):49-52
[18]刘冲,徐征,王立鼎,王跃宗.微操作系统的实时显微立体成像研究[J].中国机械工程,2002,13(18):1567-1569
[19]郭阿全,钟辉,洪小雅,席文明,孙道恒.机械手细胞微注射的深度提取及图像数据处理[J].MEMS器件与技术,2007,11:1012-1016
[20]科技导报编辑部.2004年中国重大科学、技术与工程进展[J].科技导报,2005,23(2):58-62
[21]章维一,侯丽雅.微流体数字化的科学与技术问题(Ⅰ):概念、方法和效果[J].科技导报,2005,23(8):4-9
[22]章维一,侯丽雅.微流体数字化的科学与技术问题(Ⅱ):物质数字化及物质能量信息统一数字化概念研究[J].科技导报,2006,24(3):41-47
[23]章维一,侯丽雅.影响流体流动的方法及其装置和应用[P].中国:ZL03152948.8,2006-05-24
[24]章维一,侯丽雅,刘天军.细胞显微注射仪[P].中国:200410065016.7,2006-04-26
[25]陈立国.基于显微视觉的微操作机器人系统的研究[D].哈尔滨:哈尔滨工业大学,2003
[26]安连生.应用光学[M].北京:北京理工大学出版社,2000
[27]张树霖.近场光学显微镜及其应用[M].北京:科学出版社,2000
[28]http://www.sipat.com/index-c.htm
[29]周严,侯丽雅,章维一.波形和参数在线可编程的功率电源及其应用[J].电测与仪表.2004,41(457):17-20
[30]段怡安.图像处理在人脸识别系统中的应用[D].南京:南京理工大学,2002
[31]刘连臣,卢桂章,翁春华等.微操作机器人及其显微视觉伺服控制系统的研究[J].高技术通讯,2001,6:56-58
[32]龚声蓉,刘纯平,王强等.数字图像处理与分析[M].北京:清华大学出版社,2006
[33]姚敏等.数字图像处理[M].北京:机械工业出版社,2006
[34]陈天华.数字图像处理[M].北京:清华大学出版社,2007
[35]何斌等.Visual C++数字图像处理[M].北京:人民邮电出版社,2001
[36]夏良正.数字图像处理[M].南京:东南大学出版社,1999
[37](日)谷口庆治.数字图像处理(基础篇)[M].北京:科学出版社,2001
[38]李俊山,李旭辉.数字图像处理[M].北京:清华大学出版处,2007
[39]邓菁,郑永果.基于形态学的图像二值化方法[J].计算机工程,2002,28(11):205-206
[40]沈庭芝,方子文.数字图像处理及模式识别[M].北京:北京理工大学,1998
[41]J.Canny.A computational approach to edge detection[J].IEEE Trans.on Pattern Analysis and Machine Intelligence.1986,8(6):679-698
[42]Henri Maitre.现代数字图像处理[M].北京:电子工业出版社.2006
[43]Milan Sonka,Vaclav Hlavac,Roger Boyle.图像处理、分析与机器视觉(第二版)[M].北京:人民邮电出版社.2003
[44]温遇华.微操作机器人自动作业的几项关键技术研究[D].天津:南开大学,2004
[45]李彩花,王听,王学影.边缘检测技术在工件中心位置识别中的应用[J].传感器技术,2005,24(4):73-78
[46]郑南宁.计算机视觉和模式识别[M].北京:国防工业出版社.1998
[47]David A.Forsyth,Jean Ponce等.计算机视觉:一种现代方法[M].北京:电子工业出版社.2004
[48]毕树生,宗光华.微操作机器人视觉伺服控制系统的误差分析[J].光电工程,2004,31(2):31-33,43