An image processing method for changing endoscope direction based on pupil movement

详细信息    查看全文

• 作者：Yang Cao ; Satoshi Miura ; Quanquan Liu ; Yo Kobayashi ; Kazuya Kawamura…
• 关键词：Non ; rigid face tracking ; Single ; port endoscopic surgery ; Master–slave structure ; Image processing ; Double ; screw ; drive mechanism ; Activation threshold
• 刊名：ROBOMECH Journal
• 出版年：2016
• 出版时间：December 2016
• 年：2016
• 卷：3
• 期：1
• 全文大小：2,859 KB
• 参考文献：1.ASCRS, Laparoscopic surgery—what is it?, Available from: http://​www.​fascrs.​org/​patients/​treatments_​and_​screenings/​laparoscopic_​surgery/​
  2.Horgan S, Vanuno D (2001) Robots in laparoscopic surgery. J Laparoendosc Adv Surg Tech A 11:415–419CrossRef
  3.Matern U, Koneczny S (2007) Safety, hazards and ergonomics in the operating room. Surg Endosc 21:1965–1969CrossRef
  4.Jihad K, George-Pascal H, Raj G, Mihir D, Monish A, Raymond R, Courtenay M, Inderbir G (2008) Single-Port laparoscopic surgery in urology: initial experience. Urology 71:3–6CrossRef
  5.Yasunaga T, Hashizume M, Kobayashi E, Tanoue K, Akahoshi T, Konishi K, Yamaguchi S, Kinjo N, Tomikawa M, Muragaki Y, Shimada M, Maehara Y, Dohi Y, Sakuma I, Miyamoto S (2003) Remote-controlled laparoscope manipulator system NaviotTM, for endoscopic surgery. Int Congress Ser 1256(2003):678–683CrossRef
  6.Intuitive Surgical Inc. [homepage on the Internet]. Available from: http://​www.​intuitivesurgica​l.​com
  7.Meyer A, Oleynikov D (2013) Surgical robotics, Mastery of Endoscopic and Laparoscopic Surgery, Vol 2, 4th edn. p 62–71
  8.Kraft B, Jager C, Kraft K, Leibl B, Bittner R (2004) The AESOP robot system in laparoscopic surgery: increased risk or advantage for surgeon and patient? Surg Endosc 18:1216–1223CrossRef
  9.ViKY EP, available from: http://​www.​endocontrol-medical.​com/​viky_​ep.​php
  10.Kolvenbach R, Schwiez E, Wasillijew S, Miloud A, Puerschel A, Pinter L (2004) Total laparoscopically and robotically assisted aortic aneurysm surgery: a critical evaluation. J Vasc Surg 39:771–776CrossRef
  11.Ubeda A, Ianez E, Azorın J (2011) Wireless and portable EOG-based interface for assisting disabled people. IEEE/ASME Transact Mechatron 16(5):870–873CrossRef
  12.Noonan D, Mylonas G, Shang J, Payne C, Darzi A, Yang G (2010) Gaze contingent control for an articulated mechatronic laparoscope. Proceedings of the 2010 3rd IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, The University of Tokyo, Tokyo, Japan, 26–29 September 2010
  13.Autorino R, Cadeddu JA, Desai MM, Gettman M, Gill IS, Kavoussi LR, Lima E, Montorsi F, Richstone L, Stolzenburg JU, Kaouk JH (2010) Laparoendoscopic single-site and natural orifice transluminal endoscopic surgery in urology: a critical analysis of the literature. Eur Assoc Urol 59(1):26–45CrossRef
  14.Hubens G, Coveliers H, Balliu L, Ruppert M, Vaneerdeweq W (2003) A performance study comparing manual and robotically assisted laparoscopic surgery using the da Vinci system. Surg Endosc Other Intervent Tech 17(10):1595–1599CrossRef
  15.Kobayashi Y, Tomono Y, Sekiguchi Y, Watanabe H, Toyoda K, Konishi K, Tomikawa M, Ieiri S, Tanoue K, Hashizume M, Fujie GM (2010) The international journal of medical robotics and computer assisted surgery. Int J Med Robotics Comput Assist Surg 6:454–464CrossRef
  16.Kobayashi Y, Sekiguchi Y, Noguchi T, Liu Q, Oguri S, Toyoda K, Konishi K, Uemru M, Ieiri S, Tomikwa M, Ohdaira T, Hashizume M, Fujie GM (2015) The international journal of medical robotics and computer assisted surgery. Int J Med Robotics Comput Assist Surg 11:235–246CrossRef
  17.Rassweiler J, Gözen A, Frede T, Teber D (2011) Laparoscopy vs. robotics: ergonomics—does it matter? Robotics Genitourin Surg 2011:63–78CrossRef
  18.Ohno T, Mukawa N (2004) A free-head, simple calibration, gaze tracking system that enables gaze-based interaction. ETRA 2004, Eye Tracking Research and Applications Symposium, p 115–122
  19.Hill B (1996) Retina identification, Biometrics, p 123–141
  20.Liu Q, Kobayashi Y, Zhang B, Noguchi T, Takahashi Y, Nishio Y, Cao Y, Ieiri S, Toyoda K, Uemura M, Tomikawa M, Hashizume M, Fujie M G (2014) Development of a smart surgical robot with bended forceps for infant congenital esophageal atresia surgery. 2015 IEEE International Conference on Robotics and Automation, Hong Kong, p 2430–2435
  21.Emami S, Ievgen K, Saragih J (2012) Non-rigid face tracking, mastering openCV with practical computer vision projects. Chapter 6:189–233
  22.Gower J (1975) Generalized procrustes analysis. Phychometrika 40(1):33–51CrossRef MathSciNet MATH
  23.Robinson D (1964) The mechanics of human saccadic eye movement. J Pysiol 1964(174):245–264CrossRef
  24.Marchand E, Chaumette F (2002) Virtual visual servoing: a framework for real-time augmented reality. Comput Graph Forum 21(3):289–297CrossRef
  25.Daniilidis K, Krauss C, Hanse M, Sommer G (1998) Real-time tracking of moving objects with an active camera. Real-Time Imaging 4(1):3–20CrossRef
  26.Marescaux J, Leroy J, Gagner M, Rubino F, Mutter D, Vix M, Butner SE, Smith MK (2001) Transatlantic robot-assisted telesurgery. Nature 413:379–380CrossRef
  27.Rempel D, Willms K, Anshel J, Jaschinski W, Sheedy J (2007) The effects of visual display distance on eye accommodation, head posture, and vision and neck symptoms. Hum Factors J Hum Factors Ergon Soc 49(5):830–838CrossRef
  28.Barbato G, Ficca G, Muscettola G, Fichele M, Beatrice M, Rinaldi F (2000) Diurnal variation in spontaneous eye-blink rate. Psychiatry Res 93(2):145–151CrossRef
  29.Hashizume M (2005) Fundamental training for safe endoscopic surgery, Innovative Medical Technology, Graduate School of Medical Science Kyushu University, p 49 (in Japaeses)
  30.Donnez J (2007) Human assistance in laparoscopic surgery, Atlas of operative laparoscopy and hystereoscopy, 3rd edn, p 409
  
• 作者单位：Yang Cao (1)
  Satoshi Miura (1)
  Quanquan Liu (2)
  Yo Kobayashi (1)
  Kazuya Kawamura (3)
  Shigeki Sugano (1)
  Masakatsu G. Fujie (1)
  
  1. Graduate School of Creative Science and Engineering, Waseda University, 3-4-1 Ohkubo, Shinjuku-ku, Tokyo, 169-8555, Japan
  2. Graduate School of Advanced Science and Engineering, Waseda University, 2-2 Wakamatsu-cho, Shinjuku-ku, Tokyo, 162-8480, Japan
  3. Graduate School and Faculty of Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba, 263-8522, Japan
  
• 刊物类别：Robotics and Automation; Mechatronics; Artificial Intelligence (incl. Robotics); Control; Computatio
• 刊物主题：Robotics and Automation; Mechatronics; Artificial Intelligence (incl. Robotics); Control; Computational Intelligence;
• 出版者：Springer International Publishing
• ISSN：2197-4225

文摘

Increased attention has been focused on laparoscopic surgery because of its minimal invasiveness and improved cosmetic properties. However, the procedure of laparoscopic surgery is considerably difficult for surgeons, thus paving the way for the introduction of robotic technology to reduce the surgeon’s burden. Thus, we have developed a single-port surgery assistive robot with a master–slave structure that has two surgical manipulators and a sheath manipulator for the alteration of endoscope direction. During the development of the surgical robotic system, achieving intuitive operation is very important. In this paper, we propose a new laparoscope manipulator control system based on the movement of the pupils to enhance intuitive operability. We achieve this using a webcam and an image processing method. After the pupil movement data are obtained, the master computer transforms these data into an output signal, and then the slave computer receives and uses that signal to drive the robot. The details of the system and the pupil detection procedure are explained. The aim of the present experiment is to verify the effectiveness of the image processing method applied to the alteration of endoscope direction control system. For this purpose, we need to determine an appropriate pupil motion activation threshold to begin the sheath manipulator’s movement. We used four kinds of activation threshold, measuring the time cost of a particular operation: to move the image of the endoscope to a specific target position. Moreover, we identified an appropriate activation threshold that can be used to determine whether the endoscope is moving. Keywords Non-rigid face tracking Single-port endoscopic surgery Master–slave structure Image processing Double-screw-drive mechanism Activation threshold