虚实融合交互系统辅助脑卒中后失语症康复训练
|
摘要
脑卒中后失语症是一个普遍的致残性疾病.研究表明,虚拟现实及实物交互技术对该疾病的康复训练具有重要意义.然而现有的计算机辅助言语康复系统很少能将这2类技术有效结合并发挥它们各自的优势,且没有对言语训练素材进行合理编排以适应实际的康复训练需求.提出了一种虚实融合辅助的言语康复训练系统,构建了一个"超市购物"的虚拟场景,基于语义网络理论编排言语训练素材,实现了对操作实物的实时跟踪和识别;并将其映射到虚拟购物场景中,使得脑卒中后失语症患者可以通过对实物的操作去完成在虚拟空间的购物任务;患者在虚实融合辅助的言语康复训练系统中的任务完成时间和行为表现结果表明了系统的可用性,问卷调查结果表明,系统得到了康复治疗师的普遍认可,患者和治疗师在用户访谈中提出的多项建议对未来在这一方面的研究工作具有指导意义.
Aphasia after stroke is a common disease. Studies indicated that virtual reality and tangible interactions can effectively assist rehabilitation of aphasia after stroke. However, few existing computer-assisted language training systems integrated the two methods and took the advantages of both successfully. Further, language training materials were not reasonably arranged to adapt the scenarios of aphasia rehabilitation in previous works. This paper presented a physical and virtual integrated language training system(PVLT) for the rehabilitation of aphasia after stroke. The developed training system composed of a virtual supermarket where language training materials are arranged based on a semantic network theory. Physical objects were tracked and recognized, and then integrated with the virtual supermarket to enable users to navigate the supermarket by manipulating physical objects. User performance of task completion time shows the usability of the system. Results of a questionnaire survey show that the system is appreciated by language therapists. The summarized user suggestions provide valuable insights to future research in this area.
Aphasia after stroke is a common disease. Studies indicated that virtual reality and tangible interactions can effectively assist rehabilitation of aphasia after stroke. However, few existing computer-assisted language training systems integrated the two methods and took the advantages of both successfully. Further, language training materials were not reasonably arranged to adapt the scenarios of aphasia rehabilitation in previous works. This paper presented a physical and virtual integrated language training system(PVLT) for the rehabilitation of aphasia after stroke. The developed training system composed of a virtual supermarket where language training materials are arranged based on a semantic network theory. Physical objects were tracked and recognized, and then integrated with the virtual supermarket to enable users to navigate the supermarket by manipulating physical objects. User performance of task completion time shows the usability of the system. Results of a questionnaire survey show that the system is appreciated by language therapists. The summarized user suggestions provide valuable insights to future research in this area.
引文
[1]Gao Surong.Aphasia[M].2nd ed.Beijing:Peking University Medical Press,2006(in Chinese)(高素荣.失语症[M].2版.北京:北京大学医学出版社,2006)
[2]Xia Zhilan,Ouyang Duoli.Rehabilitation therapy for language impairment caused by brain damage[J].Chinese Journal of Rehabilitation.1997,12(2):72-73(in Chinese)(夏芝兰,欧阳多利.脑损害所致语言障碍的康复治疗[J].中国康复,1997,12(2):72-73)
[3]Lányi C S,Geiszt Z,Magyar V.Using IT to inform and rehabilitate aphasic patients[J].Informing Science,2006,9:163-179
[4]Galliers J,Wilson S,Muscroft S,et al.Accessibility of 3Dgame environments for people with aphasia:an exploratory study[C]//Proceedings of the 13th International ACM SIGAC-CESS Conference on Computers and Accessibility.New York:ACM Press.2011:139-146
[5]Edmans J A,Gladman J,Walker M,et al.Mixed reality environments in stroke rehabilitation:development as rehabilitation tools[J].International Journal on Disability and Human Development,2007,6(1):39-46
[6]Rybarczyk Y,Fonseca J.Tangible interface for a rehabilitation of comprehension in aphasic patients[J].International Journal of River Basin Management,2011,9(3):247-256
[7]Song K,Lee S,Pyun S B,et al.Comparative study of tangible tabletop and computer-based training interfaces for cognitive rehabilitation[C]//Proceedings of the International Conference on Universal Access in Human-Computer Interaction.Heidelberg:Springer International Publishing,2016:414-424
[8]Pflügler J,Schlegel A,Jean-Baptiste E,et al.Using human-computer interface for rehabilitation of activities of daily living(ADL)in stroke patients:lessons from the first prototype[C]//Proceedings of the 2nd International Conference on NeuroRehabilitation.Heidelberg:Springer International Publishing,2014,7:629-636
[9]Wairagkar M,McCrindle R,Robson H,et al.Combined language and motor therapy for brain injury patients[C]//Proceedings of the 3rd Workshop on ICTs for Improving Patients Rehabilitation Research Techniques.New York:ACMPress,2015:131-134
[10]Collins A M,Loftus E F.A spreading-activation theory of semantic processing[J].Readings in Cognitive Science,1988,82(6):126-136
[11]Kim Y H,Ko M H,Parrish T B,et al.Reorganization of cortical language areas in patients with aphasia:a functional MRIstudy[J].Yonsei Medical Journal,2002,43(4):441-445
[12]Fellbaum C.WordNet[M]//The Encyclopedia of Applied Linguistics.Oxford:Blackwell Publishing Ltd,2012:231-243
[13]Nikolova S,Boyd-Graber J,Cook P R.The design of ViVA:a mixed-initiative visual vocabulary for aphasia[C]//Proceedings of the CHI’09 Extended Abstracts on Human Factors in Computing Systems.New York:ACM Press,2009:4015-4020
[14]Nikolova S,Boyd-Graber J,Fellbaum C,et al.Better vocabularies for assistive communication aids:connecting terms using semantic networks and untrained annotators[C]//Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility.New York:ACM Press,2009:171-178
[15]Ma X J,Nikolova S,Cook P R.W2ANE:when words are not enough:online multimedia language assistant for people with aphasia[C]//Proceedings of the 17th ACM International Conference on Multimedia.New York:ACM Press,2009:749-752
[16]Ma X J,Fellbaum C,Cook P R.A multimodal vocabulary for augmentative and alternative communication from sound/image label datasets[C]//Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies.Stroudsburg:Association for Computational Linguistics,2010:62-70
[17]Zhou Liang.A study of word association and semantic priming in young and old people[D].Nanjing:Nanjing Medical University,2008(in Chinese)(周亮.青年人和老年人词汇联想反应及其语义启动效应的研究[D].南京:南京医科大学,2008)
[18]Sun Li.The effect of semantic navigation strategy on aphasia naming therapy[D].Nanjing:Nanjing Medical University,2010(in Chinese)(孙丽.语义导航策略对失语症命名治疗效应的研究[D].南京:南京医科大学,2010)
[19]Gao Minxing,Jiang Zhongli,Lin Feng,et al.Observation on therapeutic effects of semantic navigation training implemented by family members on improvement of speech function in patients with aphasia[J].China Journal of Rehabilitation Medicine,2011,26(5):419-423(in Chinese)(高敏行,江钟立,林枫,等.家属实施语义导航训练法改善失语症患者言语功能的疗效观察[J].中国康复医学杂志,2011,26(5):419-423)
[20]Shapiro S C.The SNePS semantic network processing system[M]//Associative Networks.New York:Academic Press,1979:179-203
[21]Sowa J F.Semantic networks[M].2nd ed.Hoboken:Wiley,1992
[22]Besl P J,Mckay N D.A method for registration of 3-Dshapes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1992,14(2):239-256
[23]Bouaziz S,Tagliasacchi A,Pauly M.Sparse iterative closest point[J].Computer Graphics Forum,2013,32(5):113-123
[24]Huang Chaodian,Chen Hui,Peng Xiaolan,et al.Tangible user interface for speech rehabilitation training[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(9):1560-1570(in Chinese)(黄朝殿,陈辉,彭晓兰,等.实物交互在言语康复训练中的应用[J].计算机辅助设计与图形学学报,2016,28(9):1560-1570)
[2]Xia Zhilan,Ouyang Duoli.Rehabilitation therapy for language impairment caused by brain damage[J].Chinese Journal of Rehabilitation.1997,12(2):72-73(in Chinese)(夏芝兰,欧阳多利.脑损害所致语言障碍的康复治疗[J].中国康复,1997,12(2):72-73)
[3]Lányi C S,Geiszt Z,Magyar V.Using IT to inform and rehabilitate aphasic patients[J].Informing Science,2006,9:163-179
[4]Galliers J,Wilson S,Muscroft S,et al.Accessibility of 3Dgame environments for people with aphasia:an exploratory study[C]//Proceedings of the 13th International ACM SIGAC-CESS Conference on Computers and Accessibility.New York:ACM Press.2011:139-146
[5]Edmans J A,Gladman J,Walker M,et al.Mixed reality environments in stroke rehabilitation:development as rehabilitation tools[J].International Journal on Disability and Human Development,2007,6(1):39-46
[6]Rybarczyk Y,Fonseca J.Tangible interface for a rehabilitation of comprehension in aphasic patients[J].International Journal of River Basin Management,2011,9(3):247-256
[7]Song K,Lee S,Pyun S B,et al.Comparative study of tangible tabletop and computer-based training interfaces for cognitive rehabilitation[C]//Proceedings of the International Conference on Universal Access in Human-Computer Interaction.Heidelberg:Springer International Publishing,2016:414-424
[8]Pflügler J,Schlegel A,Jean-Baptiste E,et al.Using human-computer interface for rehabilitation of activities of daily living(ADL)in stroke patients:lessons from the first prototype[C]//Proceedings of the 2nd International Conference on NeuroRehabilitation.Heidelberg:Springer International Publishing,2014,7:629-636
[9]Wairagkar M,McCrindle R,Robson H,et al.Combined language and motor therapy for brain injury patients[C]//Proceedings of the 3rd Workshop on ICTs for Improving Patients Rehabilitation Research Techniques.New York:ACMPress,2015:131-134
[10]Collins A M,Loftus E F.A spreading-activation theory of semantic processing[J].Readings in Cognitive Science,1988,82(6):126-136
[11]Kim Y H,Ko M H,Parrish T B,et al.Reorganization of cortical language areas in patients with aphasia:a functional MRIstudy[J].Yonsei Medical Journal,2002,43(4):441-445
[12]Fellbaum C.WordNet[M]//The Encyclopedia of Applied Linguistics.Oxford:Blackwell Publishing Ltd,2012:231-243
[13]Nikolova S,Boyd-Graber J,Cook P R.The design of ViVA:a mixed-initiative visual vocabulary for aphasia[C]//Proceedings of the CHI’09 Extended Abstracts on Human Factors in Computing Systems.New York:ACM Press,2009:4015-4020
[14]Nikolova S,Boyd-Graber J,Fellbaum C,et al.Better vocabularies for assistive communication aids:connecting terms using semantic networks and untrained annotators[C]//Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility.New York:ACM Press,2009:171-178
[15]Ma X J,Nikolova S,Cook P R.W2ANE:when words are not enough:online multimedia language assistant for people with aphasia[C]//Proceedings of the 17th ACM International Conference on Multimedia.New York:ACM Press,2009:749-752
[16]Ma X J,Fellbaum C,Cook P R.A multimodal vocabulary for augmentative and alternative communication from sound/image label datasets[C]//Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies.Stroudsburg:Association for Computational Linguistics,2010:62-70
[17]Zhou Liang.A study of word association and semantic priming in young and old people[D].Nanjing:Nanjing Medical University,2008(in Chinese)(周亮.青年人和老年人词汇联想反应及其语义启动效应的研究[D].南京:南京医科大学,2008)
[18]Sun Li.The effect of semantic navigation strategy on aphasia naming therapy[D].Nanjing:Nanjing Medical University,2010(in Chinese)(孙丽.语义导航策略对失语症命名治疗效应的研究[D].南京:南京医科大学,2010)
[19]Gao Minxing,Jiang Zhongli,Lin Feng,et al.Observation on therapeutic effects of semantic navigation training implemented by family members on improvement of speech function in patients with aphasia[J].China Journal of Rehabilitation Medicine,2011,26(5):419-423(in Chinese)(高敏行,江钟立,林枫,等.家属实施语义导航训练法改善失语症患者言语功能的疗效观察[J].中国康复医学杂志,2011,26(5):419-423)
[20]Shapiro S C.The SNePS semantic network processing system[M]//Associative Networks.New York:Academic Press,1979:179-203
[21]Sowa J F.Semantic networks[M].2nd ed.Hoboken:Wiley,1992
[22]Besl P J,Mckay N D.A method for registration of 3-Dshapes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1992,14(2):239-256
[23]Bouaziz S,Tagliasacchi A,Pauly M.Sparse iterative closest point[J].Computer Graphics Forum,2013,32(5):113-123
[24]Huang Chaodian,Chen Hui,Peng Xiaolan,et al.Tangible user interface for speech rehabilitation training[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(9):1560-1570(in Chinese)(黄朝殿,陈辉,彭晓兰,等.实物交互在言语康复训练中的应用[J].计算机辅助设计与图形学学报,2016,28(9):1560-1570)