基于视频图像信息提取的疲劳驾驶检测技术研究
|
摘要
机动车辆与日俱增,随之而来的车辆交通安全问题也越来越受到社会的广泛关注。调查表明,疲劳驾驶在造成交通事故的危险因素中高居第三位,在死亡交通事故原因中居首位。因此,研制疲劳驾驶检测预警系统,对于避免交通事故,提高交通安全性有着重要意义。
本文在研习了国内外相关研究的基础上,分析对比各种疲劳检测方法,深入研究了基于视觉的非接触式、实时的驾驶疲劳检测方法。以视频图像信息提取和处理理论为基础,研究探讨了基于视觉的疲劳驾驶检测的关键技术,包括人脸检测、眼睛定位与跟踪、眼睛特征提取与状态识别、疲劳状态分析。
首先采用了实时性较好的基于肤色检测的人脸定位方法,将肤色与非肤色区域进行分离,较为准确地定位人脸区域;在关键环节眼睛定位中,选择了积分投影函数与混合投影函数相结合的方法,精确定位了眼睛中心,同时将Kalman滤波与MeanShift算法相结合的眼睛跟踪方法融合到实验中,实现了对序列视频中头部旋转或倾斜情况下的连续眼睛检测和定位;在此基础上对眼睛进行特征提取,主要分析了决定眼睛状态的关键特征,包括虹膜、眼角、上眼睑,并对传统眼睛模型进行了修正,利用上眼睑与下眼睑的高度差来判断眼睛的状态;最后选取目前公认最有效的疲劳分析方法PERCLOS法,结合眨眼频率对疲劳状态进行分析。
基于上述研究所实现的疲劳检测软件系统,能够实时地检测驾驶者的眼睛状态并进行疲劳识别,在实验室环境下获得了较好的实验效果。
With the increasing number of motor vehicles, the problem of traffic safety has become more and more concerned by the society. Research has shown that fatigued driving is one of the most dangerous factors which cause death in traffic accidents. Therefore, a system for monitoring the driver's level of vigilance and alerting the driver when he is fatigued is very important to prevent traffic accidents .
Based on the researches at home and abroad, various methods of fatigue detection are analysed and compared. A non-contact, real-time method based on vision is studied in-depth. Key technologies of the method are presented, including face detection, eyes location and tracking, feature detection and fatigue recogonition.
First of all, a real-time algorithm of face detection based on color is used to separate the color region from non-color region. So the face region is located accurately. Then, in the key process of eyes location, grayness projection algorithm is selected. The integral projection function and hybrid projection function are combined to acquire the eye's center accurately. And a real-time eye tracking algorithm combined with Kalman Filter and Mean Shift is integrated into the experiment to obtain a good effect of eyes tracking.
On the above basis, some features of eyes are detected, including the irises, the eye corners and the upper eyelid. And a new eye model is proposed in the system. According to the model, the height of upper eyelid is detected to show the eyes' status.
Finally, the PERCLOS method which is accepted as the most effective method of fatigue analysis currently is selected. The Eye blink frequency is also picked up to analyse the eyes' status.
The software system based on the above technologies can detect the driver's eyes and recognize his fatigue in real time and the experiment result is very good in the laboratory environment.
本文在研习了国内外相关研究的基础上,分析对比各种疲劳检测方法,深入研究了基于视觉的非接触式、实时的驾驶疲劳检测方法。以视频图像信息提取和处理理论为基础,研究探讨了基于视觉的疲劳驾驶检测的关键技术,包括人脸检测、眼睛定位与跟踪、眼睛特征提取与状态识别、疲劳状态分析。
首先采用了实时性较好的基于肤色检测的人脸定位方法,将肤色与非肤色区域进行分离,较为准确地定位人脸区域;在关键环节眼睛定位中,选择了积分投影函数与混合投影函数相结合的方法,精确定位了眼睛中心,同时将Kalman滤波与MeanShift算法相结合的眼睛跟踪方法融合到实验中,实现了对序列视频中头部旋转或倾斜情况下的连续眼睛检测和定位;在此基础上对眼睛进行特征提取,主要分析了决定眼睛状态的关键特征,包括虹膜、眼角、上眼睑,并对传统眼睛模型进行了修正,利用上眼睑与下眼睑的高度差来判断眼睛的状态;最后选取目前公认最有效的疲劳分析方法PERCLOS法,结合眨眼频率对疲劳状态进行分析。
基于上述研究所实现的疲劳检测软件系统,能够实时地检测驾驶者的眼睛状态并进行疲劳识别,在实验室环境下获得了较好的实验效果。
With the increasing number of motor vehicles, the problem of traffic safety has become more and more concerned by the society. Research has shown that fatigued driving is one of the most dangerous factors which cause death in traffic accidents. Therefore, a system for monitoring the driver's level of vigilance and alerting the driver when he is fatigued is very important to prevent traffic accidents .
Based on the researches at home and abroad, various methods of fatigue detection are analysed and compared. A non-contact, real-time method based on vision is studied in-depth. Key technologies of the method are presented, including face detection, eyes location and tracking, feature detection and fatigue recogonition.
First of all, a real-time algorithm of face detection based on color is used to separate the color region from non-color region. So the face region is located accurately. Then, in the key process of eyes location, grayness projection algorithm is selected. The integral projection function and hybrid projection function are combined to acquire the eye's center accurately. And a real-time eye tracking algorithm combined with Kalman Filter and Mean Shift is integrated into the experiment to obtain a good effect of eyes tracking.
On the above basis, some features of eyes are detected, including the irises, the eye corners and the upper eyelid. And a new eye model is proposed in the system. According to the model, the height of upper eyelid is detected to show the eyes' status.
Finally, the PERCLOS method which is accepted as the most effective method of fatigue analysis currently is selected. The Eye blink frequency is also picked up to analyse the eyes' status.
The software system based on the above technologies can detect the driver's eyes and recognize his fatigue in real time and the experiment result is very good in the laboratory environment.
引文
[1]张灵聪,王正国,朱佩芳等.汽车驾驶疲劳研究综述[J].人类工效学,2003,9(1):38-41
[2]NHTSA.Drowsy driver and automobile Report[EB/OL].http://www.nhlbi.nih.gov/health/prof/sleep/drsy drv.htm,1998
[3]Federal Motor carrier safety Administration.Research and Technology Program Driver Alertness and fatigue Focus Area Summery[EB/OL].http://www.fmcsa.dot.gov/safetyprogs/research.driver fatigue.htm,2004
[4]Jane CS,Jean WW,Bradley.Why Do People Have Drowsy Driving Crashes.AAA Foundation for Traffic Safety[EB/OL].http://www.aaafoundation.org/resources/index.cfm?button=rsrchcat,1999
[5]郭克友.驾驶员疲劳状态视觉监测技术研究[博士论文].吉林大学,2003
[6]毛喆等.汽车驾驶员驾驶疲劳监测技术研究进展[J].中国安全科学学报,2005,15(3):108-112
[7]孙伟,张为公,张小瑞,陈刚.疲劳驾驶预警系统的研究进展[J].汽车电器,2009,1:4-8
[8]J.Empson,Human Brainwaves.The Physiological Significance of the Electroencephalogram.London,U.K.Macmillan,1986(10)
[9]郑培,宋正河,周一鸣.机动车驾驶员驾驶疲劳测评方法的研究状况及发展趋势[J].中国农业大学学报.2001,6(6):101-105
[10]日本开发出防止疲劳驾驶软件,科技之光.http://japan.people.com.cn/2002/12/1/2002121124808.htm.2002年12月1日
[11]Luis M.Bergasa.Real-Time System for Monitoring Driver Vigilance.IEEE Transaction on Intelligent Transportation Systems,2006,7(6):63-77
[12]Daimler Chrysler AG..The Electronic Drawbar.[Online].Available:http://www.daimlerchrysler.com,2001,Jun
[13]Y.Matsumoto and A.Zelinsky.An algorithm for real-time stereo vision implementation of head poses and gaze direction measurements[A].In:Proc.IEEE 4th Int.Conf.Face and Gesture Recognition[C],Grenoble,France,Mar,2000:499-505
[14]S.Boverie,J.M.Leqellec,and A.Hirl.Intelligent systems for video monitoring of vehicle cockpit[A].In:Proc.Int.Congr.And Expo.ITS-Advanced Controls and Vehicle Navigation Systems,Detroit[C],MI,Feb,1998:1-5
[15]W.Shih and J.Liu.A calibration-free gaze tracking technique[A].In:Proe.15th Conf.Patterns Recognition,Barcelona[C],Spain,2000,vol(4):201-204
[16]Q.Ji and X.Yang.Real-time eye,gaze and face pose tracking for monitoring driver vigilance.Real-Time Imaging,8(5),2002:357-377
[17]王荣本,郭克友等.一种基于Gabor小波的驾驶员眼部状态识别方法的研究[J].中国图像图形学报,2003,8A(9):1043-1047
[18]李锋.机器视觉应用技术研究[硕士论文].浙江大学,2003
[19]陈艳琴.关于司机疲劳监测的人眼监测与跟踪研究[硕士论文].中南大学,2004
[20]梁路宏,艾海舟,徐光祐,张钹.人脸检测研究综述[J].计算机学报.2002,25(5):449-458
[21]李刚,高政.人脸检测技术研究与发展[J].计算机与现代文化.2003,4:7-8
[22]Yang M H,Ahuja N.Detecting human faces in color images.In:Proc IEEE Conference on Image Processing,Chicago,1998,127-139
[23]Wei G,Sethi I K.Face detection for image annotation.Pattern Recognition Letters.1999,20(11-13):1313-1321
[24]周杰,卢春雨,张长水,李衍达.人脸自动识别方法综述[J].电子学报,2000,28(4):103-103
[25]D.Chai,K.N.Ngan."Locating facial region of a head-and-shoulders color image".Automatic Face and Gesture Recognition,1998.Proceedings..Third IEEE International Conference on 14-16 April,1998:124-129
[26]Anil K.Jain.Face Detection in Color Images.IEEE TRANS,PAMI.24(5):696-706.2002,5
[27]王荣本,郭克友,刘锐等.驾驶员驾驶行为监测中的面部定位方法的研究[J].公路交通科技.2003,20(2):96-99
[28]汤秋艳.基于肤色分割的彩色图像人脸检测及特征定位[硕士论文].沈阳理工大学,2007
[29]Milan Sonka,Vaclav Hlavac,Roger Boyle.图像处理,分析与机器视觉(第二版)[M].北京:人民邮电出版社.2003
[30]裘伟.一种基于相似度及复杂度的人眼定位算法[J].苏州大学学报.2006,26(6):6-10
[31]Zhang L M,Lenders P.Knowledge-based eye detection for human face recognition[A].In:Proceedings of Knowledge-Based Intelligent Engineering Systems and Allied Technologies[C],Brighton,UK,2000,1:117-120
[32]山世光,高文,陈熙霖.基于纹理分布和变形模板的面部特征提取[J].软件学报, 2001,12(4):570-577
[33]刘沛强.人脸识别中眼睛定位的研究[J].科技咨询,2007,20(1):13-14
[34]Ban,Sang-Woo,Lee,Minho,Yang,Hyun-Seung.A face detection using biologically motivated bottom-up saliency map model and top-down Perception model[J].NeuroeomPuting,2004,2(1):475-480
[35]Yang,M.H,Ahuja N.Detecting human faces in color images[C].In Proceedings of the International Conference on Imaging Processing.Dayton,OH,USA.1998,8:124-131
[36]王忠,胡步发,严世榕.一种改进的对称变换应用于人脸图像眼睛定位[J].计算机应用,2004,24(11):119-121
[37]王磊,莫玉龙,戚飞虎.基于霍夫变换和眼睑弹性模板的眼睛特征提取[J].红外与毫米波学报,1999,18(1):53-60
[38]Kanade T.Picture processing by computer complex and recognition of human faces [Ph.D.Thesis].Kyoto:Kyoto University,1973
[39]Brunelli R,Poggio T.Face recognition:Features versus templates.IEEE Transactions on Pattern Analysis and Machine Intelligence,1993,15(10):1042-1052
[40]Feng GC,Yuen PC.Variance projection function and its application to eye detection for human face recognition.Pattern Recognition Letters,1998,19(9):899-906
[41]耿新,周志华.基于混合投影函数的眼睛定位[J].软件学报,2003,14(8)
[42]Comaniciu D,Ramesh V,Meer P.Kernel-based object tracking.IEEE Trans Pattern Analysis Machine Intelligence.2003,25(5):564-575
[43]周宏仁,敬忠良,王培德等.机动目标跟踪[M].北京,国防工业出版社,1991
[44]宋迎春.动态定位中的卡尔曼滤波研究[博士论文].中南大学,2006
[45]朱胜利.Mean Shift及相关算法在视频跟踪中的研究[硕士论文].浙江大学,2006
[46]边肇棋,张学工等.模式识别.北京,清华大学出版社,2001
[47]韩相军,关永,王雪立.基于DSP的疲劳驾驶实时监测系统研究[J].计算机技术与发展.2006,16(2):47-49
[48]李衡峰.基于综合集成的驾驶疲劳识别[硕士论文].中南大学,2005
[49]Daugman.Biometric personal identification system based on iris analysis.US.Pattent 5291560.1994
[50]Freedman H.,Davis L S.A corner finding algorithm for chain code curves.IEEE Trans.On Computers,1997,26:297-303
[51]Image and Vision Computer[J].1995,(9):695-703
[52]廖常俊等.一种改进的角点检测法[J].仪器仪表用户,2005,12(2):81-83
[53]张坤华,王敬儒,张启衡.多特征复合的角点提取方法[J].中国图像图形学报. 2002,7(1):319-324
[54]陈白帆,蔡自兴.基于尺度空间理论的Harris角点检测[J].中南大学学报,2005,36(5):29-34
[55]魏明慧.基于视觉的疲劳驾驶监测关键技术研究[硕士论文].南京理工大学,2007
[56]陈国勇.汽车安全驾驶中的人眼视线估计[硕士论文],南京理工大学,2006
[57]郑忠龙,杨杰,戈新良,杜春华.一种眼睛特征抽取的新方法[J].上海交通大学报.2005.39(12):1968-1970
[58]Feng G C,Yuen P C.Multi-cues eye detection on gray intensity image[J].Pattern Recognition,2001,34(5):1033-1046
[59]Jian-Gang Wang,Eric Sung,Ronda Venkateswarlu.Eye Gaze Estimation from a Single Image of One Eye.Proceeding of the Ninth IEEE International Conference on Computer Vision.2003
[60]夏芹,宋义伟,朱学峰.基于PERCLOS的驾驶疲劳监控方法进展[J].自动化技术与应用.2008,27(6)
[61]FHWA.PERCLOS:A Valid Physiological Measure of Alertness As Assessde by Phychomotor Vigilnaee.OFFICE OF MOTOR CARREIRS,1998
[2]NHTSA.Drowsy driver and automobile Report[EB/OL].http://www.nhlbi.nih.gov/health/prof/sleep/drsy drv.htm,1998
[3]Federal Motor carrier safety Administration.Research and Technology Program Driver Alertness and fatigue Focus Area Summery[EB/OL].http://www.fmcsa.dot.gov/safetyprogs/research.driver fatigue.htm,2004
[4]Jane CS,Jean WW,Bradley.Why Do People Have Drowsy Driving Crashes.AAA Foundation for Traffic Safety[EB/OL].http://www.aaafoundation.org/resources/index.cfm?button=rsrchcat,1999
[5]郭克友.驾驶员疲劳状态视觉监测技术研究[博士论文].吉林大学,2003
[6]毛喆等.汽车驾驶员驾驶疲劳监测技术研究进展[J].中国安全科学学报,2005,15(3):108-112
[7]孙伟,张为公,张小瑞,陈刚.疲劳驾驶预警系统的研究进展[J].汽车电器,2009,1:4-8
[8]J.Empson,Human Brainwaves.The Physiological Significance of the Electroencephalogram.London,U.K.Macmillan,1986(10)
[9]郑培,宋正河,周一鸣.机动车驾驶员驾驶疲劳测评方法的研究状况及发展趋势[J].中国农业大学学报.2001,6(6):101-105
[10]日本开发出防止疲劳驾驶软件,科技之光.http://japan.people.com.cn/2002/12/1/2002121124808.htm.2002年12月1日
[11]Luis M.Bergasa.Real-Time System for Monitoring Driver Vigilance.IEEE Transaction on Intelligent Transportation Systems,2006,7(6):63-77
[12]Daimler Chrysler AG..The Electronic Drawbar.[Online].Available:http://www.daimlerchrysler.com,2001,Jun
[13]Y.Matsumoto and A.Zelinsky.An algorithm for real-time stereo vision implementation of head poses and gaze direction measurements[A].In:Proc.IEEE 4th Int.Conf.Face and Gesture Recognition[C],Grenoble,France,Mar,2000:499-505
[14]S.Boverie,J.M.Leqellec,and A.Hirl.Intelligent systems for video monitoring of vehicle cockpit[A].In:Proc.Int.Congr.And Expo.ITS-Advanced Controls and Vehicle Navigation Systems,Detroit[C],MI,Feb,1998:1-5
[15]W.Shih and J.Liu.A calibration-free gaze tracking technique[A].In:Proe.15th Conf.Patterns Recognition,Barcelona[C],Spain,2000,vol(4):201-204
[16]Q.Ji and X.Yang.Real-time eye,gaze and face pose tracking for monitoring driver vigilance.Real-Time Imaging,8(5),2002:357-377
[17]王荣本,郭克友等.一种基于Gabor小波的驾驶员眼部状态识别方法的研究[J].中国图像图形学报,2003,8A(9):1043-1047
[18]李锋.机器视觉应用技术研究[硕士论文].浙江大学,2003
[19]陈艳琴.关于司机疲劳监测的人眼监测与跟踪研究[硕士论文].中南大学,2004
[20]梁路宏,艾海舟,徐光祐,张钹.人脸检测研究综述[J].计算机学报.2002,25(5):449-458
[21]李刚,高政.人脸检测技术研究与发展[J].计算机与现代文化.2003,4:7-8
[22]Yang M H,Ahuja N.Detecting human faces in color images.In:Proc IEEE Conference on Image Processing,Chicago,1998,127-139
[23]Wei G,Sethi I K.Face detection for image annotation.Pattern Recognition Letters.1999,20(11-13):1313-1321
[24]周杰,卢春雨,张长水,李衍达.人脸自动识别方法综述[J].电子学报,2000,28(4):103-103
[25]D.Chai,K.N.Ngan."Locating facial region of a head-and-shoulders color image".Automatic Face and Gesture Recognition,1998.Proceedings..Third IEEE International Conference on 14-16 April,1998:124-129
[26]Anil K.Jain.Face Detection in Color Images.IEEE TRANS,PAMI.24(5):696-706.2002,5
[27]王荣本,郭克友,刘锐等.驾驶员驾驶行为监测中的面部定位方法的研究[J].公路交通科技.2003,20(2):96-99
[28]汤秋艳.基于肤色分割的彩色图像人脸检测及特征定位[硕士论文].沈阳理工大学,2007
[29]Milan Sonka,Vaclav Hlavac,Roger Boyle.图像处理,分析与机器视觉(第二版)[M].北京:人民邮电出版社.2003
[30]裘伟.一种基于相似度及复杂度的人眼定位算法[J].苏州大学学报.2006,26(6):6-10
[31]Zhang L M,Lenders P.Knowledge-based eye detection for human face recognition[A].In:Proceedings of Knowledge-Based Intelligent Engineering Systems and Allied Technologies[C],Brighton,UK,2000,1:117-120
[32]山世光,高文,陈熙霖.基于纹理分布和变形模板的面部特征提取[J].软件学报, 2001,12(4):570-577
[33]刘沛强.人脸识别中眼睛定位的研究[J].科技咨询,2007,20(1):13-14
[34]Ban,Sang-Woo,Lee,Minho,Yang,Hyun-Seung.A face detection using biologically motivated bottom-up saliency map model and top-down Perception model[J].NeuroeomPuting,2004,2(1):475-480
[35]Yang,M.H,Ahuja N.Detecting human faces in color images[C].In Proceedings of the International Conference on Imaging Processing.Dayton,OH,USA.1998,8:124-131
[36]王忠,胡步发,严世榕.一种改进的对称变换应用于人脸图像眼睛定位[J].计算机应用,2004,24(11):119-121
[37]王磊,莫玉龙,戚飞虎.基于霍夫变换和眼睑弹性模板的眼睛特征提取[J].红外与毫米波学报,1999,18(1):53-60
[38]Kanade T.Picture processing by computer complex and recognition of human faces [Ph.D.Thesis].Kyoto:Kyoto University,1973
[39]Brunelli R,Poggio T.Face recognition:Features versus templates.IEEE Transactions on Pattern Analysis and Machine Intelligence,1993,15(10):1042-1052
[40]Feng GC,Yuen PC.Variance projection function and its application to eye detection for human face recognition.Pattern Recognition Letters,1998,19(9):899-906
[41]耿新,周志华.基于混合投影函数的眼睛定位[J].软件学报,2003,14(8)
[42]Comaniciu D,Ramesh V,Meer P.Kernel-based object tracking.IEEE Trans Pattern Analysis Machine Intelligence.2003,25(5):564-575
[43]周宏仁,敬忠良,王培德等.机动目标跟踪[M].北京,国防工业出版社,1991
[44]宋迎春.动态定位中的卡尔曼滤波研究[博士论文].中南大学,2006
[45]朱胜利.Mean Shift及相关算法在视频跟踪中的研究[硕士论文].浙江大学,2006
[46]边肇棋,张学工等.模式识别.北京,清华大学出版社,2001
[47]韩相军,关永,王雪立.基于DSP的疲劳驾驶实时监测系统研究[J].计算机技术与发展.2006,16(2):47-49
[48]李衡峰.基于综合集成的驾驶疲劳识别[硕士论文].中南大学,2005
[49]Daugman.Biometric personal identification system based on iris analysis.US.Pattent 5291560.1994
[50]Freedman H.,Davis L S.A corner finding algorithm for chain code curves.IEEE Trans.On Computers,1997,26:297-303
[51]Image and Vision Computer[J].1995,(9):695-703
[52]廖常俊等.一种改进的角点检测法[J].仪器仪表用户,2005,12(2):81-83
[53]张坤华,王敬儒,张启衡.多特征复合的角点提取方法[J].中国图像图形学报. 2002,7(1):319-324
[54]陈白帆,蔡自兴.基于尺度空间理论的Harris角点检测[J].中南大学学报,2005,36(5):29-34
[55]魏明慧.基于视觉的疲劳驾驶监测关键技术研究[硕士论文].南京理工大学,2007
[56]陈国勇.汽车安全驾驶中的人眼视线估计[硕士论文],南京理工大学,2006
[57]郑忠龙,杨杰,戈新良,杜春华.一种眼睛特征抽取的新方法[J].上海交通大学报.2005.39(12):1968-1970
[58]Feng G C,Yuen P C.Multi-cues eye detection on gray intensity image[J].Pattern Recognition,2001,34(5):1033-1046
[59]Jian-Gang Wang,Eric Sung,Ronda Venkateswarlu.Eye Gaze Estimation from a Single Image of One Eye.Proceeding of the Ninth IEEE International Conference on Computer Vision.2003
[60]夏芹,宋义伟,朱学峰.基于PERCLOS的驾驶疲劳监控方法进展[J].自动化技术与应用.2008,27(6)
[61]FHWA.PERCLOS:A Valid Physiological Measure of Alertness As Assessde by Phychomotor Vigilnaee.OFFICE OF MOTOR CARREIRS,1998