基于机器学习方法的人脸表情识别研究
|
摘要
对人际交往而言,除了自然语言、肢体动作以外,面部表情(facial expression)也是一种独特而重要的信息传递方式。面部表情往往能够传达很多语言所不能传达的东西。面部表情可以分为宏观表情和微观表情,宏观表情是人们在常规状态下展示出的面部信号:而微观表情则是短暂的、潜在的表情,这种表情通常发生在人们有意或无意的隐藏或压制他们的内心情感时。面部运动不仅反映了情感,也反映出了其他人类情感,如社会活动和心理变化。
人脸表情识别,是采用机器和软件对人脸表情信息进行处理,提取其特征并进行分类的过程。其目的是使计算机能够根据人的表情信息,推断人的心理状态,从而实现人机之间的智能交互。一般而言,表情识别系统主要有四个基本部分组成：表情图像获取、表情图像预处理、表情特征提取和表情分类识别。人类能迅速地理解它人的丰富表情,并为之做出回应。如果人类的这一特性能为计算机交互所利用,那将极大的推动人机交互问题的解决。由于人的丰富的面部表情很难用低级可视特征进行精确的描述,传统的面部识别系统通常存在人与计算机之间的语义鸿沟的问题。
人脸表情分析、检测和识别是计算机和模式识别领域一项重要且富有挑战性的研究课题。无论是从图像中定位出脸部,还是在图像中识别出少数有效的孤立物体,这些都是非常复杂的。现有的人脸表情识别系统旨在能发觉和理解人类的情感状态,但这些表情通常是在在不受干扰的情况下提取出来的。对观察者而言,自然的面部表情可以暗示出一种特定的情绪,然而,在实际生活中,人们可能会先模仿了别人的表情,却在最后表达出一种完全不同的情绪。为了避免结果受到影响,我们需要一种智能的计算机系统,它能根据从脸部提取到丰富的面部信息,分析表情的真实变化。尽管现有研究成果已经表明了脸部表情、情感信号和内心状态之间的关系,但是怎样让计算机更好地提取人类面部信息仍旧是一个亟待解决的问题。
过去的几十年中,大量的研究人员致力于人脸表情分析的研究,他们对面部表情的研究可大致分为：表情预处理、面部特征提取、情感分类。通常其输入数据是静态图像或视频序列。针对人脸表情的研究的主要内容是对人类的情感状态进行检测、解释以及分类,并使用恰当的方式处理这些情感信息。现有的面部表情分析方法大致可分为两类：基于几何特征的方法和基于视觉特征的方法。典型的基于视觉特征的方法是动态文理识别的方法；而基于几何特征的方法包括面部成分的形状和位置特征、面部特征点的位置等。这两种方法的区别主要在于特征的提取与描述方法不同。几何特征通常只考虑特征点的运动,而忽略了皮肤纹理变化提供的丰富信息、；另一方面,基于视觉特征的方法则容易受到光照和个体差异的影响。视觉特征的方法的优势在于该方法会生成细节信息,获得更好的识别精度,简单且性能良好。有大量的研究工作都属于这一类别,且极具实际意义。
本文首先对人脸表情识别研究的相关工作进行了总结,从面部表情描述、面部表情识别框架、特征提取和表示、分类方法几个方面,对该领域的研究现状进行了回顾。
(一)面部表情描述
对表情研究可追溯到查尔斯·罗伯特·达尔文在1872年出版的《人类与动物的感情表达》。达尔文追溯了人类特征的起源,如：紧闭的嘴唇、紧蹙的眉毛、眼泪和呻吟等特征。这些研究表明,人类表情可被分为：愤怒、害怕、开心、讨厌、悲伤以及惊奇等。达尔文的研究构成了现代人脸表情识别系统的理论基础。
计算机技术的发展极大地推动了人脸表情识别研究。1978年,Suwa等首先提出了人脸表情识别系统,1990年Mase和Pentland做出了对表情识别系统的开创性的工作。自此以后,大量的研究人员使用计算机系统从事这方面的研究。人脸表情是由于脸部肌肉的运动而产生的,从而导致了面部特征的变化。为度量面部表情的重要性,需要确定面部运动的位置以及其强度。面部表情的强度可以通过面部特征的几何变化进行度量。通常,面部表情可使用三个参数进行描述：起始点(Onset)、最大点(Apex)、结束点(Offset)。一些研究人员研究了如何自动地计算面部表情的起始点和结束点。为了捕获人的面部表情,需要对面部表情进行精确定义。面部运动编码系统(FACS)是目前广泛使用的面部运动标记系统,该系统将面部表情的变化与产生表情的肌肉的运动联系在一起。运动单元(AU)是最小的可视面部动作。面部运动编码系统提供了一种客观的方法描述面部表情,且该系统可以发现与情感相关的新的运动模式。近年来,基于FACS的研究表明：面部运动可以区分以较高的精度区别说真话的测试对象与说假话的测试对象。总体来说,FACS给许多实际应用提供了一个通用的解决方案。但该方法需要人工对实验数据进行标记,且人工不可能对动态表情进行标记。在需要进行实时反馈的应用场景中,FACS通常不适用。
与使用FACS描述面部特征细节不同的是,一些识别系统使用了基于主观标记表情原型的描述方法。主观表示用于推断面部表情背后的隐含意义,如：感情或个性等。通常,情感原型表示会根据面部表情的强度变化而变化。这种方法的主要问题在于它不能依据现有的表情子集来表示所有可能的情感类别。
(二)自动面部表情分析系统框架
自动面部表情分析能根据面部特征判断个人的特征。通常,人脸表情识别系统可以看作一个单独的表情识别系统或作为人脸识别系统的一个模块。本文提出了一个面部表情分析系统的框架描述。图像获取和人脸识别是该框架的重要组成部分。目前,很多方法都可以从静态图像中进行人脸检测。早起的大多数检测方法只能检测人脸的正面。2004年,Viola和Jonesy提出了一种基于AdaBoos学习的人脸检测算法,在进行人脸和背景分离方面,该算法被认为是最快最有效的方法。Dalal等应用HOG描述子的进行了人脸检测,实验表明,该描述子的效果优于其它已有特征。
(三)特征提取和表示
特征表示阶段需要找到最恰当的方式表示面部对象,从而进行表情识别。通常,特征表示需要将人脸的像素数据转换为基于形状、运动、颜色、纹理、空间信息等高级对象表示的数据。
Pantic团队提出了经典的基于几何特征的特征表示方法；Tian等则对眼睛、眉毛、嘴唇以及其他特征进行形状建模来提取特征。Bartlet等则提出了经典的基于视觉特征的表情特征表示方法。
尽管研究人员已经提出了不同的方法表示面部表情的特征,但这一问题仍需要进行进一步的研究。目前,小波分析是一种流行的多分辨率分析方法。小波用于将单个的输入信号分解成多个更简单、更易于处理和解释的信号,同时,基于频域的表示方法充分利用了感兴趣的区域,同时也进行了降维。文献[95]使用Radon变换和傅里叶变换进行人脸检测；文献[96]则使用基于粒子群优化算法的Random变换,以及PCA和LDA技术进行人脸检测。文献[97]基于Random变换和小波变换,提出了一种新的人脸识别的框架。
(四)分类
分类是人脸表情分析系统的最后阶段。常用的分类方法有基于神经网络的方法、支撑向量机的方法、线性判别分析的方法等、基于马尔科夫模型的方法。其中,基于神经网络的表情识别方法的识别性能优异。有一些文献使用神经网络进行表情分类。
表情是心灵的窗口,而表情与行为也有密切联系。我们可以通过人脸表情预测或理解它人的行为,因此,对计算机来说,人脸表情识别是一种有效地与人交流的方式；通过理解人脸信息,计算机能对人的特征或情感做出判断。考虑到人脸对象以及获取人脸对象的设备的多样性,需要一种预处理机制解决对象特征的全局归一性问题。对计算机来说,具有明显边界的面部对象比边界模糊的面部对象更容易识别。而边界问题主要是由姿态、噪音或光照问题引起的。因此,我们需要对图像进行特征表示,可以是图像密度、颜色、文本或这些特征的组合。
现有的人脸识别算法,在单独应用时,都难以达到实际应用需要的识别精度。因此,本文试图在现有各种算法的基础上,致力于提高识别精度的研究。本文的目标是深入考察现有识别算法的技术特点和应用特征,通过引入新的处理过程、优化模型和参数、融合方法,提高识别计算的有效性。
本文的主要工作包括：分析了图像增强对识别性能的影响,通过对噪声数据进行分析,得到了图像噪声干扰识别特征提取的潜在趋势,提出了一种针对表情特征描述的相似性增强方法,使用图像增强技术获取图像的边缘特征,提取图像的关键特征并减少图像噪声,为提高表情对象分类的有效性提供条件。针对人脸表情的变化特点,本文提出了一种基于小波变换(DWT)的图像分割以及统计分析方法,成功地提取人脸表情特征,供后续过程进行识别。大部分自然图像都包含了丰富的方位信息,如垂直或水平边界。DWT有效地消除了噪声,保留了图像边缘的细节信息。运用方位信息,我们实现了低内存、低计算复杂度的基于DWT的特征提取方法。本研究不仅为经典DWT提供了一种应用方案,也为特征提取提供了新的解决办法。在小波变换之前,本文使用了去噪算法对人脸图像进行去噪处理,然后进行小波变换,最后重构图形并进行识别。本文深入研究了分水岭转换计算方法,提出了一种使用了非刚性配准原则的计算模式,有效解决了形状和脸部组成成分的分解问题。论文还使用主成分分析法对图像的形状特征进行降维处理,最后基于神经网络进行分类。
本文的主要研究内容涵盖了表情识别的主要方面,包括面部表情表示和特征提取、面部表情分割的方法、基于滤波反投影的纹理分析、基于小波提升的纹理分析
(一)面部表示和特征提取
本部分主要讨论如何进行图像数据预处理,它是后序算法的重要基础。首先描述了图像的预处理和增强过程,并使用低级图像处理算法进行处理。本文提出的降噪过程,综合考虑了低噪声的原型对象以及亮度连续的人脸表面,从而在下一步操作之前,突出了感兴趣的对象；在此基础上提取特征进行对象分类。本文的方法一方面解决了计算维度过高的问题,同时,将图像投向低维坐标系,使得从人脸表情识别系统提取出的面部特征保持高感知质量。我们的核心思想是,跟踪面部肌肉动作,突出感兴趣的目标对象,降低了光照、噪音的影响。
技术上,本文通过基于形态学的变换和分水岭变换,较好地突出了感兴趣的目标对象的形状。在不同尺度下,本文增强了亮度值较大的像素点与暗像素点之间差异。通过使用变换操作符,不仅减少了光照和噪声的隐形,而且将图像的隐含特征块聚集在一起。本文方法的新颖之处是基于在更亮处和黑色图像区域增强对比度,用结构化像素和图像细节代替所选的像素,从而能够基于图像尺寸和分水岭细节知识,将面部特征与背景分割开来,突出了感兴趣的区域,如眼睛、鼻子和嘴。增强的面部特征是使用主成分分析法提取的,并送入神经网络用于分类。
(二)面部表情分割
本部分主要研究如何将二维面部图像分割为光照不变、且具有最小表情变化和旋转的图像块。本文提出了一种基于小波变换的增强top-hat变换操作以及bottom-hat变换操作,该方法可以量化分割算法对面部特征的感知能力,通过重建一系列的投影,重新在一个对象中组织内部结构信息。在汇总每条线的响应之前,通过采用傅里叶切片定理过滤出已提取的增强轮廓,用来估计脸部形状。该方法不仅不受旋转、照明和噪声的影响,而且可以利用已提取的不完整数据重建图像。本文通过离散小波变换和离散余弦变换对比实验,分析了提取有效特征的效果。我们证明,由于图像中的低频分量比高频分量视觉效果更显著,所以提取到的低维特征向量是训练神经网络分类器的重要数据。
为评测本文算法的有效性和实用性,我们使用了日本女性面部表情数据库(JAFFE).为了从每副图像中获取感兴趣的数据区域,使用本文的方法首先对数据进行预处理。在对象分割阶段,对象被看成具有密度结构的表面,本文使用分水岭分割自动地将面部表情特征从背景从分离出来,并高亮度的显示刚兴趣的区域,如：眼睛、眉毛、鼻子、嘴巴等区域,以便进行后序的机器学习。
本文测试了基于PCA和基于神经网络的分水岭分割的算法结果,结果表明,在JAFFE数据库上的识别精度达到98%。
(三)基于滤波反投影的纹理分析
面部表情和皮下肌肉的一种或多种运动有关,但不同的观察者看到这些表情时,可能有不同的理解。为避免产生有歧义的输出,本文研究了基于滤波反向投影的非线性转换面部描述子来估计转换参数,转换参数为后序的离散小波变换提供输入数据。Radom变换可以捕获图像的方位特征,DWT变换将图像转换为小波子空间,从而分离出有意义的数据,分离出来的数据特征向量可用于神经网络分类器。
选择JAFFE作为实验数据库,在实验阶段,本文测试了Radon投影的效果、形态逻辑操作预处理的效果、DWT方法的性能以及DCT-PCA域中滤波反向投影的效果。实验结果表明：识别精度主要处决于对输入图像的姿态、光照和面部变化的处理。为识别输入图像,对DCT系数进行PCA变换,计算训练向量中具有最大变化的特征向量。训练阶段使用了136个特征向量,测试阶段使用了70个特征向量。然后使用神经网络训练PCA过程的输出。
实验结果表明：用于消除噪声和不规则变化的形态学开操作和闭操作增强了有利于表情识别的低频成分；同时,滤波反向投影有效地增强了面部数据的提取。显然,这两种方法都有利于提取面部数据中的可视低频成分,从而提高表情识别的精度。
(四)基于小波提升的纹理分析
本部分主要利用小波提升方法和主成分分析法分析面部表情。本文提出了一种利用高通和低通滤波器的相似性的方法,以实现更快的小波变换,从而提升解决方案的有效性。我们注意到了利用非线性高通与低通滤波器的相似性,本文在提升方案中加入了中值滤波处理,利用它在压缩信号方面的潜力,即它所拥有的两种相似结构的计算能力,能够根据压缩完的一半数据样本来预测出另一半数据样本。整个转换过程分为三个阶段,这三个阶段不断的重复执行,直到预测完所有数据样本。像其他的数字图像处理一样,减少高频分量不会给图像带来视觉上的影响。
许多之前的研究结果中强调的旋转、光照和噪音等影响系统性能的因素对本文的算法不会产生影响。该系统的最后一步使用开始的两步提供的信息,来准确识别面部表情(高兴,悲伤,中性,恐惧,厌恶,愤怒和惊讶)。本文对不同的算法进行了对比实验,结果表明,这些方法是可行的,而且实验表明,本文提出的解决方案,增强了程序效能,改进了神经网络对面部表情识别的表现。为了分类脸部特征提取,我们使用JAFEE和PICS数据库的图片对神经网络进行训练。实验证明,本文方法在减少噪音、修改其他不规则的地方以及克服传统方法的缺点上,都体现出明显的有效性,从而对人脸表情识别系统提供了另一种解决方案。
人机交互设备在日常生活中的应用日渐增多,相关技术也在不断的发展,但人脸表情识别的问题仍旧没有得到解决。我们拥有的计算机设备的存储能力在不断的提高,这也刺激了我们需要更多的与其它人或事物保持联系,更多地了解他们的信息。因此,越来越多的智能设备出现在我们的日常生活中,但这也带来了一个问题,即：人和智能设备之间的交流,我们需要更完善的方式来填补人机交互之间的语义鸿沟。为了能建设更友好的人机交互系统,使用人脸表情进行有效地交流成为一种必然的方式。这意味着我们需要捕获人脸对象、记录人脸行为并且利用已有的信息进行分析和决策。本文针对人脸表情识别的问题,提出了一些有用的解决方案。
总体来说,本文的主要创新如下：
(1)将图像分割成有意义的同构块是图像分析和机器视觉中的关键问题。使用分水岭进行图像数据分割或对图像进行形态学操作,是将图像对象表示成与形状有关的特征。本文提出了一种基于Top-hat变换预处理方法,能够有效降低光照、噪音的影响。通过对top-hat和bottom-hat进行增强,并应用在分水岭转化中,以此提取有效的面部特征进行表情识别。我们发现,Top-hat变换更适于提取基于暗背景的明亮对象提取；而bottom-hat则适用于基于亮背景的暗对象提取。在实际的应用中,暗图像特征和亮图像特征均存在于不同尺度的图像中。如果能提取各种尺度下的图像暗特征和亮特征,并应用于之后的分析,那图像增强的效果将大大提高。所以,本文方法致力于捕获这些特征,通过增大不同尺度下图像亮区域和暗区域的对比,有效减少了光照与噪声信号的干扰,增强了有利于进行识别的对象的基本特征。
(2)使用中值滤波器进行小波提升,使得图像的方位特征中的非垂直或非水平的线性边界得以最大化。该方法充分利用了高通滤波器和低通滤波器的相似性,加快小波变换的实现。本文的方法证明,首先使用形态滤波器减少由于光照和噪音影响带来,然后在小波提升中使用中值滤波器,是一条在消除噪音的同时保留图像中的有用信息的有效手段。
(3)本文提出了一种提高表情识别系统的精确性和鲁棒性的方法,基于滤波反投影方法重构对象的内部结构信息,再利用一系列投影进行数学重构,增强了表情识别中的基本特征。图像的投影形成特征向量,对该向量进行Radon转换提取图像灰度；再使用反投影生成向量,近似表示原始对象的形状。为了避免直接投影的缺陷,使用傅里叶中的切片定理过滤光强分布。使用以上的小波变换方法,进行降维和图像去噪,得到了更好的图像表示方法。在此上述表示框架下,小的图像系统表示噪声,大的图像系数表示重要的图像特征。特别地,离散小波变换用于把图像对象压缩成更小更方便管理的数据,从而能从少量的数据中提取有意义的图像信息、。我们从特征向量中提取出对光照和噪音不变的脸部特征,并进行神经网络的训练。最后,基于结果图像的逆小波变化,提供了具有本质特征的重构图像,并以此作为进行神经网络分类器的训练数据。
(4)研究了反投影过滤器对离散余弦变换的影响。在反投影过滤之前,本文对图像进行了形态学开操作和闭操作,并使用Radon变换进行转换。在此变换中,DCT变换用于提取旋转不变特征。低频率的切片成分在图像中比高频成分更具有视觉意义,因此,在DCT变换中,丢弃高频系数,并量化剩余的系数。通过DCT变换的应用,将输入序列表示为基于权重余弦序列的和。其结果系数用于描述信号,从而极大的减少数据量,同时也保留了图像重要信息。为进一步减少数据信息,使用主成分分析提取图像的特征。通过提取面部数据的重要特征,保留低阶主成分,丢弃高阶成分,PCA不仅减小图像大小,减少图像数据量,从而完成图像的降维。最后,使用前馈反向传播神经网络进行训练,从而完成表情识别。
人脸表情分析在人机交互中起着至关重要的作用。表情识别与人脸面部的运动变化有关,它提供了丰富的关于人的情感、内涵及其它内在状态的信息。尽管目前仍旧没有关于“情感”一词的精确定义,但人类情感的存在是毫无疑问的,而且它是我们的日常生活的重要组成部分。人类能够理解“情感”,并能针对它人的“情感”做出相应的反应,正因为如此,人类的之间的交互活动才显得如此丰富多彩。本文根据静态数据,解决了人脸表情识别的度量的问题。首先根据数据库进行人脸对象检测,然后进行关键点的面部表情特征提取并建立模型。计算机阅读面部表情和数据提取的能力至关重要,因为这决定了计算机是否能做出正确的回应,正确的表情读取是表情分类正确与否的关键。一个性能优越的表情系统需要处理面部识别、特征提取和分类等问题,且其结果与个人的年龄、周围环境、身体姿态等外部因素均无关。今后,我们将使用更多的标准数据库对本文的算法进行测试,并进一步提高算法的适应性。同时,人脸表情识别系统还需要捕获并识别更复杂多变的表情对象,这些是我们今后进一步研究的方向。
For several decades our options for interacting with the digital world has been limited to use of mouse, keyboards and joysticks. This objects limit machine's ability to respond to human needs. As a result they have exposed an existing gap between human and machines creating room for renewed interest in affective technology. Affective computing refers to development of system and devices that recognize, interpret, process, and simulate human affects. It brings together individuals from diverse backgrounds like engineering, computer science, psychology, cognitive science, neuroscience, sociology, education, psychophysiology, value-centered design and more to push for boundaries on what can be achieved to improve human affective experience. Emotion is fundamental to human experience; it influences our perception such as learning, communication and decision-making. However, many technologists have largely ignored emotion and create often frustrating experience for people, in part because affect has been misunderstood and is hard to measure.
As machines and people begin to co-exist and cooperatively share a variety of tasks, we see a trend that is shifting from text, to full-body video-conferencing as virtual communication grow rapidly in availability and complexity. Even as this richness of communication options improve our ability to converse with others not available at the precise moment, the fact that something is missing continue to plague users of current technologies presenting a need for affective communication channels between humans and machines. Humans are experts in expressing, recognizing and distinguishing facial expressions without much effort yet for computers this is an uphill task.
To be effective in the human world, machines must respond to human emotional states. In real life situations a lot of communications that use facial expressions need no words at all to explain the situation at hand. Many people do a great deal of inference from perceived facial expressions as conveyed by statements like "You seem happy today." Facial expressions can give a true picture about one's feeling; while you may say that you are feeling fine, one look at your face may tell people otherwise. These expressions make more than a statement about a person's emotional state and mostly occur to reinforce or contradict what has been said. Automatic facial expression recognition aim at perceiving and understanding emotional states of humans based on information found in the face. These expressions reveal a part of the feeling inside a person's private life that could be extracted without intrusion making the ability to understand them more interesting.
Naturally facial expressions may suggest certain emotions visually to an observer while in reality they express a totally different emotion resulting from some proportion of the expression resembling the other. To avoid this confusing outcome we need a well-designed state of the art smart human machine interface that interpret facial behavioral change giving an insight to a variety of information that could be derived from a face. Automatic facial expression recognition is a technique that could be harnessed to solve this problem. It is considered a high-end task in computer-vision and range in complexity from locating the face in images to recognizing isolated objects belonging to a few well-established classes. Our research is built upon this theories that advance basic understanding of affect and its role in human experience. We believe it is for the same reason that researchers are making every attempt at understanding relation that exist between affective signals, mental states and how they can be extracted and tapped to bridge the gap that separate man from machines.
Despite their efforts computers still remain disconnected from the human world. Computer systems are not intelligent enough to pay attention to people and their surrounding environments. This raises need to build systems with perceptual intelligence that can realize what aspects of the surrounding environment are important and use it to interpret who, where, what, when and why of the situation. This way machines become intelligent enough to adapt their behavior to suit human needs rather than humans adapting to theirs. The answer to these questions lie in having an automated measurement system that is able to detect a human face, preprocess, extract features and classify objects in different categories. Facial recognition accuracy depends heavily on how input images have been compensated for pose, illumination and facial expressions.
The aim of this thesis is to analyze and extract behavior referenced by facial features that could be used to convey user emotions or internal state of a person in an effort to solve human machine problem. Our goal is to design a robust, high performance facial expression recognition system that can work in real-time situation with ability to observe, interpret, and generate affect features that react appropriately to natural human behavior. Building such systems is no easy task, first is due to various unpredictable facial variations that are further complicated by external environmental conditions. In addition is the difficulty in choosing suitable characterization of facial feature descriptors capable of extracting discriminative facial information given the influence of illumination, pose among other factors. Facial variations that arise from pose, age, gender, race and occlusion, also exert profound influence on the robustness of facial systems. Facial muscular assume the appearance passion people want to experience therefore a suitable facial feature descriptor would largely determine its performance giving characteristic upon which people can make conclusion regarding the person, ideas or status. Secondly, the contextual events that signify facial expressions and emotional states still remain ambiguous and vague to computers.
Our computation approach addresses the curse of dimension, mapping the image into low dimension coordinate system that preserves perceptual quality of objects enabling feature extraction. The approach entails highlighting objects of interest with aid computer vision technologies where we map and track facial muscle movement, decode and interpret these movements having leveled illumination and eliminated noise effects before feature extraction and classification. A major part of our contribution is to determine optimal ways of combining different algorithms in an attempt to improve facial recognition accuracy. We have developed a system for evaluating and extracting appropriate facial features to be used in measuring facial expression recognition problem.
Previous studies have used texture and geometric features for expression recognition, however few of them conduct and explore performance differences between features arising from image quality. The first key step is to recognize a human face with acceptable levels of false positives this has been achieved appropriately. Following this, enhancement is carried out to provide better quality data for purposes of machine interpretation before feature extraction. For emotion recognition, the extraction of meaningful patterns from the gathered data must be found. A practical approach, used in this thesis is simulation of emotions using image processing, computer vision and machine learning techniques to enable process facial expression detection enrich and facilitate interactivity between human and machine, giving rise to labels (angry, fear, sad, surprise, neutral, happy and disgust).
The first algorithm is based on watershed segmentation an approach that accurately replicate the shape pattern of objects most needed in machine learning. Segmentation is fundamental to image analysis; it entails division of image into regions of similar attributes aimed at discovering objects of interest. To eliminate drawbacks associated with watershed segmentation we adopt contrast enhancement between brighter and black image regions at the scale corresponding to different scales of structuring elements and image details based on morphological tophat and bottomhat transform operators. By incorporating this knowledge watershed segmentation is used to separate facial features from the background, highlighting regions of interest eyes, nose and mouth required for subsequent high level machine learning. Enhanced facial features are extracted using principle component analysis, a small set of eigenvectors with top eigenvalues are used to build principal components. Each principle component is considered a feature representing a point in high dimensional space. Image features obtained are used to train the neural network from this a recognition rate of98%was acieved.
Using a different approach we quantify machine perception of facial features based on internal structural information found within an object by mathematically reconstructing it from a series of projections. Radon transform was used to capture a series of line integrals across the image at different angles, capturing directional local features lying within the object. The extracted intensity profiles were filtered by applying the Fourier Slice Theorem before contributions from each line of response were summed and used to arrive at an estimate of the facial shape. The method is invariant to rotations, lighting and noise in addition it is possible to reconstruct the image even when presented with incomplete data. To determine the most efficient feature extraction technique experiments were carried out separately using the wavelet transform and discrete cosine transform.
Specifically, Haar discrete wavelet transform was used to compress image objects to a smaller more manageable data that facilitate extract enhanced facial features. Filtered back projection is used to reorganize internal structural information within objects by mathematically reconstructing it from a series of projections. These set of images were projected at180degrees and low frequency components extracted to form significant data used to train the Neural Network (NN) classifier. Experiments were carried out to assess validity and efficiency of this algorithm. The results reveal non-enhanced data gave a recognition rate of97.2%while enhanced data gave99%accuracy in determination of emotional state from facial expression recognition. While an approach that combine discrete cosine transform and principal component analysis used125projections to give a recognition rate of97.79%for non-enhanced data and enhanced data achieved98.99%accuracy in determination of emotional state from facial expression recognition.
In an alternative method adaptive wavelet lifting scheme was used to extract facial features for expression recognition. The lifting scheme makes optimal use of similarities between high and low pass filters to achieve a faster implementation of the wavelet transform. To take advantage of non-linear filters we incorporated median filters to utilize its potential at compressing signals these results from the fact that the two algorithms share a similar structure. Facial images like any other digital image contain highly correlated data and its high pass component can be discarded without much effect on the visual quality of the image. For this purpose Haar lifting scheme combined with principle component analysis were used to exploit directional correlation between adjacent pixels. A recognition rate of99.4%was achieved on JAFEE database and97%on PICS databases respectively. This was a more improved performance compared to98%and98.4%for PCA and DWT-PCA on the same images from JAFFE database. From this results feasibility of algorithms are demonstrated and proved effective at minimizing noise and other irregularities, demerits associated with conventional methods providing an alternative perspective towards facial expression recognition problem. Based on experimental analysis carried out we reveal that emotions in machines can be associated with abstract states empowering them to respond to people's needs.
The motivation behind this research a rose from the unobtrusive nature the face uses to communicate things that are difficult to embrace with written or spoken words. In addition its potential applications are many to mention a few is that such systems help deter crime, fraud and streamline business processes to save critical resources. Currently an increasing number commercial applications are either using or actively considering use of facial recognition methods as witnessed in fields like medicine and education while others in need of this technology are clinical psychology, psychiatry, neurology, pain assessment, lie detection, criminology and multimodal human computer interface. A lot is yet to be achieved in relation to interpreting facial messages and how they are produced by the neuro-muscular system. We believe that recognition of human facial expression using computers is key to promoting these fields. A system that can perform these operations accurately and in real time would form a big step in bridging the gap that exists between man and machines. This probably explains the reason why a human face is not only at the center of security concern but also a sudden renewed interest in this field.
The goal of this research was to apply image analysis and machine learning techniques to examine, extract and measure facial expression. Using this theoretical background we have examined face expression in detail, in an effort to understand and capitalize on those features of the face that can help distinguish different classes of facial expressions, increasing its significance as a problem solving method that would ultimately lead to its increased adoption in many domains. In particular, this thesis provides evidence that use of enhancement in facial feature extraction not only improves recognition rate of facial expressions but also improves the scalability and robustness of the resulting system.
人脸表情识别,是采用机器和软件对人脸表情信息进行处理,提取其特征并进行分类的过程。其目的是使计算机能够根据人的表情信息,推断人的心理状态,从而实现人机之间的智能交互。一般而言,表情识别系统主要有四个基本部分组成：表情图像获取、表情图像预处理、表情特征提取和表情分类识别。人类能迅速地理解它人的丰富表情,并为之做出回应。如果人类的这一特性能为计算机交互所利用,那将极大的推动人机交互问题的解决。由于人的丰富的面部表情很难用低级可视特征进行精确的描述,传统的面部识别系统通常存在人与计算机之间的语义鸿沟的问题。
人脸表情分析、检测和识别是计算机和模式识别领域一项重要且富有挑战性的研究课题。无论是从图像中定位出脸部,还是在图像中识别出少数有效的孤立物体,这些都是非常复杂的。现有的人脸表情识别系统旨在能发觉和理解人类的情感状态,但这些表情通常是在在不受干扰的情况下提取出来的。对观察者而言,自然的面部表情可以暗示出一种特定的情绪,然而,在实际生活中,人们可能会先模仿了别人的表情,却在最后表达出一种完全不同的情绪。为了避免结果受到影响,我们需要一种智能的计算机系统,它能根据从脸部提取到丰富的面部信息,分析表情的真实变化。尽管现有研究成果已经表明了脸部表情、情感信号和内心状态之间的关系,但是怎样让计算机更好地提取人类面部信息仍旧是一个亟待解决的问题。
过去的几十年中,大量的研究人员致力于人脸表情分析的研究,他们对面部表情的研究可大致分为：表情预处理、面部特征提取、情感分类。通常其输入数据是静态图像或视频序列。针对人脸表情的研究的主要内容是对人类的情感状态进行检测、解释以及分类,并使用恰当的方式处理这些情感信息。现有的面部表情分析方法大致可分为两类：基于几何特征的方法和基于视觉特征的方法。典型的基于视觉特征的方法是动态文理识别的方法；而基于几何特征的方法包括面部成分的形状和位置特征、面部特征点的位置等。这两种方法的区别主要在于特征的提取与描述方法不同。几何特征通常只考虑特征点的运动,而忽略了皮肤纹理变化提供的丰富信息、；另一方面,基于视觉特征的方法则容易受到光照和个体差异的影响。视觉特征的方法的优势在于该方法会生成细节信息,获得更好的识别精度,简单且性能良好。有大量的研究工作都属于这一类别,且极具实际意义。
本文首先对人脸表情识别研究的相关工作进行了总结,从面部表情描述、面部表情识别框架、特征提取和表示、分类方法几个方面,对该领域的研究现状进行了回顾。
(一)面部表情描述
对表情研究可追溯到查尔斯·罗伯特·达尔文在1872年出版的《人类与动物的感情表达》。达尔文追溯了人类特征的起源,如：紧闭的嘴唇、紧蹙的眉毛、眼泪和呻吟等特征。这些研究表明,人类表情可被分为：愤怒、害怕、开心、讨厌、悲伤以及惊奇等。达尔文的研究构成了现代人脸表情识别系统的理论基础。
计算机技术的发展极大地推动了人脸表情识别研究。1978年,Suwa等首先提出了人脸表情识别系统,1990年Mase和Pentland做出了对表情识别系统的开创性的工作。自此以后,大量的研究人员使用计算机系统从事这方面的研究。人脸表情是由于脸部肌肉的运动而产生的,从而导致了面部特征的变化。为度量面部表情的重要性,需要确定面部运动的位置以及其强度。面部表情的强度可以通过面部特征的几何变化进行度量。通常,面部表情可使用三个参数进行描述：起始点(Onset)、最大点(Apex)、结束点(Offset)。一些研究人员研究了如何自动地计算面部表情的起始点和结束点。为了捕获人的面部表情,需要对面部表情进行精确定义。面部运动编码系统(FACS)是目前广泛使用的面部运动标记系统,该系统将面部表情的变化与产生表情的肌肉的运动联系在一起。运动单元(AU)是最小的可视面部动作。面部运动编码系统提供了一种客观的方法描述面部表情,且该系统可以发现与情感相关的新的运动模式。近年来,基于FACS的研究表明：面部运动可以区分以较高的精度区别说真话的测试对象与说假话的测试对象。总体来说,FACS给许多实际应用提供了一个通用的解决方案。但该方法需要人工对实验数据进行标记,且人工不可能对动态表情进行标记。在需要进行实时反馈的应用场景中,FACS通常不适用。
与使用FACS描述面部特征细节不同的是,一些识别系统使用了基于主观标记表情原型的描述方法。主观表示用于推断面部表情背后的隐含意义,如：感情或个性等。通常,情感原型表示会根据面部表情的强度变化而变化。这种方法的主要问题在于它不能依据现有的表情子集来表示所有可能的情感类别。
(二)自动面部表情分析系统框架
自动面部表情分析能根据面部特征判断个人的特征。通常,人脸表情识别系统可以看作一个单独的表情识别系统或作为人脸识别系统的一个模块。本文提出了一个面部表情分析系统的框架描述。图像获取和人脸识别是该框架的重要组成部分。目前,很多方法都可以从静态图像中进行人脸检测。早起的大多数检测方法只能检测人脸的正面。2004年,Viola和Jonesy提出了一种基于AdaBoos学习的人脸检测算法,在进行人脸和背景分离方面,该算法被认为是最快最有效的方法。Dalal等应用HOG描述子的进行了人脸检测,实验表明,该描述子的效果优于其它已有特征。
(三)特征提取和表示
特征表示阶段需要找到最恰当的方式表示面部对象,从而进行表情识别。通常,特征表示需要将人脸的像素数据转换为基于形状、运动、颜色、纹理、空间信息等高级对象表示的数据。
Pantic团队提出了经典的基于几何特征的特征表示方法；Tian等则对眼睛、眉毛、嘴唇以及其他特征进行形状建模来提取特征。Bartlet等则提出了经典的基于视觉特征的表情特征表示方法。
尽管研究人员已经提出了不同的方法表示面部表情的特征,但这一问题仍需要进行进一步的研究。目前,小波分析是一种流行的多分辨率分析方法。小波用于将单个的输入信号分解成多个更简单、更易于处理和解释的信号,同时,基于频域的表示方法充分利用了感兴趣的区域,同时也进行了降维。文献[95]使用Radon变换和傅里叶变换进行人脸检测；文献[96]则使用基于粒子群优化算法的Random变换,以及PCA和LDA技术进行人脸检测。文献[97]基于Random变换和小波变换,提出了一种新的人脸识别的框架。
(四)分类
分类是人脸表情分析系统的最后阶段。常用的分类方法有基于神经网络的方法、支撑向量机的方法、线性判别分析的方法等、基于马尔科夫模型的方法。其中,基于神经网络的表情识别方法的识别性能优异。有一些文献使用神经网络进行表情分类。
表情是心灵的窗口,而表情与行为也有密切联系。我们可以通过人脸表情预测或理解它人的行为,因此,对计算机来说,人脸表情识别是一种有效地与人交流的方式；通过理解人脸信息,计算机能对人的特征或情感做出判断。考虑到人脸对象以及获取人脸对象的设备的多样性,需要一种预处理机制解决对象特征的全局归一性问题。对计算机来说,具有明显边界的面部对象比边界模糊的面部对象更容易识别。而边界问题主要是由姿态、噪音或光照问题引起的。因此,我们需要对图像进行特征表示,可以是图像密度、颜色、文本或这些特征的组合。
现有的人脸识别算法,在单独应用时,都难以达到实际应用需要的识别精度。因此,本文试图在现有各种算法的基础上,致力于提高识别精度的研究。本文的目标是深入考察现有识别算法的技术特点和应用特征,通过引入新的处理过程、优化模型和参数、融合方法,提高识别计算的有效性。
本文的主要工作包括：分析了图像增强对识别性能的影响,通过对噪声数据进行分析,得到了图像噪声干扰识别特征提取的潜在趋势,提出了一种针对表情特征描述的相似性增强方法,使用图像增强技术获取图像的边缘特征,提取图像的关键特征并减少图像噪声,为提高表情对象分类的有效性提供条件。针对人脸表情的变化特点,本文提出了一种基于小波变换(DWT)的图像分割以及统计分析方法,成功地提取人脸表情特征,供后续过程进行识别。大部分自然图像都包含了丰富的方位信息,如垂直或水平边界。DWT有效地消除了噪声,保留了图像边缘的细节信息。运用方位信息,我们实现了低内存、低计算复杂度的基于DWT的特征提取方法。本研究不仅为经典DWT提供了一种应用方案,也为特征提取提供了新的解决办法。在小波变换之前,本文使用了去噪算法对人脸图像进行去噪处理,然后进行小波变换,最后重构图形并进行识别。本文深入研究了分水岭转换计算方法,提出了一种使用了非刚性配准原则的计算模式,有效解决了形状和脸部组成成分的分解问题。论文还使用主成分分析法对图像的形状特征进行降维处理,最后基于神经网络进行分类。
本文的主要研究内容涵盖了表情识别的主要方面,包括面部表情表示和特征提取、面部表情分割的方法、基于滤波反投影的纹理分析、基于小波提升的纹理分析
(一)面部表示和特征提取
本部分主要讨论如何进行图像数据预处理,它是后序算法的重要基础。首先描述了图像的预处理和增强过程,并使用低级图像处理算法进行处理。本文提出的降噪过程,综合考虑了低噪声的原型对象以及亮度连续的人脸表面,从而在下一步操作之前,突出了感兴趣的对象；在此基础上提取特征进行对象分类。本文的方法一方面解决了计算维度过高的问题,同时,将图像投向低维坐标系,使得从人脸表情识别系统提取出的面部特征保持高感知质量。我们的核心思想是,跟踪面部肌肉动作,突出感兴趣的目标对象,降低了光照、噪音的影响。
技术上,本文通过基于形态学的变换和分水岭变换,较好地突出了感兴趣的目标对象的形状。在不同尺度下,本文增强了亮度值较大的像素点与暗像素点之间差异。通过使用变换操作符,不仅减少了光照和噪声的隐形,而且将图像的隐含特征块聚集在一起。本文方法的新颖之处是基于在更亮处和黑色图像区域增强对比度,用结构化像素和图像细节代替所选的像素,从而能够基于图像尺寸和分水岭细节知识,将面部特征与背景分割开来,突出了感兴趣的区域,如眼睛、鼻子和嘴。增强的面部特征是使用主成分分析法提取的,并送入神经网络用于分类。
(二)面部表情分割
本部分主要研究如何将二维面部图像分割为光照不变、且具有最小表情变化和旋转的图像块。本文提出了一种基于小波变换的增强top-hat变换操作以及bottom-hat变换操作,该方法可以量化分割算法对面部特征的感知能力,通过重建一系列的投影,重新在一个对象中组织内部结构信息。在汇总每条线的响应之前,通过采用傅里叶切片定理过滤出已提取的增强轮廓,用来估计脸部形状。该方法不仅不受旋转、照明和噪声的影响,而且可以利用已提取的不完整数据重建图像。本文通过离散小波变换和离散余弦变换对比实验,分析了提取有效特征的效果。我们证明,由于图像中的低频分量比高频分量视觉效果更显著,所以提取到的低维特征向量是训练神经网络分类器的重要数据。
为评测本文算法的有效性和实用性,我们使用了日本女性面部表情数据库(JAFFE).为了从每副图像中获取感兴趣的数据区域,使用本文的方法首先对数据进行预处理。在对象分割阶段,对象被看成具有密度结构的表面,本文使用分水岭分割自动地将面部表情特征从背景从分离出来,并高亮度的显示刚兴趣的区域,如：眼睛、眉毛、鼻子、嘴巴等区域,以便进行后序的机器学习。
本文测试了基于PCA和基于神经网络的分水岭分割的算法结果,结果表明,在JAFFE数据库上的识别精度达到98%。
(三)基于滤波反投影的纹理分析
面部表情和皮下肌肉的一种或多种运动有关,但不同的观察者看到这些表情时,可能有不同的理解。为避免产生有歧义的输出,本文研究了基于滤波反向投影的非线性转换面部描述子来估计转换参数,转换参数为后序的离散小波变换提供输入数据。Radom变换可以捕获图像的方位特征,DWT变换将图像转换为小波子空间,从而分离出有意义的数据,分离出来的数据特征向量可用于神经网络分类器。
选择JAFFE作为实验数据库,在实验阶段,本文测试了Radon投影的效果、形态逻辑操作预处理的效果、DWT方法的性能以及DCT-PCA域中滤波反向投影的效果。实验结果表明：识别精度主要处决于对输入图像的姿态、光照和面部变化的处理。为识别输入图像,对DCT系数进行PCA变换,计算训练向量中具有最大变化的特征向量。训练阶段使用了136个特征向量,测试阶段使用了70个特征向量。然后使用神经网络训练PCA过程的输出。
实验结果表明：用于消除噪声和不规则变化的形态学开操作和闭操作增强了有利于表情识别的低频成分；同时,滤波反向投影有效地增强了面部数据的提取。显然,这两种方法都有利于提取面部数据中的可视低频成分,从而提高表情识别的精度。
(四)基于小波提升的纹理分析
本部分主要利用小波提升方法和主成分分析法分析面部表情。本文提出了一种利用高通和低通滤波器的相似性的方法,以实现更快的小波变换,从而提升解决方案的有效性。我们注意到了利用非线性高通与低通滤波器的相似性,本文在提升方案中加入了中值滤波处理,利用它在压缩信号方面的潜力,即它所拥有的两种相似结构的计算能力,能够根据压缩完的一半数据样本来预测出另一半数据样本。整个转换过程分为三个阶段,这三个阶段不断的重复执行,直到预测完所有数据样本。像其他的数字图像处理一样,减少高频分量不会给图像带来视觉上的影响。
许多之前的研究结果中强调的旋转、光照和噪音等影响系统性能的因素对本文的算法不会产生影响。该系统的最后一步使用开始的两步提供的信息,来准确识别面部表情(高兴,悲伤,中性,恐惧,厌恶,愤怒和惊讶)。本文对不同的算法进行了对比实验,结果表明,这些方法是可行的,而且实验表明,本文提出的解决方案,增强了程序效能,改进了神经网络对面部表情识别的表现。为了分类脸部特征提取,我们使用JAFEE和PICS数据库的图片对神经网络进行训练。实验证明,本文方法在减少噪音、修改其他不规则的地方以及克服传统方法的缺点上,都体现出明显的有效性,从而对人脸表情识别系统提供了另一种解决方案。
人机交互设备在日常生活中的应用日渐增多,相关技术也在不断的发展,但人脸表情识别的问题仍旧没有得到解决。我们拥有的计算机设备的存储能力在不断的提高,这也刺激了我们需要更多的与其它人或事物保持联系,更多地了解他们的信息。因此,越来越多的智能设备出现在我们的日常生活中,但这也带来了一个问题,即：人和智能设备之间的交流,我们需要更完善的方式来填补人机交互之间的语义鸿沟。为了能建设更友好的人机交互系统,使用人脸表情进行有效地交流成为一种必然的方式。这意味着我们需要捕获人脸对象、记录人脸行为并且利用已有的信息进行分析和决策。本文针对人脸表情识别的问题,提出了一些有用的解决方案。
总体来说,本文的主要创新如下：
(1)将图像分割成有意义的同构块是图像分析和机器视觉中的关键问题。使用分水岭进行图像数据分割或对图像进行形态学操作,是将图像对象表示成与形状有关的特征。本文提出了一种基于Top-hat变换预处理方法,能够有效降低光照、噪音的影响。通过对top-hat和bottom-hat进行增强,并应用在分水岭转化中,以此提取有效的面部特征进行表情识别。我们发现,Top-hat变换更适于提取基于暗背景的明亮对象提取；而bottom-hat则适用于基于亮背景的暗对象提取。在实际的应用中,暗图像特征和亮图像特征均存在于不同尺度的图像中。如果能提取各种尺度下的图像暗特征和亮特征,并应用于之后的分析,那图像增强的效果将大大提高。所以,本文方法致力于捕获这些特征,通过增大不同尺度下图像亮区域和暗区域的对比,有效减少了光照与噪声信号的干扰,增强了有利于进行识别的对象的基本特征。
(2)使用中值滤波器进行小波提升,使得图像的方位特征中的非垂直或非水平的线性边界得以最大化。该方法充分利用了高通滤波器和低通滤波器的相似性,加快小波变换的实现。本文的方法证明,首先使用形态滤波器减少由于光照和噪音影响带来,然后在小波提升中使用中值滤波器,是一条在消除噪音的同时保留图像中的有用信息的有效手段。
(3)本文提出了一种提高表情识别系统的精确性和鲁棒性的方法,基于滤波反投影方法重构对象的内部结构信息,再利用一系列投影进行数学重构,增强了表情识别中的基本特征。图像的投影形成特征向量,对该向量进行Radon转换提取图像灰度；再使用反投影生成向量,近似表示原始对象的形状。为了避免直接投影的缺陷,使用傅里叶中的切片定理过滤光强分布。使用以上的小波变换方法,进行降维和图像去噪,得到了更好的图像表示方法。在此上述表示框架下,小的图像系统表示噪声,大的图像系数表示重要的图像特征。特别地,离散小波变换用于把图像对象压缩成更小更方便管理的数据,从而能从少量的数据中提取有意义的图像信息、。我们从特征向量中提取出对光照和噪音不变的脸部特征,并进行神经网络的训练。最后,基于结果图像的逆小波变化,提供了具有本质特征的重构图像,并以此作为进行神经网络分类器的训练数据。
(4)研究了反投影过滤器对离散余弦变换的影响。在反投影过滤之前,本文对图像进行了形态学开操作和闭操作,并使用Radon变换进行转换。在此变换中,DCT变换用于提取旋转不变特征。低频率的切片成分在图像中比高频成分更具有视觉意义,因此,在DCT变换中,丢弃高频系数,并量化剩余的系数。通过DCT变换的应用,将输入序列表示为基于权重余弦序列的和。其结果系数用于描述信号,从而极大的减少数据量,同时也保留了图像重要信息。为进一步减少数据信息,使用主成分分析提取图像的特征。通过提取面部数据的重要特征,保留低阶主成分,丢弃高阶成分,PCA不仅减小图像大小,减少图像数据量,从而完成图像的降维。最后,使用前馈反向传播神经网络进行训练,从而完成表情识别。
人脸表情分析在人机交互中起着至关重要的作用。表情识别与人脸面部的运动变化有关,它提供了丰富的关于人的情感、内涵及其它内在状态的信息。尽管目前仍旧没有关于“情感”一词的精确定义,但人类情感的存在是毫无疑问的,而且它是我们的日常生活的重要组成部分。人类能够理解“情感”,并能针对它人的“情感”做出相应的反应,正因为如此,人类的之间的交互活动才显得如此丰富多彩。本文根据静态数据,解决了人脸表情识别的度量的问题。首先根据数据库进行人脸对象检测,然后进行关键点的面部表情特征提取并建立模型。计算机阅读面部表情和数据提取的能力至关重要,因为这决定了计算机是否能做出正确的回应,正确的表情读取是表情分类正确与否的关键。一个性能优越的表情系统需要处理面部识别、特征提取和分类等问题,且其结果与个人的年龄、周围环境、身体姿态等外部因素均无关。今后,我们将使用更多的标准数据库对本文的算法进行测试,并进一步提高算法的适应性。同时,人脸表情识别系统还需要捕获并识别更复杂多变的表情对象,这些是我们今后进一步研究的方向。
For several decades our options for interacting with the digital world has been limited to use of mouse, keyboards and joysticks. This objects limit machine's ability to respond to human needs. As a result they have exposed an existing gap between human and machines creating room for renewed interest in affective technology. Affective computing refers to development of system and devices that recognize, interpret, process, and simulate human affects. It brings together individuals from diverse backgrounds like engineering, computer science, psychology, cognitive science, neuroscience, sociology, education, psychophysiology, value-centered design and more to push for boundaries on what can be achieved to improve human affective experience. Emotion is fundamental to human experience; it influences our perception such as learning, communication and decision-making. However, many technologists have largely ignored emotion and create often frustrating experience for people, in part because affect has been misunderstood and is hard to measure.
As machines and people begin to co-exist and cooperatively share a variety of tasks, we see a trend that is shifting from text, to full-body video-conferencing as virtual communication grow rapidly in availability and complexity. Even as this richness of communication options improve our ability to converse with others not available at the precise moment, the fact that something is missing continue to plague users of current technologies presenting a need for affective communication channels between humans and machines. Humans are experts in expressing, recognizing and distinguishing facial expressions without much effort yet for computers this is an uphill task.
To be effective in the human world, machines must respond to human emotional states. In real life situations a lot of communications that use facial expressions need no words at all to explain the situation at hand. Many people do a great deal of inference from perceived facial expressions as conveyed by statements like "You seem happy today." Facial expressions can give a true picture about one's feeling; while you may say that you are feeling fine, one look at your face may tell people otherwise. These expressions make more than a statement about a person's emotional state and mostly occur to reinforce or contradict what has been said. Automatic facial expression recognition aim at perceiving and understanding emotional states of humans based on information found in the face. These expressions reveal a part of the feeling inside a person's private life that could be extracted without intrusion making the ability to understand them more interesting.
Naturally facial expressions may suggest certain emotions visually to an observer while in reality they express a totally different emotion resulting from some proportion of the expression resembling the other. To avoid this confusing outcome we need a well-designed state of the art smart human machine interface that interpret facial behavioral change giving an insight to a variety of information that could be derived from a face. Automatic facial expression recognition is a technique that could be harnessed to solve this problem. It is considered a high-end task in computer-vision and range in complexity from locating the face in images to recognizing isolated objects belonging to a few well-established classes. Our research is built upon this theories that advance basic understanding of affect and its role in human experience. We believe it is for the same reason that researchers are making every attempt at understanding relation that exist between affective signals, mental states and how they can be extracted and tapped to bridge the gap that separate man from machines.
Despite their efforts computers still remain disconnected from the human world. Computer systems are not intelligent enough to pay attention to people and their surrounding environments. This raises need to build systems with perceptual intelligence that can realize what aspects of the surrounding environment are important and use it to interpret who, where, what, when and why of the situation. This way machines become intelligent enough to adapt their behavior to suit human needs rather than humans adapting to theirs. The answer to these questions lie in having an automated measurement system that is able to detect a human face, preprocess, extract features and classify objects in different categories. Facial recognition accuracy depends heavily on how input images have been compensated for pose, illumination and facial expressions.
The aim of this thesis is to analyze and extract behavior referenced by facial features that could be used to convey user emotions or internal state of a person in an effort to solve human machine problem. Our goal is to design a robust, high performance facial expression recognition system that can work in real-time situation with ability to observe, interpret, and generate affect features that react appropriately to natural human behavior. Building such systems is no easy task, first is due to various unpredictable facial variations that are further complicated by external environmental conditions. In addition is the difficulty in choosing suitable characterization of facial feature descriptors capable of extracting discriminative facial information given the influence of illumination, pose among other factors. Facial variations that arise from pose, age, gender, race and occlusion, also exert profound influence on the robustness of facial systems. Facial muscular assume the appearance passion people want to experience therefore a suitable facial feature descriptor would largely determine its performance giving characteristic upon which people can make conclusion regarding the person, ideas or status. Secondly, the contextual events that signify facial expressions and emotional states still remain ambiguous and vague to computers.
Our computation approach addresses the curse of dimension, mapping the image into low dimension coordinate system that preserves perceptual quality of objects enabling feature extraction. The approach entails highlighting objects of interest with aid computer vision technologies where we map and track facial muscle movement, decode and interpret these movements having leveled illumination and eliminated noise effects before feature extraction and classification. A major part of our contribution is to determine optimal ways of combining different algorithms in an attempt to improve facial recognition accuracy. We have developed a system for evaluating and extracting appropriate facial features to be used in measuring facial expression recognition problem.
Previous studies have used texture and geometric features for expression recognition, however few of them conduct and explore performance differences between features arising from image quality. The first key step is to recognize a human face with acceptable levels of false positives this has been achieved appropriately. Following this, enhancement is carried out to provide better quality data for purposes of machine interpretation before feature extraction. For emotion recognition, the extraction of meaningful patterns from the gathered data must be found. A practical approach, used in this thesis is simulation of emotions using image processing, computer vision and machine learning techniques to enable process facial expression detection enrich and facilitate interactivity between human and machine, giving rise to labels (angry, fear, sad, surprise, neutral, happy and disgust).
The first algorithm is based on watershed segmentation an approach that accurately replicate the shape pattern of objects most needed in machine learning. Segmentation is fundamental to image analysis; it entails division of image into regions of similar attributes aimed at discovering objects of interest. To eliminate drawbacks associated with watershed segmentation we adopt contrast enhancement between brighter and black image regions at the scale corresponding to different scales of structuring elements and image details based on morphological tophat and bottomhat transform operators. By incorporating this knowledge watershed segmentation is used to separate facial features from the background, highlighting regions of interest eyes, nose and mouth required for subsequent high level machine learning. Enhanced facial features are extracted using principle component analysis, a small set of eigenvectors with top eigenvalues are used to build principal components. Each principle component is considered a feature representing a point in high dimensional space. Image features obtained are used to train the neural network from this a recognition rate of98%was acieved.
Using a different approach we quantify machine perception of facial features based on internal structural information found within an object by mathematically reconstructing it from a series of projections. Radon transform was used to capture a series of line integrals across the image at different angles, capturing directional local features lying within the object. The extracted intensity profiles were filtered by applying the Fourier Slice Theorem before contributions from each line of response were summed and used to arrive at an estimate of the facial shape. The method is invariant to rotations, lighting and noise in addition it is possible to reconstruct the image even when presented with incomplete data. To determine the most efficient feature extraction technique experiments were carried out separately using the wavelet transform and discrete cosine transform.
Specifically, Haar discrete wavelet transform was used to compress image objects to a smaller more manageable data that facilitate extract enhanced facial features. Filtered back projection is used to reorganize internal structural information within objects by mathematically reconstructing it from a series of projections. These set of images were projected at180degrees and low frequency components extracted to form significant data used to train the Neural Network (NN) classifier. Experiments were carried out to assess validity and efficiency of this algorithm. The results reveal non-enhanced data gave a recognition rate of97.2%while enhanced data gave99%accuracy in determination of emotional state from facial expression recognition. While an approach that combine discrete cosine transform and principal component analysis used125projections to give a recognition rate of97.79%for non-enhanced data and enhanced data achieved98.99%accuracy in determination of emotional state from facial expression recognition.
In an alternative method adaptive wavelet lifting scheme was used to extract facial features for expression recognition. The lifting scheme makes optimal use of similarities between high and low pass filters to achieve a faster implementation of the wavelet transform. To take advantage of non-linear filters we incorporated median filters to utilize its potential at compressing signals these results from the fact that the two algorithms share a similar structure. Facial images like any other digital image contain highly correlated data and its high pass component can be discarded without much effect on the visual quality of the image. For this purpose Haar lifting scheme combined with principle component analysis were used to exploit directional correlation between adjacent pixels. A recognition rate of99.4%was achieved on JAFEE database and97%on PICS databases respectively. This was a more improved performance compared to98%and98.4%for PCA and DWT-PCA on the same images from JAFFE database. From this results feasibility of algorithms are demonstrated and proved effective at minimizing noise and other irregularities, demerits associated with conventional methods providing an alternative perspective towards facial expression recognition problem. Based on experimental analysis carried out we reveal that emotions in machines can be associated with abstract states empowering them to respond to people's needs.
The motivation behind this research a rose from the unobtrusive nature the face uses to communicate things that are difficult to embrace with written or spoken words. In addition its potential applications are many to mention a few is that such systems help deter crime, fraud and streamline business processes to save critical resources. Currently an increasing number commercial applications are either using or actively considering use of facial recognition methods as witnessed in fields like medicine and education while others in need of this technology are clinical psychology, psychiatry, neurology, pain assessment, lie detection, criminology and multimodal human computer interface. A lot is yet to be achieved in relation to interpreting facial messages and how they are produced by the neuro-muscular system. We believe that recognition of human facial expression using computers is key to promoting these fields. A system that can perform these operations accurately and in real time would form a big step in bridging the gap that exists between man and machines. This probably explains the reason why a human face is not only at the center of security concern but also a sudden renewed interest in this field.
The goal of this research was to apply image analysis and machine learning techniques to examine, extract and measure facial expression. Using this theoretical background we have examined face expression in detail, in an effort to understand and capitalize on those features of the face that can help distinguish different classes of facial expressions, increasing its significance as a problem solving method that would ultimately lead to its increased adoption in many domains. In particular, this thesis provides evidence that use of enhancement in facial feature extraction not only improves recognition rate of facial expressions but also improves the scalability and robustness of the resulting system.
引文
[1]Shichuan Du and Aleix M. Martinez, "The Resolution of facial expressions of emotion", Journal of Vision, vol.11, No.13, pp.1-13,2011.
[2]Frank Y. Shih, Chao-Fa Chuang, Patrick S. P. Wang, "performance comparisons of facial expression recognition in JAFFE database", international journal of pattern recognition and artificial intelligence, vol.22, No.3, pp.445-459,2008.
[3]Shichuan Du, Yong Tao and Aleix M. Martinez, "Compound facial expressions of emotion", Proceedings of the National Academy of Sciences of the United States of America,2014.
[4]T. L. Griffith, J. E. Sawyer, and D. M. A. Neale, "Virtualness and knowledge in teams: managing the love triangle of organizations, individuals, and information technology", MIS Quarterly vol.27. No.2, pp.265-287,2003.
[5]M. Kunz, J. Chen, S. Lautenbacher, E. Vachon-Presseau, and P. Rainville, "Cerebral Regulation of Facial Expressions of Pain", The Journal of Neuroscience, vol.31, no 24, pp. 8730-8738,2011.
[6]P. Ekman, T. S. Huang, T. J. Sejnowski, and J. C Hager, "NSF Report-Facial Expression Understanding",1992.
[7]B. Fasel, and J. Luttin, "Automatic Facial Expression Analysis:a survey", pattern Recognition, vol.36, no.2, pp.259-275,2003.
[8]M Pantic and MS Bartlett, "Machine Analysis of Facial Expressions" I-Tech Education and Publishing, PP.377-416,2007.
[9]Y. Wang, L.Guan, "Recognizing Human Emotional State from Audiovisual Signals", IEEE transactions on multimedia, vol.10, No.5, pp.936-946,2008.
[10]M.Yeasin, B. Bullot, R. Sharma, "Recognition of Facial Expressions and Measurement of Levels of Interest From Video", IEEE Transactions On Multimedia, Vol.8, No.3, pp.500-508,2006.
[11]Zlochower, J. Cohn, J. Lien, and T. Kanade. "A computer vision based method of facial expression analysis in parent-infant interaction", In International Conference on Infant Studies, Atlanta,1998.
[12]W. Rinn. "The neuropsychology of facial expression:a review of the neurological and psychological mechanisms for producing facial expressions", Psychological Bulletin, vol. 95,No.1,pp.52-77,1984.
[13]J.M Van Swearingen, J. Cohn, and A. Bajaj-Luthra, "Specific impairment of smiling increases severity of depressive symptoms in patients with facial neuromuscular disorders", Journal of Aesthetic Plastic Surgery, vol.23, pp.416-423,1999.
[14]Tian, Y.L., Kanade, T. and Cohn, J.F, "Facial Expression Analysis, Handbook of Face Recognition", New York, pp.247-276,2005.
[15]Y. Moses, Y. Adini, and S. Ullman. "Face recognition:The problem of compensating for changes in illumination direction",. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.19, No.7,721-732,1997.
[16]Ahmed Bilal Ashraf, Simon Lucey, Jeffrey F. Cohn, Tsuhan Chen, Zara Ambadar, Kenneth M. Prkachin, Patricia E. Solomon, "The painful face Pain expression recognition using active appearance models", Image and Vision Computing vol.27,No.12, pp.1788-1796,2009.
[17]Sheryl Brahnam, Chao-fa chuang, Randall s. sexton, Frank Y.,"Machine assessment of neonatal facial expressions of acute pain" Decision Support Systems, Vol.43, No 4, pp. 1242-1254,2007.
[18]Gagliardi C, Frigerio E, Burt D.M, Cazzaniga I, Perrett DI, and Borgatti R, "Facial recognition in Williams syndrome", Neuropsychologia, vol.41, no.6, pp.733-738,2003.
[19]R. Sprengelmeyer, A.W. Young, K. Mahn, U. Schroeder, D. Woitalla, T. Buttner, W. Kuhn, and H. Przuntek, "Facial expression recognition in people with medicated and unmedicated Parkinson's disease", Neuropsychologia, vol.41, no.8, pp.1047-1057,2003.
[20]E.L. Rosenberg, P. Ekman, W. Jiang, and M. Babyak, "Linkages between facial expressions of anger and transient", American Psychological Assn, US. Emotion, vol.1, no.2, pp.107-115,2001.
[21]P. Ekman, and E.L. Rosenberg, "What the face reveals:basic and applied studies of spontaneous expression using the facial action coding system (FACS) Illustrated Edition", Oxford University Press,1997.
[22]V. Haynal and M. Heller, "The faces of suicidal depression," Kahiers Psychiatriques Genevois (Medecine et Hygiene, vol.16, pp.107-117,1994.
[23]Sofia G., "Extraction of Eyes for Facial Expression Identification of Students", International Journal of Engineering Science and Technology,vol.2 (7) pp.3024-3029,2010.
[24]J. Whitehill, M. Bartlett and J. Movellan, "Automatic facial expression recognition for intelligent tutoring systems", Computer Vision and Pattern Recognition, pp.1-6,2008.
[25]A. Kapoor, W. Burleson, and R. Picard, "Automatic prediction of frustration", International Journal of Human-Computer Studies, vol.65, no.8, pp.724-736,2007.
[26]S. D'Mello, R.W Picard and A. Graesser, "Towards an affect-sensitive autotutor", In proceeding(s) of the IEEE Intelligent Systems, Special issue on Intelligent Educational Systems, vol.22, no.4, pp.62-69,2007.
[27]A. Sarrafzadeh, S. Alexander, F. Dadgostar, C. Fan, and A. Bigdeli, "See me, teach me: Facial expression and gesture recognition for intelligent tutoring systems", In Innovations in Information Technology, pp.1-5,2006.
[28]Homero V. Rios, Ana Luisa Solis, Emilio Aguirre, Lourdes Guerrero, Joaquin Pena, and Alberto Santamaria, "Facial expression recognition and modeling for virtual intelligent tutoring systems", In Proceeding(s) of the Mexican International Conference on Artificial Intelligence Advances in Artificial Intelligence, vol.10, pp115-126,2000.
[29]Dinges D. F., Venkataraman S., McGlinchey E. L., and Metaxas D. N., "Monitoring of facial stress during space flight:Optical computer", Acta Astronautica, vol.60, pp.341-350, 2007.
[30]Bartlett, M.S., Littlewort, G.C., Frank, M.G., Lainscsek,C., Fasel, I., and Movellan, J.R., "Automatic Recognition of Facial Actions in Spontaneous Expressions", Journal of Multimedia, vol.1, No.6, pp.22-35,2006.
[31]K.D. Craig, S.A. Hyde, and C.J. Patrick, "Genuine, suppressed, "Pain, vol.46,No.2 pp 161-172,1991
[32]P.Ekman, and M.G. Frank, "Appearing truthful generalizes," Journal of personality, vol.86, No.3, pp.486-495,2004.
[33]M. A. Sayette, D. W. Smith, M.J. Breiner, and G. T. Wilson, "The effect of alcohol on emotional response to a social," Journal of Studies on Alcohol, vol.53, pp.541-545,1992.
[34]Deepthi.S, Archana.G.S, and Jagathy Raj.V.P, "Facial Expression Recognition Using Artificial Neural Networks" Journal of Computer Engineering, Volume 8, No.4, pp.1-6, 2013.
[35]Pushpaja V. Saudagare, and D.S. Chaudhari, "Efficient Face Recognition System using Artificial Neural Network," International Journal of Computer Applications, Vol.41, No.21, pp.12-15,2012.
[36]S.Adebayo Daramola, and O. Sandra Odeghe, "Facial Expression Recognition using Neural Network An Overview," International Journal of Soft Computing and Engineering (IJSCE), Vol.2, No.1, pp.224-22,2012.
[37]Ongalo P. N Fedha, Huang Dong Jun, Richard Rimiru, "Neural Network Based Classifier for a Segmented Facial Expression Recognition System Based on Haar Wavelet Transform", Global Journal of Computer Science and Technology, vol.12, No.7, pp.7-13,2012.
[38]Ongalo P. N. Fedha, Huang Dong Jun, Thekra Abbas," Machine Learning Performance on Face Expression Recognition using Filtered Backprojection in DCT-PCA Domain", JCSI International Journal of Computer Science Issues, Vol.10, No.1,2013.
[39]Ongalo P. N Fedha, Huang Dong Jun, Richard Rimiru, "Watershed segmented Face Expression Recognition Using Principal Component Analysis and Neural Network", International Journal of Advanced Computer Technology, Vol.5, No.10,2013.
[40]Ongalo P. N. Fedha, Huang Dong Jun, Richard Rimiru, Lilian Sing'oei, "Face Expression Recognition based on lifting wavelet transform and principle component analysis", Journal of Convergence Information Technology, vol.8,No.12,2013.
[41]Zhang B., Shan S., Chen X and Gao W., "Histogram of gabor phase patterns (HGPP):a novel object representation approach for face recognition," IEEE Transactions on Image Processing, vol.16, no.l, pp.57-68,2007.
[42]K. Mase, A. Pentland, "Recognition of facial expression from optical flow", IEICE Trans. E vol.74, No.10, pp.3474-3483,1991
[43]B. Jiang, M. F. Valstar, B. Martinez, and M. Pantic, "A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modelling", IEEE Transactions of Systems, Man and Cybernetics--Part B. vol.44, No.2, pp.161-174,2014.
[44]Z. Zheng, M. Pantic, G.I. Roisman, and T. S. Huang, "A survey of affect recognition methods:audio, visual, and spontaneous expressions", In proceeding(s) of the IEEE Trans. Pattern Analysis and Machine Intelligence, vol.31, no.1, pp.39-58,2009.
[45]Zhihong Z., Pantic M., Roisman G and Huang T., "A survey of affect recognition methods: audio, visual, and spontaneous expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.31, no.l, pp.39-58,2009.
[46]Murtaza M., Sharif M., Raza M and Shah J., "Analysis of face recognition under varying facial expression:a survey," The International Arab Journal of Information Technology, Vol 10, No 4,2013.
[47]V. Bettadapura, "Face Expression Recognition and Analysis:The State of the Art", Tech Report arXiv:1203.6722, April 2012.
[48]Yi Ji, Khalid Idrissi, "Automatic facial expression recognition based on spatiotemporal descriptors", Pattern Recognition Letters, vol.33, No.10, pp.1373-1380,2012.
[49]Vaibhavkumar J. Mistry, Mahesh M. Goyani,"A literature survey on Facial Expression Recognition using Global Features", International Journal of Engineering and Advanced Technology (IJEAT) ISSN:Vol.2, No.4,pp.2249-8958,2013
[50]Stephen Moore, Eng Jon Ong, Richard Bowden, "Facial Expression Recognition Using Spatiotemporal Boosted Discriminatory Classifiers", ICIAR,vol.l, pp.405-414,2010.
[51]Takeo Kanade, Jeffrey Cohn, Ying Li Tian, "Comprehensive Database for Facial Expression Analysis," Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition pp.46-53,2000.
[52]Http://www.kasrl.org/jaffe_download.html.
[53]P. Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, and Zara Ambadar, "The extended cohn-kanade dataset(CK+):A complete dataset for action unit and emotion-specified expression", Computer Vision and Pattern Recognition Workshop, pp.94-101,2010.
[54]The MMI Facial Expression Database:www.mmifacedb.com.
[55]Samal, A. and Iyengar, P.A. "Automatic recognition and analysis of human faces and facial expressions:A survey", Pattern Recognition, Vol.25, No.1, pp.65-77,1992.
[56]Pantic, M. and Patras, I, "Dynamics of facial expression:Recognition of facial actions and their temporal segments from face profile image sequences", IEEE Trans. On Systems, Man and Cybernetics-Part B, Vol.36, No.2, pp.433-449,2006.
[57]Sander Koelstra, Maja Pantic, and Ioannis (Yiannis) Patras, "A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models", IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol.32, No.11,2010.
[58]Littlewort G, Whitehill J, Wu T, Fasel I, Frank M, Movellan J, and Bartlett M, "The Computer Expression Recognition Toolbox (CERT)," Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp.298-305,2011.
[59]Valstar, M.F. and Pantic, M, "Fully automatic facial action unit detection and temporal analysis, Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition", Vol.3, pp.149-156, New York,2006.
[60]Pantic, M. and Rothkrantz, L.J.M, "Toward an Affect-Sensitive Multimodal Human-Computer Interaction", Proceedings of the IEEE, Spec. Issue on Human-Computer Multimodal Interface, Vol.91, No.9, pp.1370-1390,2003.
[61]Pantic, M., Pentland, A., Nijholt, A. & Huang, T, "Human Computing and machine understanding of human behaviour:A Survey", Proc. ACM Int'l Conf. Multimodal Interfaces, pp.239-248,2006.
[62]N. Sebe, M.S. Lew, Y. Sun, I. Cohen, T. Gevers, T.S. Huang, "Authentic Facial Expression Analysis," Image and Vision Computing, vol.25,No.12, pp.1856-1863,2007.
[63]I. Kotsia, I. Buciu, I. Pitas, "An Analysis of Facial Expression Recognition under Partial Facial Image Occlusion," Image and Vision Computing, vol.26, no.7, pp.1052-1067,2008.
[64]Philipp Michel, and Rana El Kaliouby, "Real Time Facial Expression Recognition in Video Using Support Vector Machines," in Proc.5th Int. Conf. Multimodal Interfaces, Vancouver, BC,, Canada,258-264,2003.
[65]Michel Valstar, Bihan Jiang, Marc Mehu, Maja Pantic, Klaus Scherer," The First Facial Expression Recognition and Analysis Challenge", IEEE Int'l. Conf. Face and Gesture Recognition (FG'11),pp.921-926,2011.
[66]M. Pantic and L. Rothkrantz, "Expert system for automatic analysis of facial expression", Image and Vision Computing, vol.18, No.11, pp.881-905,2000.
[67]Ming-Hsuan Yang, David J. Kriegman, Narendra Ahuja, "Detecting Faces in Images:A Survey", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.24,no.l, pp.34-58, Jan,2002.
[68]M. Turk, A. Pentland, "Eigenfaces For Recognition", Journal Of Cognitive Neuroscience, vol.3, pp.71-86,1991.
[69]K.-K. Sung and T. Poggio, "Example-based learning for view-based human face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no.1, pp.39-51, 1998.
[70]H. A. Rowley, S. Baluja, T. Kanade, "Neural network-based face detection", Pattern Analysis and Machine Intelligence, IEEE Transactions,vol.20, No.1, pp.23-38,1998.
[71]E. Osuna, R. Freund, F. Girosi., "Training support vector machines:An application to face detection," 1997.
[72]H. Schneiderman and T. Kanade, "A statistical method for 3D object detection applied to faces and cars," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'00), pp.746-751, June 2000.
[73]P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol.1, pp.1511-1518, December 2001.
[74]P. Viola, M.J. Jones, "Robust real-time object detection,"Int. Journal of Computer Vision, vol.57, no.2, pp.137-154, Dec.2004.
[75]S. Z. Li, L. Zhu, Z. Q. Zhang, A. Blake, H. Zhang, H. Shum., "Statistical learning of multi-view face detection," In Proceedings of The 7th European Conference on Computer Vision. Copenhagen, Denmark,2002.
[76]N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection." in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol.1, pp.886-893, June 2005.
[77]Stan Z. Li, RuFeng Chu, ShengCai Liao, Lun Zhang. "Illumination Invariant Face Recognition Using Near-infrared Images". IEEE Transactions on Pattern Analysis and Machine Intelligence (Special issue on Biometrics:Progress and Directions). Vol.29, No.4, pp.627-39. April 2007.
[78]S. Z. Li and Z. Zhang, "FloatBoost learning and statistical face detection", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.26, no.9, pp.1112-1123, 2004.
[79]Stan Z. Li, XiaoGuang Lv, XinWen Hou, XianHua Peng, QianSheng Cheng. "Learning Multi-View Face Subspaces and Facial Pose Estimation Using Independent Component Analysis", IEEE Transactions on Image Processing, Vol.14,No.6, pp.705-712, June 2005.
[80]P. Belhumeur, J. Hespanha, D. Kriegman, "Eigenfaces vs. fisherfaces:recognition using class specific linear projection", IEEE Trans. Pattern Anal. Mach. Intell. Vol.19, No.7, pp.711-720,1997.
[81]A. K. Jain, S. Z. Li, Handbook of Face Recognition, NJ, USA, Springer-Verlag New York, Inc., Secaucus,2005.
[82]Tian, Y.L., Kanade, T. and Cohn, J.F., "Recognizing action units for facial expression analysis", IEEE Trans, Pattern Analysis and Machine Intelligence, Vol.23, No.2, pp.97-115,2001.
[83]Y. Tian, T. Kanade, and J. Cohn, "Evaluation of Gabor-Wavelet Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity," Proc. IEEE Conf. Face and Gesture Recognition, pp.218-223,2002.
[84]Peiyao Li, Son Lam Phung Abdesselam Bouzerdoum, Fok Hing Chi Tivive, "Improved Facial, Expression Recognition with Trainable 2-D Filters and Support Vector Machines," International Conference on Pattern Recognition, pp.3732-3735,2010.
[85]Y. Tian, T. Kanade, and J. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.23, no.2, pp.97-115, Feb.2001.
[86]S. Lucey, A. Ashraf, and J. Cohn, "Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face," Face Recognition, K. Delac and M. Grgic, eds., pp.275-286,1-Tech Education and Publishing,2007.
[87]Padgett C. and Cottrell G.W., ""Representing Face Images for Emotion," Proceedings of Conference on Advances in Neural Information, pp.894-900,1996.
[88]J. Lien, "Automatic recognition off facial expression using hidden Markov models and estimation of expression intensity," Ph.D. Thesis, The Robotics Institute, CMU, April 1998.
[89]Murugappan Murugappan, Nagarajan Ramachandran, Yaacob Sazali,"Classification of human emotion from EEG using discrete wavelet transform," Bio-medical signal processing, vol.23, pp.390-396,2010.
[90]Ze-Nian Li.Mark S. Drew, "Fundamentals of multimedia", Pearson Education Asia limited and China Machine press,2004.
[91]Merzagora, A.C., Bunce, S., Izzetoglu, M. and Onaral, B.,"Wavelet analysis for EEG feature extraction in deceptive detection", IEEE Proceedings on EBMS,6,2434-2437,2006.
[92]Satiyan M., Nagarajan R., "Recognition of facial expression using Haar-like feature extraction method.," Proceedings of the 3rd IEEE International Conference on Intelligent and Advanced Systems (ICIAS), Kuala Lumpur, Malaysia, PP.1-4,2010.
[93]Deqiang Li., Pedrycz W., Nicolino J.P., "Fuzzy Wavelet Packet Based Feature Extraction Method and Its Application to Biomedical Signal Classification," IEEE Trans. Biomed. Eng, vol.52, No.6, pp.1132-1139,2005.
[94]Kwak KC, Pedrycz W, "Face recognition using fuzzy integral and wavelet decomposition method," IEEE Trans Syst Man Cybern Part B, vol.34, No.4, pp.1666-1675,2004.
[95]Jadhao D.V., Holambe R.S., "Feature Extraction and Dimensionality Reduction Using Radon and Fourier Transform with Application to Face Recognition," Proceedings of the International Conference on Computational Intelligence and Multimedia Applications, vol.2, pp.254-260,2007.
[96]Hamid M.S., Waleed A. AL. J., Majid A. A "Face Recognition Using Improved FFT Based Radon by PSO and PCA Techniques," International Journal of Image Processing (IJIP), vol. 6, No.1,2012.
[97]Jadhav D.V., Holambe R.S., "Feature extraction using radon and wavelet transforms with application to face recognition," Neurocomputing, (Elsevier Press), vol.72, pp.1951-1959, 2009.
[98]Dattatray V. Jadhav, Raghunath S. Holambe, "Radon and discrete cosine transforms based feature extraction and dimensionality reduction approach for face recognition," signal processing, vol.88, No.10, pp.2604-2609,2008
[99]Dattatray V. Jadhav, Raghunath S.Holombe "Rotation, illumination invariant polynomial kernel fisher discriminant analysis using Radon and discrete cosine transforms based features for face recognition," Pattern Recogn, Lett, vol.31, pp.1002-1009,2010.
[100]Srisuk S, Petrou M, Kurutach W, Kadyrov A, "Face authentication using trace transform," Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.305-312,2003.
[101]L. Ma and K. Khorasani, "Facial Expression Recognition Using Constructive Feedforward Neural Networks", IEEE Transactions On Systems, Man, And Cybernetics—Part B: Cybernetics, Vol.34, No.3,2004.
[102]H.-H Tsai, Y.-S Lai, Y.-C. Zhang, "using SVM to design facial expression recognition for shape and texture features", Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Vol.5, pp.2697-2704,2010.
[103]Frank Y. Shih, Chao-Fa Chuang, Patrick S. P. Wang, "Performance comparisons of facial expression recognition in JAFFE database", International Journal of Pattern Recognition and Artificial Intelligence vol.22, No.3, pp.445-459,2008.
[104]M Schmidt, M Schels, F Schwenker, "A hidden markov model based approach for facial expression recognition in image sequences," Artificial Neural Networks in Pattern recognition, pp.149-160,2010.
[105]Kobayashi H., Hara F., "Facial Interaction between.Animated 3D Face Robot and Proceedings of International Conference on Systems," Man,Cybernetics, vol.3, pp.732-737, 1997.
[106]Kobayashi, H., Hara, F, "The Recognition of Basic Facial Expressions by Neural," Proceedings of International Joint Conference on Neural Network, pp.460-466.,1991
[107]Zhang Z., Lyons M., Schuster M., Akamatsu S., "Comparison between Geometry-Based and Gabor Wavelets-Based Facial Expression Recognition Using Multi," Proceedings of International Conference on Automatic Face, pp.454-459,1998.
[108]Zhao J., Kearney G.,"Classifying Facial Emotions by Backpropagation Neural," in Proceedings of Conference on Neural Information, vol.1, pp.454-457,1996.
[109]Z. Zhang, "Feature-based facial expression recognition:Sensitivity analysis and experiments with a multi-layer perceptron," International Journal of Pattern Recognition and Artificial Intelligence, Vol.13, No.6,pp 893-911,1999.
[110]Picard, R.W, Affective Computing, MIT Press, Cambridge, USA,1997.
[111]PICS:http://pics.psych.stir.ac.uk
[112]I. Buciu, C. Kotropoulos and I. Pitas, "ICA and Gabor representation for facial expression recognition," Proc. IEEE Int. Conf. Image Processing pp.855-858,2003.
[113]H. Rowley, S. Baluja, T. Kanade,"Human face detection in visual scenes:Technical Report", CMU-CS-95-158R,School of Computer Science, Carnegie Mellon University, 1995.
[114]Mikael Nilsson, Jorgen Nordberg,and Ingvar Claesson "Face detection using local SMQT features and split up SNOW classifier" in IEEE International conference on Acoustics, Speech, and signal processing(ICASSP),vol 2, pp.589-592,2007.
[115]Kak A. C, Slaney M, "Principles of Computerized Tomographic Imaging," Society of Industrial and Applied Mathematics,2001.
[116]T.F. Cootes, D. Cooper, C.J. Taylor and J. Graham, "Active Shape Models-Their Training and Application," Computer Vision and Image Understanding, Vol.61,No.1, pp. 38-59,1995.
[117]M. Sonka, V. Hlavac, and R. Boyle, "Image Processing, Analysis, and Machine Vision", Pacific Grove, California:PWS Publishing, second ed.,1999.
[118]Shameem Akthar, D. RajyaLakshmi, Syed Abdul Sattar, "Suppression of Over Segmentation in Watershed Transform Using Pre Processing Method", International Journal of Computer Technology and Electronics Engineering, Volume 2, Issue 2,2012.
[119]Vincent L. and Soille P. "Watershed in Digital Space:An Efficient Algorithm Based on Immersion Simulations," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.13, no.6, pp.583-593,1991
[120]F. Meyer, "Levelings and morphological segmentation, in:Proceedings of SIBGRAPI'98", Rio de Janeiro, Brazil, pp.28-35,1998.
[121]Meyer F., Beucher S. "Morphological segmentation," J. Visual Commun and Image Repres, vol.1, no.1, pp.21-45,1990.
[122]Chowdhury M. I. and Robinson J. A., "Improving Image Segmentation Using Edge Information," in Proceedings of the 1st IEEE Conference on Electrical and Computer Engineering, vol.1, pp.312-316,2000.
[123]S. Beucher, "Watershed, hierarchical segmentation and waterfall algorithm, Proc. Mathematical Morphology and its Applications to Image Processing," Fontainebleau, The Netherlands:Kluwer Ac. Publ., Nld, pp.69-76,1994.
[124]A.N. Moga, M. Gabbouj, "Parallel image component labelling with watershed transformation, IEEE transactions on pattern analysis and machine intelligence", Vol.19, No.5, pp.441-450,1997.
[125]J. Weickert, "Fast segmentation methods based on partial differential equations and the watershed transform," in Proc. DAGM Symp. pp.93-100,1998.
[126]J. Sijbers, P. Scheunders, M. Verhoye, A. Van der Linden, D. Van Dyck, and E. Raman, "Watershed-based segmentation of 3D MR data for volume quantization," Magn Reson. Imag, vol.15,No.6, pp.679-688,1997.
[127]Luc Vincent, "Morphological Grayscale Reconstruction in Image Analysis:Applications and Efficient Algorithms", IEEE Transactions on Image Processing, Vol.2, No.2, pp.176-201,1993.
[128]V. Grau, A. U. J. Mewes, M. Alcaniz, Member, IEEE, R. Kikinis, and S. K. Warfield, " Improved Watershed Transform for Medical Image nSegmentation Using Prior Information, " IEEE Transactions on Medical Imaging, vol.23, No.4, pp.447-458,2004.
[129]Ghassan Hamarneh, Xiaoxing Li, "Watershed segmentation using prior shape and appearance knowledge," Image and Vision Computing vol.27(1-2):, No.l,pp.59-68,2009.
[130]Megha Sharma, Seema Verma, "Implementation of Watershed Based Image Segmentation Algorithm using Distance Transform for Grid Computing", International Journal Of Computers & Technology, Vol 10, No 6,pp.1715-1719,2013
[131]J. B. T. M. Roerdink and A. Meijster, "The watershed transform:Definitions,algorithms and parallelization strategies," Fundamenta Informaticae, vol.41, pp.187-228,2000.
[132]K. Haris, S. N. Efstratiadis, N. Maglaveras, and A. K. Katsaggelos, "Hybrid image segmentation using watersheds and fast region merging," IEEE Transactions on Image Processing, vol.7,No.12, pp.1684-1699,1998.
[133]L. Sirovich, M. Kirby, "Application of the Karhunen-Loeve procedure for the characterization of human Faces", In proceeding(s) of the IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.12, no.1, pp.103-108,1990.
[134]E. Gomathi, K. Baskaran, "Age Classification and Face Recognition Using Neural Network and Fusion of Wavelet and Curvelet Transform", Journal of Applied Sciences Research, vol.(9) No.3, pp.1366-1374,2013.
[135]Wei Wang, Lihong Xu, Lu Bing, "Smile Expression Recognition Using Enhanced Locality Preserving Projection and Special Two-Class Relevance Vector Machine", IJACT: International Journal of Advancements in Computing Technology, Vol.4, No.17, pp.530-539,2012.
[136]CHEN Juanjuan, ZHAO Zheng, SUN Han, ZHANG Gang, "Facial Expression Recognition Based on PCA Reconstruction", In proceeding(s) of the 5th International Conference on Computer Science & Education, Hefei, pp.195-198,2010.
[137]Cohn, J.F., "Advances in Behavioral Science Using Automated Facial Image Analysis and Synthesis", IEEE Signal Processing Magazine, Vol.27, No.6, pp.128,2010.
[138]D. L. Donoho and I. M. Johnstone, "Adapting to Unknown Smoothness via Wavelet Shrinkage", Journal of the American Statistical Association, Vol.90, No.432, pp.1200-1224,1995.
[139]D. L. Donoho, "Denoising by soft thresholding", IEEE Trans.information Theory, Vol.41, No.3, pp.613-627,1995.
[140]T. Acharya, C. Chakrabarti, "A Survey on Lifting-based Discrete Wavelet Transform Architectures", Journal of VLSI Signal Processing, vol.42, No.3, pp.321-339,2006.
[141]J.Madhavan,K.Porkumaran,"Performance Comparison Of PCA, Dwt-PCA and LWT-PCA For Face Image Retrieval", Computer Science & Engineering an International Journal, Vol.2, No.6, pp.41-50,2012.
[142]W. Sweldens, "The Lifting Scheme:A Custom-Design Construction of Biorthogonal Wavelets", Applied and Computational Harmonic Analysis, vol.3, No.2, pp.186-200,1996.
[143]Prachi Agarwal, Naveen Prakash, "An Efficient Back Propagation Neural Network Based Face Recognition System Using Haar Wavelet Transform and PCA", International Journal of Computer Science and Mobile Computing, Vol.2, No.5, pp.386-395,2013.
[144]M. Gargesha, P. Kuchi, and I. D. K. Torkkola, "Facial expression recognition using artificial neural networks", Artifficial Neural Computation Systems,2002.
[145]R. C. Gonzalez, R. E. Woods, and S. L. Eddins. Digital Image Processing Using MATLAB. Prentice Hall, Englewood Cliffs, NJ, USA,2004.
[2]Frank Y. Shih, Chao-Fa Chuang, Patrick S. P. Wang, "performance comparisons of facial expression recognition in JAFFE database", international journal of pattern recognition and artificial intelligence, vol.22, No.3, pp.445-459,2008.
[3]Shichuan Du, Yong Tao and Aleix M. Martinez, "Compound facial expressions of emotion", Proceedings of the National Academy of Sciences of the United States of America,2014.
[4]T. L. Griffith, J. E. Sawyer, and D. M. A. Neale, "Virtualness and knowledge in teams: managing the love triangle of organizations, individuals, and information technology", MIS Quarterly vol.27. No.2, pp.265-287,2003.
[5]M. Kunz, J. Chen, S. Lautenbacher, E. Vachon-Presseau, and P. Rainville, "Cerebral Regulation of Facial Expressions of Pain", The Journal of Neuroscience, vol.31, no 24, pp. 8730-8738,2011.
[6]P. Ekman, T. S. Huang, T. J. Sejnowski, and J. C Hager, "NSF Report-Facial Expression Understanding",1992.
[7]B. Fasel, and J. Luttin, "Automatic Facial Expression Analysis:a survey", pattern Recognition, vol.36, no.2, pp.259-275,2003.
[8]M Pantic and MS Bartlett, "Machine Analysis of Facial Expressions" I-Tech Education and Publishing, PP.377-416,2007.
[9]Y. Wang, L.Guan, "Recognizing Human Emotional State from Audiovisual Signals", IEEE transactions on multimedia, vol.10, No.5, pp.936-946,2008.
[10]M.Yeasin, B. Bullot, R. Sharma, "Recognition of Facial Expressions and Measurement of Levels of Interest From Video", IEEE Transactions On Multimedia, Vol.8, No.3, pp.500-508,2006.
[11]Zlochower, J. Cohn, J. Lien, and T. Kanade. "A computer vision based method of facial expression analysis in parent-infant interaction", In International Conference on Infant Studies, Atlanta,1998.
[12]W. Rinn. "The neuropsychology of facial expression:a review of the neurological and psychological mechanisms for producing facial expressions", Psychological Bulletin, vol. 95,No.1,pp.52-77,1984.
[13]J.M Van Swearingen, J. Cohn, and A. Bajaj-Luthra, "Specific impairment of smiling increases severity of depressive symptoms in patients with facial neuromuscular disorders", Journal of Aesthetic Plastic Surgery, vol.23, pp.416-423,1999.
[14]Tian, Y.L., Kanade, T. and Cohn, J.F, "Facial Expression Analysis, Handbook of Face Recognition", New York, pp.247-276,2005.
[15]Y. Moses, Y. Adini, and S. Ullman. "Face recognition:The problem of compensating for changes in illumination direction",. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.19, No.7,721-732,1997.
[16]Ahmed Bilal Ashraf, Simon Lucey, Jeffrey F. Cohn, Tsuhan Chen, Zara Ambadar, Kenneth M. Prkachin, Patricia E. Solomon, "The painful face Pain expression recognition using active appearance models", Image and Vision Computing vol.27,No.12, pp.1788-1796,2009.
[17]Sheryl Brahnam, Chao-fa chuang, Randall s. sexton, Frank Y.,"Machine assessment of neonatal facial expressions of acute pain" Decision Support Systems, Vol.43, No 4, pp. 1242-1254,2007.
[18]Gagliardi C, Frigerio E, Burt D.M, Cazzaniga I, Perrett DI, and Borgatti R, "Facial recognition in Williams syndrome", Neuropsychologia, vol.41, no.6, pp.733-738,2003.
[19]R. Sprengelmeyer, A.W. Young, K. Mahn, U. Schroeder, D. Woitalla, T. Buttner, W. Kuhn, and H. Przuntek, "Facial expression recognition in people with medicated and unmedicated Parkinson's disease", Neuropsychologia, vol.41, no.8, pp.1047-1057,2003.
[20]E.L. Rosenberg, P. Ekman, W. Jiang, and M. Babyak, "Linkages between facial expressions of anger and transient", American Psychological Assn, US. Emotion, vol.1, no.2, pp.107-115,2001.
[21]P. Ekman, and E.L. Rosenberg, "What the face reveals:basic and applied studies of spontaneous expression using the facial action coding system (FACS) Illustrated Edition", Oxford University Press,1997.
[22]V. Haynal and M. Heller, "The faces of suicidal depression," Kahiers Psychiatriques Genevois (Medecine et Hygiene, vol.16, pp.107-117,1994.
[23]Sofia G., "Extraction of Eyes for Facial Expression Identification of Students", International Journal of Engineering Science and Technology,vol.2 (7) pp.3024-3029,2010.
[24]J. Whitehill, M. Bartlett and J. Movellan, "Automatic facial expression recognition for intelligent tutoring systems", Computer Vision and Pattern Recognition, pp.1-6,2008.
[25]A. Kapoor, W. Burleson, and R. Picard, "Automatic prediction of frustration", International Journal of Human-Computer Studies, vol.65, no.8, pp.724-736,2007.
[26]S. D'Mello, R.W Picard and A. Graesser, "Towards an affect-sensitive autotutor", In proceeding(s) of the IEEE Intelligent Systems, Special issue on Intelligent Educational Systems, vol.22, no.4, pp.62-69,2007.
[27]A. Sarrafzadeh, S. Alexander, F. Dadgostar, C. Fan, and A. Bigdeli, "See me, teach me: Facial expression and gesture recognition for intelligent tutoring systems", In Innovations in Information Technology, pp.1-5,2006.
[28]Homero V. Rios, Ana Luisa Solis, Emilio Aguirre, Lourdes Guerrero, Joaquin Pena, and Alberto Santamaria, "Facial expression recognition and modeling for virtual intelligent tutoring systems", In Proceeding(s) of the Mexican International Conference on Artificial Intelligence Advances in Artificial Intelligence, vol.10, pp115-126,2000.
[29]Dinges D. F., Venkataraman S., McGlinchey E. L., and Metaxas D. N., "Monitoring of facial stress during space flight:Optical computer", Acta Astronautica, vol.60, pp.341-350, 2007.
[30]Bartlett, M.S., Littlewort, G.C., Frank, M.G., Lainscsek,C., Fasel, I., and Movellan, J.R., "Automatic Recognition of Facial Actions in Spontaneous Expressions", Journal of Multimedia, vol.1, No.6, pp.22-35,2006.
[31]K.D. Craig, S.A. Hyde, and C.J. Patrick, "Genuine, suppressed, "Pain, vol.46,No.2 pp 161-172,1991
[32]P.Ekman, and M.G. Frank, "Appearing truthful generalizes," Journal of personality, vol.86, No.3, pp.486-495,2004.
[33]M. A. Sayette, D. W. Smith, M.J. Breiner, and G. T. Wilson, "The effect of alcohol on emotional response to a social," Journal of Studies on Alcohol, vol.53, pp.541-545,1992.
[34]Deepthi.S, Archana.G.S, and Jagathy Raj.V.P, "Facial Expression Recognition Using Artificial Neural Networks" Journal of Computer Engineering, Volume 8, No.4, pp.1-6, 2013.
[35]Pushpaja V. Saudagare, and D.S. Chaudhari, "Efficient Face Recognition System using Artificial Neural Network," International Journal of Computer Applications, Vol.41, No.21, pp.12-15,2012.
[36]S.Adebayo Daramola, and O. Sandra Odeghe, "Facial Expression Recognition using Neural Network An Overview," International Journal of Soft Computing and Engineering (IJSCE), Vol.2, No.1, pp.224-22,2012.
[37]Ongalo P. N Fedha, Huang Dong Jun, Richard Rimiru, "Neural Network Based Classifier for a Segmented Facial Expression Recognition System Based on Haar Wavelet Transform", Global Journal of Computer Science and Technology, vol.12, No.7, pp.7-13,2012.
[38]Ongalo P. N. Fedha, Huang Dong Jun, Thekra Abbas," Machine Learning Performance on Face Expression Recognition using Filtered Backprojection in DCT-PCA Domain", JCSI International Journal of Computer Science Issues, Vol.10, No.1,2013.
[39]Ongalo P. N Fedha, Huang Dong Jun, Richard Rimiru, "Watershed segmented Face Expression Recognition Using Principal Component Analysis and Neural Network", International Journal of Advanced Computer Technology, Vol.5, No.10,2013.
[40]Ongalo P. N. Fedha, Huang Dong Jun, Richard Rimiru, Lilian Sing'oei, "Face Expression Recognition based on lifting wavelet transform and principle component analysis", Journal of Convergence Information Technology, vol.8,No.12,2013.
[41]Zhang B., Shan S., Chen X and Gao W., "Histogram of gabor phase patterns (HGPP):a novel object representation approach for face recognition," IEEE Transactions on Image Processing, vol.16, no.l, pp.57-68,2007.
[42]K. Mase, A. Pentland, "Recognition of facial expression from optical flow", IEICE Trans. E vol.74, No.10, pp.3474-3483,1991
[43]B. Jiang, M. F. Valstar, B. Martinez, and M. Pantic, "A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modelling", IEEE Transactions of Systems, Man and Cybernetics--Part B. vol.44, No.2, pp.161-174,2014.
[44]Z. Zheng, M. Pantic, G.I. Roisman, and T. S. Huang, "A survey of affect recognition methods:audio, visual, and spontaneous expressions", In proceeding(s) of the IEEE Trans. Pattern Analysis and Machine Intelligence, vol.31, no.1, pp.39-58,2009.
[45]Zhihong Z., Pantic M., Roisman G and Huang T., "A survey of affect recognition methods: audio, visual, and spontaneous expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.31, no.l, pp.39-58,2009.
[46]Murtaza M., Sharif M., Raza M and Shah J., "Analysis of face recognition under varying facial expression:a survey," The International Arab Journal of Information Technology, Vol 10, No 4,2013.
[47]V. Bettadapura, "Face Expression Recognition and Analysis:The State of the Art", Tech Report arXiv:1203.6722, April 2012.
[48]Yi Ji, Khalid Idrissi, "Automatic facial expression recognition based on spatiotemporal descriptors", Pattern Recognition Letters, vol.33, No.10, pp.1373-1380,2012.
[49]Vaibhavkumar J. Mistry, Mahesh M. Goyani,"A literature survey on Facial Expression Recognition using Global Features", International Journal of Engineering and Advanced Technology (IJEAT) ISSN:Vol.2, No.4,pp.2249-8958,2013
[50]Stephen Moore, Eng Jon Ong, Richard Bowden, "Facial Expression Recognition Using Spatiotemporal Boosted Discriminatory Classifiers", ICIAR,vol.l, pp.405-414,2010.
[51]Takeo Kanade, Jeffrey Cohn, Ying Li Tian, "Comprehensive Database for Facial Expression Analysis," Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition pp.46-53,2000.
[52]Http://www.kasrl.org/jaffe_download.html.
[53]P. Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, and Zara Ambadar, "The extended cohn-kanade dataset(CK+):A complete dataset for action unit and emotion-specified expression", Computer Vision and Pattern Recognition Workshop, pp.94-101,2010.
[54]The MMI Facial Expression Database:www.mmifacedb.com.
[55]Samal, A. and Iyengar, P.A. "Automatic recognition and analysis of human faces and facial expressions:A survey", Pattern Recognition, Vol.25, No.1, pp.65-77,1992.
[56]Pantic, M. and Patras, I, "Dynamics of facial expression:Recognition of facial actions and their temporal segments from face profile image sequences", IEEE Trans. On Systems, Man and Cybernetics-Part B, Vol.36, No.2, pp.433-449,2006.
[57]Sander Koelstra, Maja Pantic, and Ioannis (Yiannis) Patras, "A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models", IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol.32, No.11,2010.
[58]Littlewort G, Whitehill J, Wu T, Fasel I, Frank M, Movellan J, and Bartlett M, "The Computer Expression Recognition Toolbox (CERT)," Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp.298-305,2011.
[59]Valstar, M.F. and Pantic, M, "Fully automatic facial action unit detection and temporal analysis, Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition", Vol.3, pp.149-156, New York,2006.
[60]Pantic, M. and Rothkrantz, L.J.M, "Toward an Affect-Sensitive Multimodal Human-Computer Interaction", Proceedings of the IEEE, Spec. Issue on Human-Computer Multimodal Interface, Vol.91, No.9, pp.1370-1390,2003.
[61]Pantic, M., Pentland, A., Nijholt, A. & Huang, T, "Human Computing and machine understanding of human behaviour:A Survey", Proc. ACM Int'l Conf. Multimodal Interfaces, pp.239-248,2006.
[62]N. Sebe, M.S. Lew, Y. Sun, I. Cohen, T. Gevers, T.S. Huang, "Authentic Facial Expression Analysis," Image and Vision Computing, vol.25,No.12, pp.1856-1863,2007.
[63]I. Kotsia, I. Buciu, I. Pitas, "An Analysis of Facial Expression Recognition under Partial Facial Image Occlusion," Image and Vision Computing, vol.26, no.7, pp.1052-1067,2008.
[64]Philipp Michel, and Rana El Kaliouby, "Real Time Facial Expression Recognition in Video Using Support Vector Machines," in Proc.5th Int. Conf. Multimodal Interfaces, Vancouver, BC,, Canada,258-264,2003.
[65]Michel Valstar, Bihan Jiang, Marc Mehu, Maja Pantic, Klaus Scherer," The First Facial Expression Recognition and Analysis Challenge", IEEE Int'l. Conf. Face and Gesture Recognition (FG'11),pp.921-926,2011.
[66]M. Pantic and L. Rothkrantz, "Expert system for automatic analysis of facial expression", Image and Vision Computing, vol.18, No.11, pp.881-905,2000.
[67]Ming-Hsuan Yang, David J. Kriegman, Narendra Ahuja, "Detecting Faces in Images:A Survey", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.24,no.l, pp.34-58, Jan,2002.
[68]M. Turk, A. Pentland, "Eigenfaces For Recognition", Journal Of Cognitive Neuroscience, vol.3, pp.71-86,1991.
[69]K.-K. Sung and T. Poggio, "Example-based learning for view-based human face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no.1, pp.39-51, 1998.
[70]H. A. Rowley, S. Baluja, T. Kanade, "Neural network-based face detection", Pattern Analysis and Machine Intelligence, IEEE Transactions,vol.20, No.1, pp.23-38,1998.
[71]E. Osuna, R. Freund, F. Girosi., "Training support vector machines:An application to face detection," 1997.
[72]H. Schneiderman and T. Kanade, "A statistical method for 3D object detection applied to faces and cars," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'00), pp.746-751, June 2000.
[73]P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol.1, pp.1511-1518, December 2001.
[74]P. Viola, M.J. Jones, "Robust real-time object detection,"Int. Journal of Computer Vision, vol.57, no.2, pp.137-154, Dec.2004.
[75]S. Z. Li, L. Zhu, Z. Q. Zhang, A. Blake, H. Zhang, H. Shum., "Statistical learning of multi-view face detection," In Proceedings of The 7th European Conference on Computer Vision. Copenhagen, Denmark,2002.
[76]N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection." in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol.1, pp.886-893, June 2005.
[77]Stan Z. Li, RuFeng Chu, ShengCai Liao, Lun Zhang. "Illumination Invariant Face Recognition Using Near-infrared Images". IEEE Transactions on Pattern Analysis and Machine Intelligence (Special issue on Biometrics:Progress and Directions). Vol.29, No.4, pp.627-39. April 2007.
[78]S. Z. Li and Z. Zhang, "FloatBoost learning and statistical face detection", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.26, no.9, pp.1112-1123, 2004.
[79]Stan Z. Li, XiaoGuang Lv, XinWen Hou, XianHua Peng, QianSheng Cheng. "Learning Multi-View Face Subspaces and Facial Pose Estimation Using Independent Component Analysis", IEEE Transactions on Image Processing, Vol.14,No.6, pp.705-712, June 2005.
[80]P. Belhumeur, J. Hespanha, D. Kriegman, "Eigenfaces vs. fisherfaces:recognition using class specific linear projection", IEEE Trans. Pattern Anal. Mach. Intell. Vol.19, No.7, pp.711-720,1997.
[81]A. K. Jain, S. Z. Li, Handbook of Face Recognition, NJ, USA, Springer-Verlag New York, Inc., Secaucus,2005.
[82]Tian, Y.L., Kanade, T. and Cohn, J.F., "Recognizing action units for facial expression analysis", IEEE Trans, Pattern Analysis and Machine Intelligence, Vol.23, No.2, pp.97-115,2001.
[83]Y. Tian, T. Kanade, and J. Cohn, "Evaluation of Gabor-Wavelet Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity," Proc. IEEE Conf. Face and Gesture Recognition, pp.218-223,2002.
[84]Peiyao Li, Son Lam Phung Abdesselam Bouzerdoum, Fok Hing Chi Tivive, "Improved Facial, Expression Recognition with Trainable 2-D Filters and Support Vector Machines," International Conference on Pattern Recognition, pp.3732-3735,2010.
[85]Y. Tian, T. Kanade, and J. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.23, no.2, pp.97-115, Feb.2001.
[86]S. Lucey, A. Ashraf, and J. Cohn, "Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face," Face Recognition, K. Delac and M. Grgic, eds., pp.275-286,1-Tech Education and Publishing,2007.
[87]Padgett C. and Cottrell G.W., ""Representing Face Images for Emotion," Proceedings of Conference on Advances in Neural Information, pp.894-900,1996.
[88]J. Lien, "Automatic recognition off facial expression using hidden Markov models and estimation of expression intensity," Ph.D. Thesis, The Robotics Institute, CMU, April 1998.
[89]Murugappan Murugappan, Nagarajan Ramachandran, Yaacob Sazali,"Classification of human emotion from EEG using discrete wavelet transform," Bio-medical signal processing, vol.23, pp.390-396,2010.
[90]Ze-Nian Li.Mark S. Drew, "Fundamentals of multimedia", Pearson Education Asia limited and China Machine press,2004.
[91]Merzagora, A.C., Bunce, S., Izzetoglu, M. and Onaral, B.,"Wavelet analysis for EEG feature extraction in deceptive detection", IEEE Proceedings on EBMS,6,2434-2437,2006.
[92]Satiyan M., Nagarajan R., "Recognition of facial expression using Haar-like feature extraction method.," Proceedings of the 3rd IEEE International Conference on Intelligent and Advanced Systems (ICIAS), Kuala Lumpur, Malaysia, PP.1-4,2010.
[93]Deqiang Li., Pedrycz W., Nicolino J.P., "Fuzzy Wavelet Packet Based Feature Extraction Method and Its Application to Biomedical Signal Classification," IEEE Trans. Biomed. Eng, vol.52, No.6, pp.1132-1139,2005.
[94]Kwak KC, Pedrycz W, "Face recognition using fuzzy integral and wavelet decomposition method," IEEE Trans Syst Man Cybern Part B, vol.34, No.4, pp.1666-1675,2004.
[95]Jadhao D.V., Holambe R.S., "Feature Extraction and Dimensionality Reduction Using Radon and Fourier Transform with Application to Face Recognition," Proceedings of the International Conference on Computational Intelligence and Multimedia Applications, vol.2, pp.254-260,2007.
[96]Hamid M.S., Waleed A. AL. J., Majid A. A "Face Recognition Using Improved FFT Based Radon by PSO and PCA Techniques," International Journal of Image Processing (IJIP), vol. 6, No.1,2012.
[97]Jadhav D.V., Holambe R.S., "Feature extraction using radon and wavelet transforms with application to face recognition," Neurocomputing, (Elsevier Press), vol.72, pp.1951-1959, 2009.
[98]Dattatray V. Jadhav, Raghunath S. Holambe, "Radon and discrete cosine transforms based feature extraction and dimensionality reduction approach for face recognition," signal processing, vol.88, No.10, pp.2604-2609,2008
[99]Dattatray V. Jadhav, Raghunath S.Holombe "Rotation, illumination invariant polynomial kernel fisher discriminant analysis using Radon and discrete cosine transforms based features for face recognition," Pattern Recogn, Lett, vol.31, pp.1002-1009,2010.
[100]Srisuk S, Petrou M, Kurutach W, Kadyrov A, "Face authentication using trace transform," Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.305-312,2003.
[101]L. Ma and K. Khorasani, "Facial Expression Recognition Using Constructive Feedforward Neural Networks", IEEE Transactions On Systems, Man, And Cybernetics—Part B: Cybernetics, Vol.34, No.3,2004.
[102]H.-H Tsai, Y.-S Lai, Y.-C. Zhang, "using SVM to design facial expression recognition for shape and texture features", Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Vol.5, pp.2697-2704,2010.
[103]Frank Y. Shih, Chao-Fa Chuang, Patrick S. P. Wang, "Performance comparisons of facial expression recognition in JAFFE database", International Journal of Pattern Recognition and Artificial Intelligence vol.22, No.3, pp.445-459,2008.
[104]M Schmidt, M Schels, F Schwenker, "A hidden markov model based approach for facial expression recognition in image sequences," Artificial Neural Networks in Pattern recognition, pp.149-160,2010.
[105]Kobayashi H., Hara F., "Facial Interaction between.Animated 3D Face Robot and Proceedings of International Conference on Systems," Man,Cybernetics, vol.3, pp.732-737, 1997.
[106]Kobayashi, H., Hara, F, "The Recognition of Basic Facial Expressions by Neural," Proceedings of International Joint Conference on Neural Network, pp.460-466.,1991
[107]Zhang Z., Lyons M., Schuster M., Akamatsu S., "Comparison between Geometry-Based and Gabor Wavelets-Based Facial Expression Recognition Using Multi," Proceedings of International Conference on Automatic Face, pp.454-459,1998.
[108]Zhao J., Kearney G.,"Classifying Facial Emotions by Backpropagation Neural," in Proceedings of Conference on Neural Information, vol.1, pp.454-457,1996.
[109]Z. Zhang, "Feature-based facial expression recognition:Sensitivity analysis and experiments with a multi-layer perceptron," International Journal of Pattern Recognition and Artificial Intelligence, Vol.13, No.6,pp 893-911,1999.
[110]Picard, R.W, Affective Computing, MIT Press, Cambridge, USA,1997.
[111]PICS:http://pics.psych.stir.ac.uk
[112]I. Buciu, C. Kotropoulos and I. Pitas, "ICA and Gabor representation for facial expression recognition," Proc. IEEE Int. Conf. Image Processing pp.855-858,2003.
[113]H. Rowley, S. Baluja, T. Kanade,"Human face detection in visual scenes:Technical Report", CMU-CS-95-158R,School of Computer Science, Carnegie Mellon University, 1995.
[114]Mikael Nilsson, Jorgen Nordberg,and Ingvar Claesson "Face detection using local SMQT features and split up SNOW classifier" in IEEE International conference on Acoustics, Speech, and signal processing(ICASSP),vol 2, pp.589-592,2007.
[115]Kak A. C, Slaney M, "Principles of Computerized Tomographic Imaging," Society of Industrial and Applied Mathematics,2001.
[116]T.F. Cootes, D. Cooper, C.J. Taylor and J. Graham, "Active Shape Models-Their Training and Application," Computer Vision and Image Understanding, Vol.61,No.1, pp. 38-59,1995.
[117]M. Sonka, V. Hlavac, and R. Boyle, "Image Processing, Analysis, and Machine Vision", Pacific Grove, California:PWS Publishing, second ed.,1999.
[118]Shameem Akthar, D. RajyaLakshmi, Syed Abdul Sattar, "Suppression of Over Segmentation in Watershed Transform Using Pre Processing Method", International Journal of Computer Technology and Electronics Engineering, Volume 2, Issue 2,2012.
[119]Vincent L. and Soille P. "Watershed in Digital Space:An Efficient Algorithm Based on Immersion Simulations," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.13, no.6, pp.583-593,1991
[120]F. Meyer, "Levelings and morphological segmentation, in:Proceedings of SIBGRAPI'98", Rio de Janeiro, Brazil, pp.28-35,1998.
[121]Meyer F., Beucher S. "Morphological segmentation," J. Visual Commun and Image Repres, vol.1, no.1, pp.21-45,1990.
[122]Chowdhury M. I. and Robinson J. A., "Improving Image Segmentation Using Edge Information," in Proceedings of the 1st IEEE Conference on Electrical and Computer Engineering, vol.1, pp.312-316,2000.
[123]S. Beucher, "Watershed, hierarchical segmentation and waterfall algorithm, Proc. Mathematical Morphology and its Applications to Image Processing," Fontainebleau, The Netherlands:Kluwer Ac. Publ., Nld, pp.69-76,1994.
[124]A.N. Moga, M. Gabbouj, "Parallel image component labelling with watershed transformation, IEEE transactions on pattern analysis and machine intelligence", Vol.19, No.5, pp.441-450,1997.
[125]J. Weickert, "Fast segmentation methods based on partial differential equations and the watershed transform," in Proc. DAGM Symp. pp.93-100,1998.
[126]J. Sijbers, P. Scheunders, M. Verhoye, A. Van der Linden, D. Van Dyck, and E. Raman, "Watershed-based segmentation of 3D MR data for volume quantization," Magn Reson. Imag, vol.15,No.6, pp.679-688,1997.
[127]Luc Vincent, "Morphological Grayscale Reconstruction in Image Analysis:Applications and Efficient Algorithms", IEEE Transactions on Image Processing, Vol.2, No.2, pp.176-201,1993.
[128]V. Grau, A. U. J. Mewes, M. Alcaniz, Member, IEEE, R. Kikinis, and S. K. Warfield, " Improved Watershed Transform for Medical Image nSegmentation Using Prior Information, " IEEE Transactions on Medical Imaging, vol.23, No.4, pp.447-458,2004.
[129]Ghassan Hamarneh, Xiaoxing Li, "Watershed segmentation using prior shape and appearance knowledge," Image and Vision Computing vol.27(1-2):, No.l,pp.59-68,2009.
[130]Megha Sharma, Seema Verma, "Implementation of Watershed Based Image Segmentation Algorithm using Distance Transform for Grid Computing", International Journal Of Computers & Technology, Vol 10, No 6,pp.1715-1719,2013
[131]J. B. T. M. Roerdink and A. Meijster, "The watershed transform:Definitions,algorithms and parallelization strategies," Fundamenta Informaticae, vol.41, pp.187-228,2000.
[132]K. Haris, S. N. Efstratiadis, N. Maglaveras, and A. K. Katsaggelos, "Hybrid image segmentation using watersheds and fast region merging," IEEE Transactions on Image Processing, vol.7,No.12, pp.1684-1699,1998.
[133]L. Sirovich, M. Kirby, "Application of the Karhunen-Loeve procedure for the characterization of human Faces", In proceeding(s) of the IEEE Transaction on Pattern Analysis and Machine Intelligence, vol.12, no.1, pp.103-108,1990.
[134]E. Gomathi, K. Baskaran, "Age Classification and Face Recognition Using Neural Network and Fusion of Wavelet and Curvelet Transform", Journal of Applied Sciences Research, vol.(9) No.3, pp.1366-1374,2013.
[135]Wei Wang, Lihong Xu, Lu Bing, "Smile Expression Recognition Using Enhanced Locality Preserving Projection and Special Two-Class Relevance Vector Machine", IJACT: International Journal of Advancements in Computing Technology, Vol.4, No.17, pp.530-539,2012.
[136]CHEN Juanjuan, ZHAO Zheng, SUN Han, ZHANG Gang, "Facial Expression Recognition Based on PCA Reconstruction", In proceeding(s) of the 5th International Conference on Computer Science & Education, Hefei, pp.195-198,2010.
[137]Cohn, J.F., "Advances in Behavioral Science Using Automated Facial Image Analysis and Synthesis", IEEE Signal Processing Magazine, Vol.27, No.6, pp.128,2010.
[138]D. L. Donoho and I. M. Johnstone, "Adapting to Unknown Smoothness via Wavelet Shrinkage", Journal of the American Statistical Association, Vol.90, No.432, pp.1200-1224,1995.
[139]D. L. Donoho, "Denoising by soft thresholding", IEEE Trans.information Theory, Vol.41, No.3, pp.613-627,1995.
[140]T. Acharya, C. Chakrabarti, "A Survey on Lifting-based Discrete Wavelet Transform Architectures", Journal of VLSI Signal Processing, vol.42, No.3, pp.321-339,2006.
[141]J.Madhavan,K.Porkumaran,"Performance Comparison Of PCA, Dwt-PCA and LWT-PCA For Face Image Retrieval", Computer Science & Engineering an International Journal, Vol.2, No.6, pp.41-50,2012.
[142]W. Sweldens, "The Lifting Scheme:A Custom-Design Construction of Biorthogonal Wavelets", Applied and Computational Harmonic Analysis, vol.3, No.2, pp.186-200,1996.
[143]Prachi Agarwal, Naveen Prakash, "An Efficient Back Propagation Neural Network Based Face Recognition System Using Haar Wavelet Transform and PCA", International Journal of Computer Science and Mobile Computing, Vol.2, No.5, pp.386-395,2013.
[144]M. Gargesha, P. Kuchi, and I. D. K. Torkkola, "Facial expression recognition using artificial neural networks", Artifficial Neural Computation Systems,2002.
[145]R. C. Gonzalez, R. E. Woods, and S. L. Eddins. Digital Image Processing Using MATLAB. Prentice Hall, Englewood Cliffs, NJ, USA,2004.