非负块配准模型与优化算法研究
|
摘要
非负矩阵分解是二十世纪九十年代末期新兴的数据降维方法，经过十多年的发展，已广泛应用于模式识别、数据挖掘和信息检索等领域。非负矩阵分解不同于传统的SVD分解、QR分解、Cholesky分解、LU分解和特征值分解，它将非负数据矩阵近似成两个非负子矩阵的乘积。由于非负矩阵分解把数据表示成特征的纯加性叠加，这种数据表示方法天然地隐含稀疏特性，与人脑对信号的响应机制相一致，能够有效地抑制信号噪音。同时，由于特征中不含任何负元素，符合真实世界的物理假设。因此，非负矩阵分解已经广泛应用于文本聚类、邮件监控、盲源信号分离、音频信号分析、人脸识别、图像标注、图像分割、光谱图像分析、基因微阵列数据分析等领域。近年来，非负矩阵分解引起越来越多的关注，美国贝尔实验室、田纳西大学、威克森林大学、佐治亚理工学院、芬兰的赫尔辛基大学、日本的RIKEN脑科学研究所、台湾清华大学、浙江大学等研究机构都开展了非负矩阵分解的研究工作。它们的研究成果极大地推动了非负矩阵分解技术的发展，把非负矩阵分解的应用拓展到互联网、信息安全、遥感图像和数学模型求解等领域。因此，开展非负矩阵分解的研究具有重要的现实意义。
本文从框架模型入手，在数学上建立非负块配准模型，从统一的角度分析非负数据降维算法，并用以开发新的非负矩阵分解模型。根据非负块配准模型的分析结果，本文提出非负判别局部块配准模型，克服了传统非负矩阵分解模型的缺点，提高了非负矩阵分解的分类效果。为了克服传统非负矩阵分解优化算法收敛速度慢的缺点，本文利用牛顿法快速搜索最优步长，提出非负块配准快速梯度下降算法。为了克服传统非负最小二乘问题优化算法的缺点，本文利用最优梯度法在无需线搜索的情况下以二阶收敛速度求解非负最小二乘问题。在此基础上提出非负矩阵分解的高效求解算法，并开发非负块配准最优梯度法。为了克服传统优化算法应用于流数据处理时计算开销过大的缺点，本文提出非负矩阵分解在线优化算法，利用鲁棒随机近似算法更新基矩阵，提高在线优化算法的鲁棒性。本文的主要创新点包括：
1.非负块配准模型提出基于块配准的非负矩阵分解框架模型——非负块配准模型。自非负矩阵分解提出以后，提出多种改进模型。利用局部视觉特征近似正交的特点，提出局部非负矩阵分解模型。利用流形学习技术，提出图罚分非负矩阵分解模型保持数据几何结构。利用Fisher判别技术，提出判别非负矩阵分解模型引入标签信息。然而，这些非负矩阵分解模型是由研究人员根据各自的需求和经验而设计的，内在差异巨大，难以理解其共同特性，实际应用中给工程人员选择模型带来困难。本文基于块配准框架，建立非负块配准模型，从统一的角度理解已有非负矩阵分解模型，揭示其内在差异和共同特性，指导工程人员选择模型，帮助研究人员开发新的非负矩阵分解模型。利用拉格朗日乘子法，提出乘法法则算法优化非负块配准模型，并用辅助函数技术证明算法的收敛性。该算法可用于求解非负块配准模型的大多数派生模型，包括标准的非负矩阵分解模型。
2.非负判别局部块配准模型根据非负块配准模型的分析结果，提出非负判别局部块配准模型。从非负块配准模型的角度看，局部非负矩阵分解为样本和基向量分别建立由自身组成的样本块，在局部优化过程保持数据的能量；图罚分非负矩阵分解为样本建立由自身和有限最近邻组成的样本块，在局部优化过程保持数据几何结构，但是忽略样本判别信息；判别非负矩阵分解为样本和各类中心点建立样本块且样本块分别由所有同类样本和中心点组成，在局部优化过程保持数据判别信息，但是由全部样本组成的样本块要求数据服从高斯分布。非负判别局部块配准模型克服了已有非负矩阵分解模型的缺点，为每个样本建立两类样本块：类内块由同类样本中有限最近邻组成，局部优化过程保持数据局部几何结构，放宽数据高斯分布假设；类间块由不同类样本中有限最近邻组成，局部优化过程最大化类间边界，从而保持数据判别信息。因此，非负判别局部块配准模型的分类效果较好、鲁棒性较强。本文利用全局配准技术把两种局部优化过程映射到全局坐标系进行并把二者结合，套用非负块配准的乘法法则算法优化所提非负判别局部块配准模型。
3.非负块配准快速梯度下降算法从梯度下降的角度改进非负块配准乘法法则算法，利用牛顿法实现快速线搜索，提出非负块配准快速梯度下降算法。快速线搜索沿着调整负梯度方向搜索最优步长，在不超出第一象限边界的情况下更新矩阵因子，大大提高乘法法则的收敛速度。利用凸函数的Jesen不等式，证明快速梯度法的收敛性。为了克服矩阵因子整体的最优步长可能为1的缺点，即蜕化成乘法法则，本文为矩阵因子的每列（或行）设置步长，用多变量牛顿法搜索步长向量，提出多步长快速线搜索方法。为了降低多变量牛顿法计算复杂度高，本文改进步长设置策略，提出平衡多步长快速线搜索。利用多步长和平衡多步长快速线搜索，本文提出多步长和平衡多步长快速梯度下降算法，并利用凸函数的Jesen不等式证明它们的收敛性。
4.非负块配准最优梯度法通过分析非负块配准优化子问题的性质，利用最优梯度法交替更新矩阵因子，提出非负块配准最优梯度法。非负矩阵分解优化算法是目前的热点问题，继乘法法则之后出现了非负最小二乘法、投影梯度法、伪牛顿法和Active Set方法等一系列方法。然而，乘法法则收敛速度慢且存在零元素问题；非负最小二乘法无法从理论上保证收敛性；投影梯度法的线搜索过程计算开销过高；伪牛顿法在求解过程中计算Hessian矩阵的逆，计算开销大且存在数值不稳定问题；Active Set在矩阵不满秩时会出现数值问题，且难以用于优化非负块配准模型。因此，非负矩阵分解的高效优化算法仍然是个开放性问题。本文将非负矩阵分解优化问题看成两个子问题，数学上证明了两个子问题都是凸问题且其梯度是Lipschitz连续的，从而利用最优梯度法以O(1/k2)的收敛速度求解每个子问题，从而提出非负矩阵分解高效优化算法，克服了传统非负矩阵分解优化算法的缺点。通过分析非负块配准问题的子问题的性质，本文提出非负块配准最优梯度法。
5.非负矩阵分解在线优化算法提出非负矩阵分解在线优化算法，利用鲁棒随机近似算法以在线的方式更新基矩阵。非负矩阵分解优化算法的空间复杂度与样本维数和样本规模成正比，由于计算机存储器容量的限制，难以满足流数据处理的需求。此外，新样本到达时，传统非负矩阵分解算法需要重新启动以更新分解结果，带来不断增加的巨大时间开销。因此，研究人员提出在线非负矩阵分解算法，利用新到达的样本更新分解结果，克服传统非负矩阵分解算法在时间复杂度和空间复杂度两方面的缺点。然而，已有的在线非负矩阵分解算法的收敛速度受噪音、矩阵不满秩等因素影响，存在数值不稳定问题。本文提出在新样本到达时利用鲁棒随机近似算法以O(1/√k)的收敛速度更新基矩阵，提高在线非负矩阵分解的鲁棒性。利用准鞅理论，本文证明了所提算法的收敛性。为了克服空间复杂度过高的缺点，本文提出用缓冲池技术存储有限量的历史样本，用新样本替换缓冲池中的旧样本以保证基矩阵引入最新的样本统计信息。
Non-negative matrix factorization (NMF) is a novel dimension reduction methodwhich has been widely used in pattern recognition, data mining and information retrievalsince the end of the twentieth century. Distinguished from other matrix decompositionmethods such as singular value decomposition, QR decomposition, Cholesky decompo-sition, LU decomposition, and Eigen-value decomposition, NMF approximates a non-negative matrix by the product of two low-rank non-negative matrices. Since NMF rep-resents sample by an additive combination of features, it yields natural sparse and parts-based representation. Such data representation is consistent with the physiological andpsychological evidence in human brain, and thus NMF usually suppresses the noises. Inaddition, NMF based data representation is consistent with the physical hypothesis in thereal-world because it allow neither negative entries in the features. Therefore, NMF iswidely used in text mining, email surveillance, blind source separation, video analysis,face recognition, image annotation, image segmentation, spectral analysis, gene micro-array analysis etc. Recently, NMF has received more and more attentions from severalinstitutes such as the Bell Laboratory, University of Tennessee, Wake Forest University,Georgia Institute of Technology, University of Helsinki, RIKEN Brain Science Institute,National Tsing Hua University, and Zhejiang University. Their significant works extendthe applications of NMF to many fields such as Internet, Information Security, RemoteSensing and Mathematical Modeling. Thus studies on NMF are practically important.
The thesis first builds up a non-negative patch alignment framework (NPAF) whichunderstands existing non-negative dimension reduction algorithms in a unified viewpointand helps developing new NMF extensions. Then we proposed non-negative discrimi-native locality alignment (NDLA) by using NPAF. NDLA overcomes several drawback-s of NMF and improves its classification performance. Thirdly, to overcome the slowconvergence deficiency of traditional NMF optimization algorithms, we proposed a fastgradient descent (FGD) method which searches the optimal stepsize along the rescalednegative gradient direction by using Newton method. Fourthly, to overcome the defi-ciencies of traditional optimization algorithms for non-negative least squares (NLS), weapplied the optimal gradient method (OGM) to optimizing NLS which converges at therate of O(1/k2) without line search. Furthermore, we proposed an efficient NMF solver as well as an efficient NPAF solver based on OGM. Finally, we proposed an online NM-F optimization algorithm based on robust stochastic approximation (RSA) to reduce thehigh computational overhead of traditional NMF optimization algorithms when runningon streaming data. Benefit from the RSA algorithm, the proposed online NMF algorithmperforms robustly in practice. There are following main contributions:
1. Non-negative patch alignment framework Since NMF was proposed, there weremany works on NMF models. In particular, Li et al. proposed local NMF (L-NMF) which yields parts-based features in most case based on the fact that localvisual features are approximately orthogonal. To incorporate the geometry struc-ture embedded in the data, Cai et al. proposed graph regularized NMF (GNMF).To incorporate the label information, Zaferiou et al. proposed discriminative NMF(DNMF) by using Fisher’s discriminant criteria. However, all these methods aredesigned by researcher’s intuition and experience, and thus it is difficult to under-stand their intrinsic difference. The thesis proposed a non-negative patch alignmentframework which interprets NMF and its extensions from a unified viewpoint. N-PAF not only helps engineers to select suitable models in practice, but also helpsresearchers to develop new NMF models under the NPAF. By using the Lagrangemultipliermethod,wedevelopedamultiplicativeupdaterule(MUR)foroptimizingNPAF. We proved the convergence of MUR by using auxiliary function technique.Note that MUR could optimize most NPAF based models including the traditionalNMF model.
2. Non-negative discriminative locality alignment From the viewpoint of NPAF, L-NMF builds local patch for each sample and base vector by themselves and retainsas much energy as possible in their part optimizations. GNMF builds local patchfor each sample by itself and few limited nearest neighbors and preserves the localgeometric structure in the part optimization, but it completely ignores the label in-formation. Although DNMF incorporates the discriminative information, the builtpatches are global and thus DNMF performs not well especially when data does notobey the Gaussian distribution. We proposed non-negative discriminative localityalignment to overcome the aforementioned drawbacks. NDLA builds two types ofpatchesforeachsample,i.e.,inner-classpatchandbetween-classpatch,whereintheinner-class patch composes of itself and its limited nearest neighbors form the same class as the given sample and the between-class patch composes of itself and itslimited nearest neighbors from the different classes against the given sample. Thepartoptimizationontheinner-classpatchminimumsthedistancesbetweensamplesin identical class to preserve the local geometric structure while the part optimiza-tion on the between-class patch maximums the margins between different classes toincorporate the discriminative information. We alignment both part optimizationsby using the alignment strategy and arrived at the objective of NDLA by combiningthem together. Note that NDLA can be optimized by using the proposed MUR forNPAF.
3. Fast gradient descent for NPAF To overcome the slow convergence deficiency ofMUR, we revisited the proposed MUR from the viewpoint of gradient descent andproposed a fast gradient descent method for NPAF. FGD searches the optimal stepsize along the rescaled negative gradient by using the Newton method and updatesthe matrix factor without exceeding the positive orthant. As FGD uses the optimalstep size, it greatly accelerates the convergence of MUR. By using the Jesen’s in-equality, we proved the convergence of FGD. However, the optimal stepsize mayreduce to1in some cases which makes FGD degenerate to MUR. To overcomessuch deficiency, we set one stepsize for each column (or row) of matrix factor andproposed a multiple stepsize FGD (MFGD). In MFGD, the optimal step size vectoris obtained by using the multivariate Newton method. To reduce the computationalcomplexity of MFGD, we advanced the step size setting strategy and proposed bal-anced multiple stepsize FGD (BFGD). Note that the convergences of both MFGDand BFGD can be proved by using the Jesen’s inequality.
4. OptimalgradientmethodforNPAFByprovingtheconvexityofeachsub-problemof NPAF, we applied the optimal gradient method to alternatively optimizing thematrix factors. The NMF optimization algorithms have received much attention.There exist many methods such as multiplicative update rule, projected gradien-t, quasi-Newton, and active set method. However, they suffer from one or morefollowing drawbacks:(1) slow convergence;(2) high computational complexity;and (3) numerical instability. Therefore, how to efficiently optimize NMF is stil-l an open problem. The thesis proves that each sub-problem of NMF is convexand the gradient is Lipschitz continuous, and thus the optimal gradient method can be applied to optimize each matrix factor at the optimal convergence rate O(1/k2)without line search. Similarly, by proving the convexities of both sub-problems ofNPAF, we proposed an efficient NPAF optimization algorithm based on OGM.
5. OnlineRSAalgorithmforNMFSincethespacecomplexityofthetraditionalNM-F optimization algorithms is proportional to the sample dimensionality and samplenumbers. It prohibits NMF on the streaming dataset. In addition, the traditional N-MFalgorithmsshouldberestartedonthearrivalofnewsampleorchunkofsampleswhich brings in increasing huge time overheads. Therefore, someone proposed on-line NMF algorithms which update both matrix factors on the arrival of new sampleor chunk of samples. However, the existing online NMF algorithms suffer from nu-merical instability problem caused by noise of data and rank-deficiencies of matrixfactors. Thethesisproposedanewonlineoptimizationalgorithmwhichupdatesthebasis matrix by using robust stochastic approximation on the arrival of new sam-ple or chunk of samples. The RSA algorithm converges at the rate of O(1/√k) inmost cases and greatly enhances the robustness of online NMF. By using the theoryof quasi-martingale, we proved the convergence of the proposed online optimiza-tion algorithms. To overcome the high space complexity problem, we introduced abuffer to store limited historical samples and advanced the RSA based algorithm.It replaces the oldest sample to incorporate enough statistical information into thebasis matrix on the arrival of new sample.
本文从框架模型入手，在数学上建立非负块配准模型，从统一的角度分析非负数据降维算法，并用以开发新的非负矩阵分解模型。根据非负块配准模型的分析结果，本文提出非负判别局部块配准模型，克服了传统非负矩阵分解模型的缺点，提高了非负矩阵分解的分类效果。为了克服传统非负矩阵分解优化算法收敛速度慢的缺点，本文利用牛顿法快速搜索最优步长，提出非负块配准快速梯度下降算法。为了克服传统非负最小二乘问题优化算法的缺点，本文利用最优梯度法在无需线搜索的情况下以二阶收敛速度求解非负最小二乘问题。在此基础上提出非负矩阵分解的高效求解算法，并开发非负块配准最优梯度法。为了克服传统优化算法应用于流数据处理时计算开销过大的缺点，本文提出非负矩阵分解在线优化算法，利用鲁棒随机近似算法更新基矩阵，提高在线优化算法的鲁棒性。本文的主要创新点包括：
1.非负块配准模型提出基于块配准的非负矩阵分解框架模型——非负块配准模型。自非负矩阵分解提出以后，提出多种改进模型。利用局部视觉特征近似正交的特点，提出局部非负矩阵分解模型。利用流形学习技术，提出图罚分非负矩阵分解模型保持数据几何结构。利用Fisher判别技术，提出判别非负矩阵分解模型引入标签信息。然而，这些非负矩阵分解模型是由研究人员根据各自的需求和经验而设计的，内在差异巨大，难以理解其共同特性，实际应用中给工程人员选择模型带来困难。本文基于块配准框架，建立非负块配准模型，从统一的角度理解已有非负矩阵分解模型，揭示其内在差异和共同特性，指导工程人员选择模型，帮助研究人员开发新的非负矩阵分解模型。利用拉格朗日乘子法，提出乘法法则算法优化非负块配准模型，并用辅助函数技术证明算法的收敛性。该算法可用于求解非负块配准模型的大多数派生模型，包括标准的非负矩阵分解模型。
2.非负判别局部块配准模型根据非负块配准模型的分析结果，提出非负判别局部块配准模型。从非负块配准模型的角度看，局部非负矩阵分解为样本和基向量分别建立由自身组成的样本块，在局部优化过程保持数据的能量；图罚分非负矩阵分解为样本建立由自身和有限最近邻组成的样本块，在局部优化过程保持数据几何结构，但是忽略样本判别信息；判别非负矩阵分解为样本和各类中心点建立样本块且样本块分别由所有同类样本和中心点组成，在局部优化过程保持数据判别信息，但是由全部样本组成的样本块要求数据服从高斯分布。非负判别局部块配准模型克服了已有非负矩阵分解模型的缺点，为每个样本建立两类样本块：类内块由同类样本中有限最近邻组成，局部优化过程保持数据局部几何结构，放宽数据高斯分布假设；类间块由不同类样本中有限最近邻组成，局部优化过程最大化类间边界，从而保持数据判别信息。因此，非负判别局部块配准模型的分类效果较好、鲁棒性较强。本文利用全局配准技术把两种局部优化过程映射到全局坐标系进行并把二者结合，套用非负块配准的乘法法则算法优化所提非负判别局部块配准模型。
3.非负块配准快速梯度下降算法从梯度下降的角度改进非负块配准乘法法则算法，利用牛顿法实现快速线搜索，提出非负块配准快速梯度下降算法。快速线搜索沿着调整负梯度方向搜索最优步长，在不超出第一象限边界的情况下更新矩阵因子，大大提高乘法法则的收敛速度。利用凸函数的Jesen不等式，证明快速梯度法的收敛性。为了克服矩阵因子整体的最优步长可能为1的缺点，即蜕化成乘法法则，本文为矩阵因子的每列（或行）设置步长，用多变量牛顿法搜索步长向量，提出多步长快速线搜索方法。为了降低多变量牛顿法计算复杂度高，本文改进步长设置策略，提出平衡多步长快速线搜索。利用多步长和平衡多步长快速线搜索，本文提出多步长和平衡多步长快速梯度下降算法，并利用凸函数的Jesen不等式证明它们的收敛性。
4.非负块配准最优梯度法通过分析非负块配准优化子问题的性质，利用最优梯度法交替更新矩阵因子，提出非负块配准最优梯度法。非负矩阵分解优化算法是目前的热点问题，继乘法法则之后出现了非负最小二乘法、投影梯度法、伪牛顿法和Active Set方法等一系列方法。然而，乘法法则收敛速度慢且存在零元素问题；非负最小二乘法无法从理论上保证收敛性；投影梯度法的线搜索过程计算开销过高；伪牛顿法在求解过程中计算Hessian矩阵的逆，计算开销大且存在数值不稳定问题；Active Set在矩阵不满秩时会出现数值问题，且难以用于优化非负块配准模型。因此，非负矩阵分解的高效优化算法仍然是个开放性问题。本文将非负矩阵分解优化问题看成两个子问题，数学上证明了两个子问题都是凸问题且其梯度是Lipschitz连续的，从而利用最优梯度法以O(1/k2)的收敛速度求解每个子问题，从而提出非负矩阵分解高效优化算法，克服了传统非负矩阵分解优化算法的缺点。通过分析非负块配准问题的子问题的性质，本文提出非负块配准最优梯度法。
5.非负矩阵分解在线优化算法提出非负矩阵分解在线优化算法，利用鲁棒随机近似算法以在线的方式更新基矩阵。非负矩阵分解优化算法的空间复杂度与样本维数和样本规模成正比，由于计算机存储器容量的限制，难以满足流数据处理的需求。此外，新样本到达时，传统非负矩阵分解算法需要重新启动以更新分解结果，带来不断增加的巨大时间开销。因此，研究人员提出在线非负矩阵分解算法，利用新到达的样本更新分解结果，克服传统非负矩阵分解算法在时间复杂度和空间复杂度两方面的缺点。然而，已有的在线非负矩阵分解算法的收敛速度受噪音、矩阵不满秩等因素影响，存在数值不稳定问题。本文提出在新样本到达时利用鲁棒随机近似算法以O(1/√k)的收敛速度更新基矩阵，提高在线非负矩阵分解的鲁棒性。利用准鞅理论，本文证明了所提算法的收敛性。为了克服空间复杂度过高的缺点，本文提出用缓冲池技术存储有限量的历史样本，用新样本替换缓冲池中的旧样本以保证基矩阵引入最新的样本统计信息。
Non-negative matrix factorization (NMF) is a novel dimension reduction methodwhich has been widely used in pattern recognition, data mining and information retrievalsince the end of the twentieth century. Distinguished from other matrix decompositionmethods such as singular value decomposition, QR decomposition, Cholesky decompo-sition, LU decomposition, and Eigen-value decomposition, NMF approximates a non-negative matrix by the product of two low-rank non-negative matrices. Since NMF rep-resents sample by an additive combination of features, it yields natural sparse and parts-based representation. Such data representation is consistent with the physiological andpsychological evidence in human brain, and thus NMF usually suppresses the noises. Inaddition, NMF based data representation is consistent with the physical hypothesis in thereal-world because it allow neither negative entries in the features. Therefore, NMF iswidely used in text mining, email surveillance, blind source separation, video analysis,face recognition, image annotation, image segmentation, spectral analysis, gene micro-array analysis etc. Recently, NMF has received more and more attentions from severalinstitutes such as the Bell Laboratory, University of Tennessee, Wake Forest University,Georgia Institute of Technology, University of Helsinki, RIKEN Brain Science Institute,National Tsing Hua University, and Zhejiang University. Their significant works extendthe applications of NMF to many fields such as Internet, Information Security, RemoteSensing and Mathematical Modeling. Thus studies on NMF are practically important.
The thesis first builds up a non-negative patch alignment framework (NPAF) whichunderstands existing non-negative dimension reduction algorithms in a unified viewpointand helps developing new NMF extensions. Then we proposed non-negative discrimi-native locality alignment (NDLA) by using NPAF. NDLA overcomes several drawback-s of NMF and improves its classification performance. Thirdly, to overcome the slowconvergence deficiency of traditional NMF optimization algorithms, we proposed a fastgradient descent (FGD) method which searches the optimal stepsize along the rescalednegative gradient direction by using Newton method. Fourthly, to overcome the defi-ciencies of traditional optimization algorithms for non-negative least squares (NLS), weapplied the optimal gradient method (OGM) to optimizing NLS which converges at therate of O(1/k2) without line search. Furthermore, we proposed an efficient NMF solver as well as an efficient NPAF solver based on OGM. Finally, we proposed an online NM-F optimization algorithm based on robust stochastic approximation (RSA) to reduce thehigh computational overhead of traditional NMF optimization algorithms when runningon streaming data. Benefit from the RSA algorithm, the proposed online NMF algorithmperforms robustly in practice. There are following main contributions:
1. Non-negative patch alignment framework Since NMF was proposed, there weremany works on NMF models. In particular, Li et al. proposed local NMF (L-NMF) which yields parts-based features in most case based on the fact that localvisual features are approximately orthogonal. To incorporate the geometry struc-ture embedded in the data, Cai et al. proposed graph regularized NMF (GNMF).To incorporate the label information, Zaferiou et al. proposed discriminative NMF(DNMF) by using Fisher’s discriminant criteria. However, all these methods aredesigned by researcher’s intuition and experience, and thus it is difficult to under-stand their intrinsic difference. The thesis proposed a non-negative patch alignmentframework which interprets NMF and its extensions from a unified viewpoint. N-PAF not only helps engineers to select suitable models in practice, but also helpsresearchers to develop new NMF models under the NPAF. By using the Lagrangemultipliermethod,wedevelopedamultiplicativeupdaterule(MUR)foroptimizingNPAF. We proved the convergence of MUR by using auxiliary function technique.Note that MUR could optimize most NPAF based models including the traditionalNMF model.
2. Non-negative discriminative locality alignment From the viewpoint of NPAF, L-NMF builds local patch for each sample and base vector by themselves and retainsas much energy as possible in their part optimizations. GNMF builds local patchfor each sample by itself and few limited nearest neighbors and preserves the localgeometric structure in the part optimization, but it completely ignores the label in-formation. Although DNMF incorporates the discriminative information, the builtpatches are global and thus DNMF performs not well especially when data does notobey the Gaussian distribution. We proposed non-negative discriminative localityalignment to overcome the aforementioned drawbacks. NDLA builds two types ofpatchesforeachsample,i.e.,inner-classpatchandbetween-classpatch,whereintheinner-class patch composes of itself and its limited nearest neighbors form the same class as the given sample and the between-class patch composes of itself and itslimited nearest neighbors from the different classes against the given sample. Thepartoptimizationontheinner-classpatchminimumsthedistancesbetweensamplesin identical class to preserve the local geometric structure while the part optimiza-tion on the between-class patch maximums the margins between different classes toincorporate the discriminative information. We alignment both part optimizationsby using the alignment strategy and arrived at the objective of NDLA by combiningthem together. Note that NDLA can be optimized by using the proposed MUR forNPAF.
3. Fast gradient descent for NPAF To overcome the slow convergence deficiency ofMUR, we revisited the proposed MUR from the viewpoint of gradient descent andproposed a fast gradient descent method for NPAF. FGD searches the optimal stepsize along the rescaled negative gradient by using the Newton method and updatesthe matrix factor without exceeding the positive orthant. As FGD uses the optimalstep size, it greatly accelerates the convergence of MUR. By using the Jesen’s in-equality, we proved the convergence of FGD. However, the optimal stepsize mayreduce to1in some cases which makes FGD degenerate to MUR. To overcomessuch deficiency, we set one stepsize for each column (or row) of matrix factor andproposed a multiple stepsize FGD (MFGD). In MFGD, the optimal step size vectoris obtained by using the multivariate Newton method. To reduce the computationalcomplexity of MFGD, we advanced the step size setting strategy and proposed bal-anced multiple stepsize FGD (BFGD). Note that the convergences of both MFGDand BFGD can be proved by using the Jesen’s inequality.
4. OptimalgradientmethodforNPAFByprovingtheconvexityofeachsub-problemof NPAF, we applied the optimal gradient method to alternatively optimizing thematrix factors. The NMF optimization algorithms have received much attention.There exist many methods such as multiplicative update rule, projected gradien-t, quasi-Newton, and active set method. However, they suffer from one or morefollowing drawbacks:(1) slow convergence;(2) high computational complexity;and (3) numerical instability. Therefore, how to efficiently optimize NMF is stil-l an open problem. The thesis proves that each sub-problem of NMF is convexand the gradient is Lipschitz continuous, and thus the optimal gradient method can be applied to optimize each matrix factor at the optimal convergence rate O(1/k2)without line search. Similarly, by proving the convexities of both sub-problems ofNPAF, we proposed an efficient NPAF optimization algorithm based on OGM.
5. OnlineRSAalgorithmforNMFSincethespacecomplexityofthetraditionalNM-F optimization algorithms is proportional to the sample dimensionality and samplenumbers. It prohibits NMF on the streaming dataset. In addition, the traditional N-MFalgorithmsshouldberestartedonthearrivalofnewsampleorchunkofsampleswhich brings in increasing huge time overheads. Therefore, someone proposed on-line NMF algorithms which update both matrix factors on the arrival of new sampleor chunk of samples. However, the existing online NMF algorithms suffer from nu-merical instability problem caused by noise of data and rank-deficiencies of matrixfactors. Thethesisproposedanewonlineoptimizationalgorithmwhichupdatesthebasis matrix by using robust stochastic approximation on the arrival of new sam-ple or chunk of samples. The RSA algorithm converges at the rate of O(1/√k) inmost cases and greatly enhances the robustness of online NMF. By using the theoryof quasi-martingale, we proved the convergence of the proposed online optimiza-tion algorithms. To overcome the high space complexity problem, we introduced abuffer to store limited historical samples and advanced the RSA based algorithm.It replaces the oldest sample to incorporate enough statistical information into thebasis matrix on the arrival of new sample.
引文
[1] Bellman R. Adaptive Control Processes: A Guided Tour [M]. Princeton, NJ:Princetion University Press,1961.
[2] Duda R, Hart P, Stock D. Pattern Classification [M]. Wiley Interscience,2001.
[3] Hotelling H. Analysis of a Complex of Statistical Variables into Principal Compo-nents [J]. Journal of Educational Psychology.1933,24:417–441.
[4] FisherRA.TheUseofMultipleMeasurementsinTaxonomicProblems[J].Annalsof Eugenics.1936,7:179–188.
[5] ComonP.Independentcomponentanalysis,Anewconcept?[J].SignalProcessing.1994,36(3):287–314.
[6] Shawe-Taylor J, Bartlett P, Williamson R, et al. Structural risk minimization overdata-dependent hierarchies [J]. IEEE Transactions on Information Theory.1998,44(5):1926–1940.
[7] Scholkopf B, Smola A, Muller K-R. Nonlinear Component Analysis as a KernelEigenvalue Problem [J]. Neural Computation.1998,10:1299–1319.
[8] Bach F R, Jordan M I. Kernel Independent Component Analysis [J]. Journal ofMachine Learning Research.2002,3:1–48.
[9] Mika S, R tsch G, Weston J, et al. Constructing descriptive and discriminativenonlinear features: Rayleigh coefficients in kernel feature spaces [J]. IEEE Trans-actions on Pattern Analysis and Machine Intelligence.2003,25(5):623–628.
[10] Seung H, Lee D D. The manifold ways of perception [J]. Science.2000,290(5500):2268–2269.
[11] Roweis S, Saul L. Nonlinear Dimensionality Reduction by Locally Linear Embed-ding [J]. Science.2000,29:2323–2326.
[12] Tenenbaum J, Silva V, Langford J. A Global Geometric Framework for NonlinearDimensionality Reduction [J]. Science.2000,290:2319–2323.
[13] BelkinM,NiyogiP.LaplacianEigenmapsandSpectralTechniquesforEmbeddingand Clustering [C]. In Advances in Neural Information Process Systems.2002:585–591.
[14] Roweis S, Saul L. Global coordination of local linear models [C]. In Advances inneural information.2002.
[15] Donoho D, Grimes C. Hessian Eigenmaps: New Locally Linear Embedding Tech-niques for High-dimensional Data [C]. In National Academy of Sciences.2003:5591–5596.
[16] Weinberger K Q, Sha F, Saul L K. Learning a kernel matrix for nonlinear dimen-sionalityreduction[C].InTwenty-firstinternationalconferenceonMachinelearn-ing. New York, New York, USA,2004:106–113.
[17] Zhang Z, Zha H. Principal Manifolds and Nonlinear Dimension Reduction viaLocalTangentSpaceAlignment [J].SIAM Journal ofScientificComputing.2005,26(1):313–338.
[18] Zhang T, Li X, Tao D, et al. Local coordinates alignment and its linearization [C].In Neural Information Processing System.2008:643–652.
[19] He X, Cai D, Yan S. Neighborhood preserving embedding [C]. In ICCV.2005.
[20] Kokiopoulou E, Saad Y. Orthogonal neighborhood preserving projections [J].IEEE transactions on pattern analysis and machine intelligence.2005,29(12):2143–56.
[21] He X F, Niyogi P. Locality Preserving Projections [C]. In Advances in NeuralInformation Process Systems.2003:1–8.
[22] Xu D, Yan S, Tao D, et al. Marginal Fisher Analysis and Its Variants for HumanGait Recognition and Content-Based Image Retrieval [J]. IEEE Transactions onImage Processing.2007,16(11):2811–2821.
[23] Zhang T, Tao D, Yang J. Discriminative Locality Alignment [C]. In The10th Eu-ropean Conference on Computer Vision.2008:725–738.
[24] Simard P, LeCun Y, Denker J. Efficient Pattern Recognition Using a New Trans-formation Distance [C]. In Advances in Neural Information Processing Systems.1992.
[25] Chang H, Yeung D, Xiong Y. Super-resolution through neighbor embedding [C].In International Conference on Computer Vision and Pattern Recognition.2004.
[26] ChanT,ZhangJ.AnImprovedSuper-ResolutionwithManifoldLearningandHis-togram Matching [C]. In International Conference on Pattern Biometric.2006.
[27] Elgammal A, Lee C. Separating style and content on a nonlinear manifold [C]. InInternational Conference on Computer Vision and Pattern Recognition.2004.
[28] Zhang Q, Souvenir R, Pless R. Segmentation Informed by Manifold Learning [C].In EMMCVPR.2005.
[29] Lee D D, Seung H S. Learning the Parts of Objects by Non-negative Matrix Fac-torization.[J]. Nature.1999,401(6755):788–91.
[30] Li S, Hou X, Zhang H, et al. Learning spatially localized, parts-based representa-tion [C]. In IEEE Conference on Computer Vision and Pattern Recognition.2001:207–212.
[31] Zafeiriou S, Tefas A, Buciu I, et al. Exploiting Discriminant Information in Non-negative Matrix Factorization with Application to Frontal Face Verification.[J].IEEE transactions on neural networks.2006,17(3):683–95.
[32] Cai D, He X, Wu X, et al. Non-negative Matrix Factorization on Manifold [C]. InIEEE International Conference on Data Mining. December2008:63–72.
[33] Lee J S, Lee D D, Choi S, et al. Application of Non-negative Matrix Factorizationto Dynamic Positron Emission Tomography [C]. In3rd International ConferenceonIndependentComponentAnalysisandBlindSignalSeparation.2001:629–632.
[34] Donoho D, Stodden V. When Does Non-Negative Matrix Factorization Give aCorrectDecompositionintoParts?[C].InAdvancesinNeuralInformationProcessSystems.2004:1141–1148.
[35] Hoyer P. Non-Negative Matrix Factorization with Sparseness Constraints [C]. InJournal of Machine Learning Research.2004:1457–1469.
[36] Lee D D, Seung H. Algorithms for Non-negative Matrix Factorization [C]. In Ad-vances in Neural Information Process Systems.2001:556–562.
[37] Févotte C, Bertin N, Durrieu J-L. Nonnegative matrix factorization with theItakura-Saito divergence: with application to music analysis.[J]. Neural computa-tion.2009,21(3):793–830.
[38] Cichocki A, Lee H, Kim Y-d, et al. Nonnegative matrix factorization with α-divergence [J]. Pattern Recognition Letters.2008,29(9):1433–1440.
[39] Cichocki A, Cruces S, Amari S-i. Generalized Alpha-Beta Divergences and TheirApplicationtoRobustNonnegativeMatrixFactorization[J].Entropy.2011,13(1):134–170.
[40] Ding C, Xiaofeng H, DSimon H. On the Equivalence of Nonnegative Matrix Fac-torizationandSpectralClustering[C].InSIAMDataMiningConference.Septem-ber2006:606–610.
[41] Ding C, Li T, Peng W. On the equivalence between Non-negative Matrix Factor-ization and Probabilistic Latent Semantic Indexing [J]. Computational Statistics&Data Analysis.2008,52(8):3913–3927.
[42] Vavasis S A. On the complexity of nonnegative matrix factorization [J]. SIAMJournal on Optimization.2009,20(3):1364–1377.
[43] Lin C-J. Projected Gradient Methods for Nonnegative Matrix Factorization [J].Neural computation.2007,19(10):2756–2779.
[44] Kim J, Park H. Toward Faster Nonnegative Matrix Factorization: A New Algo-rithm and Comparisons [J].2008Eighth IEEE International Conference on DataMining.2008:353–362.
[45] Moussaoui S, Brie D, Mohammad-Djafari A, et al. Separation of Non-NegativeMixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sam-pling [J]. IEEE Transactions on Signal Processing.2006,54(11):4133–4145.
[46] Cemgil A T. Bayesian inference for nonnegative matrix factorisation models [J].Computational intelligence and neuroscience.2009,2009:785152.
[47] Kanjani K, Texas A. Parallel Non Negative Matrix Factorization for DocumentClustering [J]. Technical report, Texas A&M University.2007(May).
[48] Bucak S S, Gunsel B. Incremental Subspace Learning via Non-negative MatrixFactorization [J]. Pattern Recognition.2009,42:788–797.
[49] Mairal J, Bach F, Project-team I W, et al. Online Learning for Matrix Factorizationand Sparse Coding [J]. Journal of Machine Learning Research.2010,11:19–60.
[50] Cai D, He X, Wang X. Locality Preserving Nonnegative Matrix Factorization [J].International Joint Conference on Artificial Intelligence.2008:1010–1015.
[51] Owen A B, Perry P O. Bi-cross-validation of the SVD and the non-negative matrixfactorization [J]. Annals of Applied Statistics.2009,3(2):564–594.
[52] Kanagal B, Sindhwani V. Rank Selection in Low-rank Matrix Approximations:A Study of Cross-Validation for NMFs [C]. In Advances in Neural InformationSystems.2010.
[53] Yang J, Yan S, Fu Y, et al. Non-negative Graph Embedding [C]. In IEEE Confer-ence on Computer Vision and Pattern Recognition. June2008:1–8.
[54] Cai D, He X, Member S, et al. Graph Regularized Non-negative Matrix Factor-ization for Data Representation [J]. IEEE Transactions on Pattern Analysis andMachine Intelligence.2011,33(8):1548–1560.
[55] Yan S, Xu D, Zhang B, et al. Graph Embedding and Extensions: A General Frame-workforDimensionalityReduction[J].IEEETransactionsonPatternAnalysisandMachine Intelligence.2007,29(1):40–51.
[56] CaiD,HeX,HanJ.SpectralRegressionforEfficientRegularizedSubspaceLearn-ing [C]. In11th IEEE International Conference on Computer Vision.2007.
[57] Cai D, He X, Han J. SRDA: An Efficient Algorithm for Large-Scale Discrimi-nant Analysis [J]. IEEE Transactions on Knowledge and Data Engineering.2008,20(1):1–8.
[58] Guillamet D, Vitria J, Schiele B. Introducing a weighted non-negative matrix fac-torization for image classification [J]. Pattern Recognition Letters.2003,24(14):2447–2454.
[59] Ho N, Vandooren P. Non-negative matrix factorization with fixed row and columnsums [J]. Linear Algebra and its Applications.2008,429(5-6):1020–1025.
[60] Cybenko G, Crespi V. Learning Hidden Markov Models using Non-Negative Ma-trix Factorization [J]. IEEE Transactions on Information Theory.2011,57(6):3963–3970.
[61] Finesso L, Grassi A, Spreij P. Two-step Nonnegative Matrix Factorization Algo-rithm for the Approximate Realization of Hidden Markov Models [C]. In19thInternational Symposium on Mathematical Theory of Networks and Systems. Bu-dapest, Hungary,2010:369–374.
[62] Lakshminarayanan B, Raich R. Non-negative Matrix Factorization for ParameterEstimation in Hidden Markov Models [C]. In IEEE International Workshop onMachine Learning for Signal Processing. Kittila,2010:89–94.
[63] Yang Z, Zhang H, Yuan Z, et al. Kullback-Leibler Divergence for NonnegativeMatrix Factorization [J]. International Conference on Artificial Neural Networks.2011:250–257.
[64] Cichocki A, Zdunek R, Phan A, et al. Nonnegative Matrix and Tensor Factor-izations: Applications to Exploratory Multi-way Data Analysis and Blind SourceSeparation [M]. Chichester, UK: John Wiley,2009.
[65] Kompass R. A generalized divergence measure for nonnegative matrix factoriza-tion [J]. Neural computation.2007,19(3):780–791.
[66] CichockiA,ZdunekR,AmariSI.Csiszar’sDivergencesforNon-NegativeMatrixFactorization: Family of New Algorithms [J]. Independent Component Analysisand Blind Signal Separation.2006,3889:32–39.
[67] Dhillon I S, Sra S. Generalized Nonnegative Matrix Approximations with Breg-man Divergences [C]. In Advances in Neural Information Process Systems.2005.
[68] Guillamet D, Vitria J. Evaluation of distance metrics for recognition based on non-negative matrix factorization [J]. Pattern Recognition Letters.2003,24(9-10):1599–1605.
[69] Sandler R, Lindenbaum M. Nonnegative Matrix Factorization with Earth Mover’sDistance Metric [C]. In IEEE Conference on Computer Vision and Pattern Recog-nition. January2009:1873–1880.
[70] OlshausenB,FieldD.Sparse CodingwithAnOvercompleteBasisSet:AStrategyEmployed by V1?[J]. Vision Research.1997,37(23):3311–3325.
[71] Hoyer P. Non-negative Sparse Coding [C]. In2th IEEE Workshop on Neural Net-works for Signal Processing.2002:557–565.
[72] Stadlthanner K, Theis F J, Lang E W, et al. Sparse Nonnegative Matrix Factoriza-tion Applied to Microarray Data Sets [J]. Computational Intelligence and Bioin-spired Systems.2005,3512:249–256.
[73] Kim H, Park H. Nonnegative Matrix Factorization Based on Alternating Nonneg-ativity Constrained Least Squares and Active Set Method [J]. SIAM Journal onMatrix Analysis and Applications.2008,30(2):713–730.
[74] Kim J, Park H. Sparse Nonnegative Matrix Factorization for Clustering [J]. CSETechnical Reports, Georgia Institute of Technology.2008.
[75] Kim D, Sra S, Dhillon I S. Fast Newton-type Methods for the Least Squares Non-negative Matrix Approximation Problem [C]. In IEEE International Conferenceon Data Mining.2007:343–354.
[76] Kim H, Park H. Sparse non-negative matrix factorizations via alternating non-negativity-constrainedleastsquaresformicroarraydataanalysis.[J].Bioinformat-ics.2007,23(12):1495–502.
[77] Peharz R, Stark M, Pernkopf F. Sparse nonnegative matrix factorization using l0-constraints [C]. In IEEE International Workshop on Machine Learning for SignalProcessing (MLSP). Kittila,2010:83–88.
[78] Heiler M, Schnorr C. Learning Sparse Representations by Non-Negative MatrixFactorization and Sequential Cone Programming [J]. Journal of Machine LearningResearch.2006,7:1385–1407.
[79] Liu H, Palatucci M, Zhang J. Blockwise Coordinate Descent Procedures for theMulti-task Lasso, with Applications to Neural Semantic Basis Discovery [C]. In26th Annual International Conference on Machine Learning. New York, New Y-ork, USA,2009.
[80] TandonR,SraS.Sparsenonnegativematrixapproximation:newformulationsandalgorithms [J]. Max Planck Institute for Biological Cybernetics, Technical Report.2010.
[81] GillisN,GlineurF.Usingunderapproximationsforsparsenonnegativematrixfac-torization [J]. Pattern Recognition.2010,43(4):1676–1687.
[82] Kim W, Chen B, Kim J, et al. Sparse nonnegative matrix factorization for proteinsequence motif discovery [J]. Expert Systems with Applications.2011,38(10):13198–13207.
[83] Weyrauch B V B, Huang J, Heisele B. Component-based Face Recognition with3DMorphableModels[C].InIEEEWorkshoponFaceProcessinginVideo.Wash-ington, D.C.,2004.
[84] SamariaF,HarterA.ParameterisationofaStochasticModelforHumanFaceIden-tification [C]. In IEEE Workshop on Applications of Computer Vision. SarasotaFL,1994:138–142.
[85] Feng T, Li S Z, Shum H Y, et al. Local non-negative matrix factorization as avisual representation [C]. In2nd International Conference on Development andLearning. ICDL2002.2002:178–183.
[86] GaoY,ChurchG.Improvingmolecularcancerclassdiscoverythroughsparsenon-negative matrix factorization.[J]. Bioinformatics.2005,21(21):3970–5.
[87] Pauca V P, Shahnaz F, Berry M W, et al. Text Mining using Non-Negative MatrixFactorization [C]. In4th SIAM International Conference on Data Mining.2004:452–456.
[88] Schmidt M N, Laurberg H. Nonnegative matrix factorization with Gaussian pro-cess priors [J]. Computational Intelligence and Neuroscience.2008,2008.
[89] Liao S, Lei Z, Li S Z. Nonnegative Matrix Factorization with Gibbs Random Fieldmodeling[J].IEEE12thInternationalConferenceonComputerVisionWorkshop-s.2009:79–86.
[90] Ding C, Li T, Peng W, et al. Orthogonal Nonnegative Matrix Tri-FactorizationsforClustering[C].In12thACMSIGKDDinternationalconferenceonKnowledgediscovery and data mining. New York, New York, USA,2006:126–135.
[91] Li Z, Wu X, Peng H. Nonnegative Matrix Factorization on Orthogonal Sub-space [J]. Pattern Recognition Letters.2010,31(9):905–911.
[92] Choi S. Algorithms for orthogonal nonnegative matrix factorization [C]. In IEEEInternational Joint Conference on Neural Networks. June2008:1828–1832.
[93] Yoo J, Choi S. Non-negative Matrix Factorization with Orthogonality Con-straints [J]. Journal of Computing Science and Engineering.2010,4(2):97–109.
[94] Mirzal A. Converged Algorithms for Orthogonal Nonnegative Matrix Factoriza-tions [J]. arXiv:1010.5290v2.2011.
[95] Lin C-J. On the Convergence of Multiplicative Update Algorithms for Nonnega-tiveMatrixFactorization[J].IEEETransactionsonNeuralNetworks.2007,18(6):1589–1596.
[96] Liu C, He K, Zhou J, et al. Generalized Discriminant Orthogonal Non-negative Matrix [J]. Journal Of Computational Information Systems.2010,6(6):1743–1750.
[97] Zafeiriou S, Tefas A, Pitas I. Discriminant NMF-faces for Frontal Face Verifica-tion [C]. In IEEE Workshop on Machine Learning for Signal Processing.2005:355–359.
[98] Wang Y, Turk M. Fisher non-negative matrix factorization for learning local fea-tures [C]. In Asian Conference on Computer Vision.2004:27–30.
[99] Xue Y, Tong C S, Chen W S. A Modified Non-negative Matrix Factorization Al-gorithmforFaceRecognition[J].18thInternationalConferenceonPatternRecog-nition.2006:495–498.
[100] ZafeiriouS,TefasA,BuciuI,etal.Class-SpecificDiscriminantNon-negativeMa-trix Factorization for Frontal Face Verification [C]. In Lecture Notes in ComputerScience.2005:206–215.
[101] Belkin M. Problems of Learning on Manifolds [D].[S. l.]: University of Chicago,Chicago, IL,2003.
[102] Belkin M, Niyogi P, Sindhwani V. Manifold Regularization: A Geometric Frame-work for Learning from Labeled and Unlabeled Examples [J]. Journal of MachineLearning Research.2006,7:2399–2434.
[103] Bian W, Tao D. Manifold Regularization for SIR with Rate Root-n Conver-gence [C]. In Advances in Neural Information Process Systems.2009:1–8.
[104] An S, Yoo J, Choi S. Manifold-respecting discriminant nonnegative matrix factor-ization [J]. Pattern Recognition Letters.2011,32(6):832–837.
[105] Gu Q, Zhou J. Local Learning Regularized Nonnegative Matrix Factorization [C].In21stInternationalJointConferenceonArtificialIntelligence.2009:1046–1051.
[106] Bottou L, Vapnik V. Local Learning Algorithms [J]. Neural Computation.1992,4(6):888–900.
[107] Wu M, Scholkopf B. A Local Learning Approach for Clustering [C]. In Advancesin Neural Information Processing Systems.2006:1529–1536.
[108] Shen B, Si L. Nonnegative Matrix Factorization Clustering on Multiple Manifold-s [C]. In24th AAAI Conference on Artificial Intelligence.2010:575–580.
[109] Ding C, Li T, Jordan M I. Convex and semi-nonnegative matrix factorization-s.[J].IEEEtransactionsonpatternanalysisandmachineintelligence.2010,32(1):45–55.
[110] Wang H, Nie F, Huang H, et al. Fast Nonnegative Matrix Tri-Factorization forLarge-Scale Data Co-Clustering [C]. In International Joint Conference on Artifi-cial Intelligence.2009:1553–1558.
[111] Guillamet D, Bressan M, Vitria J. A weighted non-negative matrix factorizationfor local representations [C]. In IEEE Computer Society Conference on ComputerVision and Pattern Recognition. CVPR2001.2001:942–947.
[112] Kim Y D, Choi S. Weighted nonnegative matrix factorization [J].2009IEEEInternational Conference on Acoustics, Speech and Signal Processing.2009:1541–1544.
[113] Zhang S, Wang W, Ford J, et al. Learning from Incomplete Ratings Using Non-negative Matrix Factorization [C]. In SIAM International Conference on DataMining.2006:548–552.
[114] GuQ,ZhouJ,DingC.CollaborativeFiltering:WeightedNonnegativeMatrixFac-torization Incorporating User and Item Graphs [C]. In SIAM International Confer-ence on Data Mining.2008:199–210.
[115] Yoo J, Choi S. Weighted Nonnegative Matrix Co-Tri-Factorization for Collabora-tive Prediction [J]. Lecture Notes in Computer Science.2009,5828:396–411.
[116] Lee H, Yoo J, Choi S. Semi-supervised nonnegative matrix factorization [J]. IEEESignal Processing Letters.2010,17(1):4–7.
[117] Laurberg H, Hansen L K. On Affine Non-negative Matrix Factorization [C]. InICASSP.2007:653–656.
[118] Yuan Z, Oja E. Projective Nonnegative Matrix Factorization for Image Compres-sionandFeatureExtraction[C].InInternationalSymposiumonIndependentCom-ponent Analysis and Blind Signal Separation.2004:1–8.
[119] Yang Z, Oja E. Linear and Nonlinear Projective Nonnegative Matrix Factoriza-tion [J]. IEEE transactions on neural networks.2010,21(5):734–49.
[120] Pan B, Lai J, Chen W S. Nonlinear nonnegative matrix factorization based onMercerkernelconstruction[J].PatternRecognition.2011,44(10-11):2800–2810.
[121] Zhang D, Zhou Z H, Chen S. Non-negative Matrix Factorization on Kernels [J].Trends in Artificial Intelligence.2006,4099:404–412.
[122] Buciu I, Nikolaidis N, Pitas I. Nonnegative matrix factorization in polynomial fea-ture space [J]. IEEE Transactions on Neural Networks.2008,19(6):1090–100.
[123] Zafeiriou S, Petrou M. Nonlinear Non-negative Component Analysis Algorithm-s [J]. IEEE Transactions on Image Processing.2010,19(4):1050–1066.
[124] Zhang Z, Li T, Ding C, et al. Binary Matrix Factorization with Applications [C].In IEEE International Conference on Data Mining.2007.
[125] Zdunek R. Data Clustering with Semi-Binary Nonnegative Matrix Factoriza-tion [C]. In Artificial Intelligence and Soft Computing.2008.
[126] Lu H, Vaidya J, Atluri V, et al. Extended Boolean Matrix Decomposition [C]. InIEEE International Conference on Data Mining. December2009:317–326.
[127] KameokaH,OneN,KashinoK,etal.ComplexNMF:Anewsparserepresentationfor acoustic signals [C]. In IEEE International Conference on Acoustics, Speechand Signal Processing. April2009:3437–3440.
[128] Ozerov A, Fevotte C. Multichannel nonnegative matrix factorization in convolu-tive mixtures. With application to blind audio source separation [C]. In IEEE In-ternational Conference on Acoustics, Speech and Signal Processing. April2009:3137–3140.
[129] Ozerov A, Fevotte C. Multichannel Nonnegative Matrix Factorization in Convo-lutive Mixtures for Audio Source Separation [J]. IEEE Transactions on Audio,Speech, and Language Processing.2010,18(3):550–563.
[130] Wang W. Squared Euclidean Distance based Convolutive Non-negative MatrixFactorizationwithMultiplicativeLearningRulesforAudioPatternSeparation[C].In IEEE International Symposium on Signal Processing and Information Technol-ogy.2007:347–352.
[131] Lee H, Choi S. Group Nonnegative Matrix Factorization for EEG Classifica-tion [C]. In International Conference on Machine Learning.2009:320–327.
[132] Lee H, Cichocki A, Choi S. Kernel nonnegative matrix factorization for spectralEEG feature extraction [J]. Neurocomputing.2009,72(13-15):3182–3190.
[133] Gillis N, Plemmons R J. Sparse Nonnegative Matrix Underapproximation and it-s Application to Hyperspectral Image Analysis [C]. In Hyperspectral Image andSignal Processing: Evolution in Remote Sensing.2011.
[134] LaurbergH,ChristensenMG,PlumbleyMD,etal.TheoremsonPositiveData:Onthe Uniqueness of NMF.[J]. Computational intelligence and neuroscience.2008,2008(2):764206.
[135] Welling M, Chemudugunta C, Sutter N. Deterministic Latent Variable Models andtheir Pitfalls [C]. In SIAM International Conference on Data Mining.2008.
[136] Xu W, Liu X, Gong Y. Document clustering based on non-negative matrix fac-torization [C]. In The26th annual international ACM SIGIR conference on Re-search and development in informaion retrieval. New York, New York, USA,2003:267–273.
[137] Gaussier E, Goutte C. Relation between PLSA and NMF and Implications Cat-egories and Subject Descriptors [C]. In28th annual international ACM SpecialInterest Group on Information Retrieval.2005:601–602.
[138] Ding C, Li T, Peng W. Nonnegative Matrix Factorization and Probabilistic LatentSemantic Indexing: Equivalence, Chi-square Statistic, and a Hybrid Method [C].In21st National Conference on Artificial Intelligence.2006:342–347.
[139] LiT,DingC.TheRelationshipsAmongVariousNonnegativeMatrixFactorizationMethods for Clustering [J]. Sixth International Conference on Data Mining.2006:362–371.
[140] Peng W, Li T. On the equivalence between nonnegative tensor factorization andtensorial probabilistic latent semantic analysis [J]. Applied Intelligence.2010.
[141] Wild S, Curry J, Dougherty A. Improving non-negative matrix factorization-s through structured initialization [J]. Pattern Recognition.2004,37(11):2217–2232.
[142] Langville A N, Meyer C D, Albright R. Initializations for the Nonnegative Ma-trix Factorization [C]. In ACM SIGKDD International Conference on KnowledgeDiscovery and Data Mining.2006.
[143] Zheng Z, Yang J, Zhu Y. Initialization enhancer for non-negative matrix factoriza-tion[J].EngineeringApplicationsofArtificialIntelligence.2007,20(1):101–110.
[144] Xue Y, Tong C, Chen Y, et al. Clustering-based initialization for non-negativematrix factorization [J]. Applied Mathematics and Computation.2008,205(2):525–536.
[145] BoutsidisC,GallopoulosE.SVDbasedinitialization:Aheadstartfornonnegativematrix factorization [J]. Pattern Recognition.2008,41(4):1350–1362.
[146] JanecekA,TanY.UsingPopulationBasedAlgorithmsforInitializingNonnegativeMatrix Factorization [J]. Advances in Swarm Intelligence.2011,6729:307–316.
[147] Rezaei M, Boostani R. An Efficient Initialization Method for Nonnegative MatrixFactorization [J]. Journal of Applied Science.2011,11(2):354–359.
[148] Wild S. Seeding Non-Negative Matrix Factorizations with the Spherical K-MeansClustering by [D].[S. l.]: University of Colorado,2003.
[149] Gonzalez E F, Zhang Y. Accelerating the Lee-Seung Algorithm for NonnegativeMatrix Factorization [R].2005.
[150] Gillis N, Glineur F. Accelerated Multiplicative Updates and Hierarchical ALSAlgorithms for Nonnegative Matrix Factorization [J]. Neural computation.2011,23(9):2421–56.
[151] Bertin N, Badeau R, Vincent E. Fast bayesian nmf algorithms enforcing harmonic-ity and temporal continuity in polyphonic music transcription [J]. IEEE Workshopon Applications of Signal Processing to Audio and Acoustics.2009:29–32.
[152] Badeau R, Bertin N, Vincent E. Stability analysis of multiplicative update algo-rithms and application to nonnegative matrix factorization [J]. IEEE transactionson neural networks.2010,21(12):1869–81.
[153] Sra S. Block-Iterative Algorithms for Non-Negative Matrix Approximation [C].In IEEE International Conference on Data Mining.2008:1037–1042.
[154] Berry M, Browne M, Langville A, et al. Algorithms and applications for approxi-mate nonnegative matrix factorization [J]. Computational Statistics&Data Anal-ysis.2007,52(1):155–173.
[155] Cichocki A, Zdunek R, Amari S-i. Hierarchical ALS Algorithms for NonnegativeMatrixand3DTensorFactorization[J].LectureNotesinComputerScience.2007,4666:169–176.
[156] Cichocki A, Phan A H, Caiafa C. Flexible HALS algorithms for sparse non-negative matrix/tensor factorization [C]. In IEEE Workshop on Machine Learningfor Signal Processing. October2008:73–78.
[157] Ho N D, Dooren P V, Blondel V D. Descent Methods for Nonnegative Matrix Fac-torization [J]. Numerical Linear Algebra in Signals, Systems and Control.2011,80:251–293.
[158] Liu H, Zhou Y. Rank-two residue iteration method for nonnegative matrix factor-ization [J]. Neurocomputing.2011.
[159] Biggs M, Ghodsi A, Vavasis S. Nonnegative matrix factorization via rank-onedowndate [J].25th international conference on Machine learning.2008:64–71.
[160] Bonettini S. Inexact Block Coordinate Descent Methods with Application to theNonnegative Matrix Factorization [J]. IMA Journal of Numerical Analysis.2011.
[161] Grippo L, Sciandrone M. On the Convergence of the Block Nonlinear Gauss-Seidel Method Under Convex Constraints [J]. Operations Research Letters.2000,26(3):127–136.
[162] Merrit M, Zhang Y. Interior-Point Gradient Method for Large-Scale Totally Non-negative Least Squares Problems [J]. Journal of Optimization Theory and Appli-cations.2005,126(1):191–202.
[163] Han L, Neumann M, Prasad U. Alternating Projected Barzilai-Borwein Methodsfor Nonnegative Matrix Factorization [J]. Electronic Transactions on NumericalAnalysis.2009,36(06):54–82.
[164] Zdunek R, Cichocki A. Nonnegative matrix factorization with Quasi-Newton op-timization [C]. In Lecture Notes in Artificial Intelligence. June2006:2309–2320.
[165] Zdunek R, Cichocki A. Non-negative Matrix Factorization with Quasi-NewtonOptimization [C]. In The8th International Conference on Artificial Intelligenceand Soft Computing.2006:870–879.
[166] Hsieh C J, Dhillon I S. Fast Coordinate Descent Methods with Variable Selectionfor Non-Negative Matrix Factorization [R].2011.
[167] Schmidt M N, Winther O, Hansen L K. Bayesian Non-negative Matrix Factoriza-tion [J]. Lecture Notes in Artificial Intelligence.2009,5441:540–547.
[168] Wang F, Li P. Efficient Nonnegative Matrix Factorization with Random Projec-tions [C]. In The10th SIAM International Conference on Data Mining.2009:281–292.
[169] Cichocki A, Amari S I, Hori G, et al. Extended SMART Algorithms for Non-negative Matrix Factorization [J]. Lecture Notes in Artificial Intelligence.2006,4029:548–562.
[170] Cichocki A, Zdunek R. Multilayer Nonnegative Matrix Factorization using Pro-jected Gradient Approaches [J]. International Journal Neural Systems.2007,17(6):431–446.
[171] Gillis N, Glineur F. A Multilevel Approach for Nonnegative Matrix Factoriza-tion [J]. CORE Discussion Paper.2010.
[172] Dong C, Zhao H, Wang W. Parallel Nonnegative Matrix Factorization AlgorithmontheDistributedMemoryPlatform[J].InternationalJournalofParallelProgram-ming.2009,38(2):117–137.
[173] Cao B, Shen D, Sun J-t, et al. Detect and Track Latent Factors with Online Non-negative Matrix Factorization [C]. In The20th International Joint Conference onArtificial Intelligence. San Francisco,2007:2689–2694.
[174] Wang F, Tan C, Knoig A C, et al. Efficient Document Clustering via Online Non-negativeMatrixFactorization[C].InIEEEInternationalConferenceonDataMin-ing.2010.
[175] Robila S, Maciak L. Considerations on Parallelizing Nonnegative Matrix Factor-ization for Hyperspectral Data Unmixing [J]. IEEE Geoscience and Remote Sens-ing Letters.2009,6(1):57–61.
[176] LiuC,YangH-c,FanJ,etal.Distributednonnegativematrixfactorizationforweb-scale dyadic data analysis on mapreduce [C]. In19th international conference onWorld wide web. New York, New York, USA,2010.
[177] Sun Z, Li T, Rishe N. Large-Scale Matrix Factorization Using MapReduce [J].IEEE International Conference on Data Mining Workshops.2010:1242–1248.
[178] Gautam B P, Shrestha D, Iaeng M. Document Clustering Through Non-NegativeMatrixFactorization:ACaseStudyofHadoopforComputationalTimeReductionof Large Scale Documents [C]. In26th annual international ACM SIGIR confer-ence on Research and development in informaion retrieval.2010:1–6.
[179] ShahnazF,BerryM,PaucaV,etal.Documentclusteringusingnonnegativematrixfactorization [J]. Information Processing&Management.2006,42(2):373–386.
[180] Park S, An D U, Char B, et al. Document Clustering with Cluster Refinement andNon-negative Matrix Factorization [J]. Lecture Notes in Computer Science.2009,5864:281–288.
[181] Hu C, Zhang B, Yan S, et al. Mining Ratio Rules Via Principal Sparse Non-Negative Matrix Factorization [J]. Fourth IEEE International Conference on DataMining.2004:407–410.
[182] Psorakis I, Roberts S, Ebden M, et al. Overlapping community detection usingBayesian non-negative matrix factorization.[J]. Physical review. E, Statistical,nonlinear, and soft matter physics.2011,83(6-2):066114.
[183] Newman M. Networks: An Introduction [M]. Oxford University Press,2010.
[184] Kersting K, Wahabzada M, Thurau C, et al. Convex NMF on Non-Convex MassivData [J]. Knowledge Discovery and Machine Learning Cone, Technical Report.2005.
[185] Li T, Ding C, Jordan M I. Solving Consensus and Semi-supervised ClusteringProblems Using Nonnegative Matrix Factorization [J]. Seventh IEEE Internation-al Conference on Data Mining.2007,2(1):577–582.
[186] Chen Y, Rege M, Dong M, et al. Non-negative matrix factorization for semi-supervised data clustering [J]. Knowledge and Information Systems.2008,17(3):355–379.
[187] Chen Y, Wang L, Dong M. Non-Negative Matrix Factorization for Semi-supervised Heterogeneous Data Co-clustering [J]. IEEE Transactions on Knowl-edge and Data Engineering.2010,22(10):1459–1474.
[188] Guillamet D, Schiele B, Vitria J. Analyzing non-negative matrix factorization forimage classification [J]. Object recognition supported by user interaction for ser-vice robots.2002,2:116–119.
[189] GuillametD,VitriaJ.Non-negativeMatrixFactorizationforFaceRecognition[J].Lecture Notes in Artificial Intelligence.2002,2504:336–344.
[190] Okun O G. Non-negative Matrix Factorization and Classifiers: Experimental S-tudy [C]. In Fourth IASTED International Conference Proceedings, Visualization,Imaging, and Image Processing.2004.
[191] Kim H, Park H. Discriminant analysis using nonnegative matrix factorization fornonparametric multiclass classification [C]. In IEEE International Conference onGranular Computing.2006:182–187.
[192] Guillamet D, Vitria J. Determining a suitable metric when using non-negative ma-trix factorization [C]. In16th International Conference on Pattern Recognition.2002:128–131.
[193] Kim J, Choi J, Yi J, et al. Effective representation using ICA for face recognitionrobust to local distortion and partial occlusion.[J]. IEEE transactions on patternanalysis and machine intelligence.2005,27(12):1977–81.
[194] Neo H F, Teoh B J, Ngo C L. A Novel Spatially Confined Non-Negative MatrixFactorization for Face Recognition [C]. In IAPR Conference on Machine VisionApplications.2005:502–505.
[195] ZhangD,ChenS,ZhouZ-h.Two-DimensionalNon-negativeMatrixFactorizationfor Face Representation and Recognition [J]. Lecture Notes in Computer Science.2005,3723:350–363.
[196] Buciu I, Nikolaidis N, Pitas I. A comparative study of NMF, DNMF, and LNM-F algorithms applied for face recognition [C]. In The European Association forSignal Processing.2006.
[197] Chen W S, Pan B, Fang B, et al. Incremental Nonnegative Matrix Factorizationfor Face Recognition [J]. Mathematical Problems in Engineering.2008,2008.
[198] Oh H, Lee K, Lee S. Occlusion invariant face recognition using selective localnon-negative matrix factorization basis images [J]. Image and Vision Computing.2008,26(11):1515–1523.
[199] Zhang T, Fang B, Tang Y Y, et al. Topology Preserving Non-negative Matrix Fac-torization for Face Recognition [J]. IEEE transactions on image processing.2008,17(4):574–84.
[200] Yin H, Liu H. Nonnegative matrix factorization with bounded total variationalregularization for face recognition [J]. Pattern Recognition Letters.2010,31(16):2468–2473.
[201] Buciu I, Pitas I. Application of non-negative and local non negative matrix fac-torization to facial expression recognition [J]. International Conference on PatternRecognition,.2004,1:288–291.
[202] Zhi R, Flierl M, Ruan Q, et al. Graph-preserving sparse nonnegative matrix fac-torization with application to facial expression recognition [J]. IEEE transactionson systems, man, and cybernetics. Part B, Cybernetics.2011,41(1):38–52.
[203] Teoh A, Neo H, Ngo D. Sorted locally confined non-negative matrix factorizationin face verification [C]. In International Conference on Communications, Circuitsand Systems.2005.
[204] Yuan L, Mu Z C, Zhang Y. Ear Recognition using Improved Non-Negative MatrixFactorization [C]. In18th International Conference on Pattern Recognition.2006:501–504.
[205] Soukup D, Bajla I. Robust object recognition under partial occlusions using NM-F [J]. Computational intelligence and neuroscience.2008,2008.
[206] Barman P C, Lee S-Y. Document Classification with Unsupervised NonnegativeMatrixFactorizationandSupervisedPercetronLearning[C].In2007InternationalConference on Information Acquisition. July2007:182–186.
[207] Chikhi N F, Rothenburger B, Aussenac-gilles N, et al. Authoritative DocumentsIdentification Based On nonnegative Matrix Factorization [C]. In IEEE Interna-tional Conference on Information Reuse and Integration.2008:262–267.
[208] Berry M W,Gillis N, Glineur F.Document classification using nonnegative matrixfactorization and underapproximation [C]. In IEEE International Symposium onCircuits and Systems. May2009:2782–2785.
[209] BenAbdallah J, Caicedo J C, Gonzalez F a, et al. Multimodal Image AnnotationUsing Non-negative Matrix Factorization [C]. In IEEE/WIC/ACM InternationalConference on Web Intelligence and Intelligent Agent Technology. August2010:128–135.
[210] Okun O, Priisalu H. Nonnegative matrix factorization for pattern recognition [C].In5th International Conference on Visualization, Imaging, and Image Processing.Benidorm, Spain,2005:546–551.
[211] Okun O, Priisalu H. Fast Nonnegative Matrix Factorization and Its Application forProteinFoldRecognition[J].EURASIPJournalonAdvancesinSignalProcessing.2006,2006:1–9.
[212] Li Y, Cichocki A. Non-negative matrix factorization and its application in blindsparse source separation with less sensors than sources [J]. The International Jour-nal for Computation and Mathematics in Electrical and Electronic Engineering.2005,24(2):695–706.
[213] Cichocki A, Zdunek R, Amari S. New Algorithms for Non-Negative Matrix Fac-torization in Applications to Blind Source Separation [C]. In IEEE Internation-al Conference on Acoustics Speed and Signal Processing Proceedings.2006:621–624.
[214] ZhangJ,WeiL,FengX,etal.Patternexpressionnonnegativematrixfactorization:algorithm and applications to blind source separation [J]. Computational intelli-gence and neuroscience.2008,2008:168–177.
[215] Mouri M, Funase A, Cichocki A, et al. Applying Non-negative Matrix Factor-ization for Global Signal Elimination from Electromagnetic Signals [C]. In15thEuropean Signal Processing Conference. Poznan, Poland,2007:2444–2448.
[216] Smaragdis P, Brown J C. Non-Negative Matrix Factorization for Polyphonic Mu-sic Transcription [C]. In IEEE Workshop on Application of Signal Processing toAudio and Acoustics.2003:177–180.
[217] Benetos E, Kotti M, Kotropoulos C. Applying Supervised Classifiers Based onNon-negative Matrix Factorization to Musical Instrument Classification [J]. IEEEInternational Conference on Multimedia and Expo.2006:2105–2108.
[218] Bertin N, Badeau R, Vincent E. Enforcing Harmonicity and Smoothness inBayesian Non-Negative Matrix Factorization Applied to Polyphonic Music Tran-scription [J]. IEEE Transactions on Audio, Speech, and Language Processing.2010,18(3):538–549.
[219] Cheng C C, Hu D J, Saul L K. Nonnegative Matrix Factorizatino for Real TimeMusical Analysis and Sight-Reading Evaluation [C]. In IEEE International Con-ference on Acoustics, Speech and Signal Processing.2008:2017–2020.
[220] Holzapfel A, Stylianou Y. Musical Genre Classification Using Nonnegative Ma-trix Factorization-Based Features [J]. IEEE Transactions on Audio, Speech, andLanguage Processing.2008,16(2):424–434.
[221] Costantini G, Todisco M, Saggio G, et al. Musical Onset Detection by meansof Non-Negative Matrix Factorization [C]. In Latest Trends on Systems.2010:206–209.
[222] Behnke S. Discovering hierarchical speech features using convolutional non-negativematrixfactorization[C].InInternationalJointConferenceonNeuralNet-works.2003:2758–2763.
[223] Sha F, Saul L K. Real-Time Pitch Determination of One or More Voices by Non-negativeMatrixFactorization[J].AdvancesinNeuralInformationProcessingSys-tems.2005,17.
[224] Kellis E, Galanis N, Natsis K, et al. Non-negative Matrix Factorization for NoteOnset Detection of Audio Signals [C]. In16th IEEE Signal Processing SocietyWorkshop on Machine Learning for Signal Processing. August2011:447–452.
[225] Virtanen T. Monaural Sound Source Separation by Nonnegative Matrix Factoriza-tion With Temporal Continuity and Sparseness Criteria [J]. IEEE Transactions onAudio, Speech and Language Processing.2007,15(3):1066–1074.
[226] Stouten V, Demuynck K, Van hamme H. Discovering Phone Patterns in SpokenUtterancesbyNon-NegativeMatrixFactorization[J].IEEESignalProcessingLet-ters.2008,15:131–134.
[227] Ogrady P, Pearlmutter B. Discovering speech phones using convolutive non-negative matrix factorisation with a sparseness constraint [J]. Neurocomputing.2008,72(1-3):88–101.
[228] Schuller B, Weninger F. Discrimination of Speech and Non-linguistic Vocaliza-tions by Non-negative Matrix Factorization [C]. In IEEE International Conferenceon Acoustics Speech and Signal Processing. Dallas, TX,2010:5054–5057.
[229] Weninger F, Schuller B, Batliner A, et al. Recognition of Nonprototypical Emo-tions in Reverberated and Noisy Speech by Nonnegative Matrix Factorization [J].EURASIP Journal on Advances in Signal Processing.2011,2011:1–16.
[230] Miao L, Qi H. Endmember Extraction From Highly Mixed Data Using MinimumVolume Constrained Nonnegative Matrix Factorization [J]. IEEE Transactions onGeoscience and Remote Sensing.2007,45(3):765–777.
[231] Jia S, Qian Y. L1/2Constrained Nonnegative Matrix Factorization for Hyperspec-tral Unmixing [J]. IEEE Transactions on Geoscience and Remote Sensing.2009,47(1):161–173.
[232] Sajda P, Du S, Brown T R, et al. Nonnegative matrix factorization for rapid re-covery of constituent spectra in magnetic resonance chemical shift imaging of thebrain.[J]. IEEE transactions on medical imaging.2004,23(12):1453–65.
[233] HamzaA,BradyD.Reconstructionofreflectancespectrausingrobustnonnegativematrix factorization [J]. IEEE Transactions on Signal Processing.2006,54(9):3637–3642.
[234] Pauca V, Piper J, Plemmons R. Nonnegative matrix factorization for spectral dataanalysis [J]. Linear Algebra and its Applications.2006,416(1):29–47.
[235] Liu W, Yuan K, Ye D. Reducing microarray data via nonnegative matrix factor-ization for visualization and clustering analysis.[J]. Journal of biomedical infor-matics.2008,41(4):602–6.
[236] Wang R, Zhang S, Wang Y, et al. Clustering complex networks and biologicalnetworksbynonnegativematrixfactorizationwithvarioussimilaritymeasures[J].Neurocomputing.2008,72(1-3):134–141.
[237] Fogel P, Young S S, Hawkins D M, et al. Inferential, robust non-negative matrixfactorization analysis of microarray data.[J]. Bioinformatics.2007,23(1):44–9.
[238] Fujiwara T, Ishikawa S, Hoshida Y, et al. Non-Negative Matrix Factorization ofLung Adenocarcinoma Expression Profiles [C]. In Conference on Genome Infor-matics.2005:3–4.
[239] Chen Z, Cichochi A, Rutkowski T M. Constrained Non-negative Matrix Factor-izationMethodsforEEGAnalysisInEarlyDetectionofAlzheimerDisease[C].InIEEEInternationalConferenceonAcoustics,SpeechandSignalProcessing.2006.
[240] Kim S P, Rao Y N, Erdogmus D, et al. Determining Patterns in Neural Activityfor Reaching Movements Using Nonnegative Matrix Factorization [J]. EURASIPJournal on Advances in Signal Processing.2005,2005(19):3113–3121.
[241] Lee J S, Lee D D, Choi S, et al. Non-negative matrix factorization of dynamicimages in nuclear medicine [C]. In IEEE Nuclear Science Symposium ConferenceRecord.2002:2027–2030.
[242] Lohmann G, Volz K G, Ullsperger M. Using non-negative matrix factorization forsingle-trial analysis of fMRI data.[J]. NeuroImage.2007,37(4):1148–60.
[243] Drakakis K, Rickard S, Frein R D, et al. Analysis of Financial Data using Non-Negative Matrix Factorization [C]. In International Mathematical Forum.2008:1853–1870.
[244] Ribeiro B, Silva C, Vieira A, et al. Extracting Discriminative Features Using Non-negative Matrix Factorization in Financial Distress Data [J]. Lecture Notes in Ar-tificial Intelligence.2009,5495:537–547.
[245] Xu B, Lu J, Huang G. A constrained non-negative matrix factorization in infor-mation retrieval [C]. In Fifth IEEE Workshop on Mobile Computing Systems andApplications.2003:273–277.
[246] TsugeS,ShishiboriM.DimensionalityReductionusingNon-negativeMatrixFac-torization for Information Retrieval [C]. In IEEE International Conference on Sys-tems, Man, and Cybernetics.2001:960–965.
[247] Berry M W, Browne M. Email Surveillance Using Non-negative Matrix Factor-ization [J]. Computational and Mathematical Organization Theory.2006,11(3):249–264.
[248] Szupiluk R, Wojewnik P, Zabkowski T. Ensemble methods with non-negative ma-trix factorization for non-payment prevention system [C]. In11th WSEAS Inter-national Conference on SYSTEMS. Agios Nikolaos, Crete Island, Greece,2007:385–388.
[249] Guan X, Wang W, Zhang X. Fast intrusion detection based on a non-negative ma-trixfactorizationmodel[J].JournalofNetworkandComputerApplications.2009,32(1):31–44.
[250] Wachsmuth E, Oram M, Perrett D. Recognition of Objects and Their ComponentParts: Responses of Single Units in the Temporal Cortex of the Macaque [J]. Cere-bral Cortex.1994,4:509–522.
[251] LogothetisN,SheinbergD.VisualObjectRecognition[J].AnnualReviewofNeu-roscience.1996,19:577–621.
[252] Palmer S. Hierarchical Structure in Perceptual Representation [J]. Cognitive Psy-chology.1977,9:441–474.
[253] Zhang T, Tao D, Li X, et al. Patch Alignment for Dimensionality Reduction [J].IEEETransactionsonKnowledgeandDataEngineering.2009,21(9):1299–1313.
[254] Bertsekas D. Nonlinear Programming [M].2nd ed. Belmont: MA: Athena Scien-tific,1999.
[255] Saul L K, Roweis S T. Think Globally, Fit Locally: Unsupervised Learning ofLow Dimensional Manifolds [J]. Journal of Machine Learning Research.2003,4:119–155.
[256] Graham D, Allinson N. Characterising Virtual Eigensignatures for General Pur-pose Face Recognition [M]. Berlin: Springer,1998.
[257] LeCun Y, Bottou L, Yoshua B, et al. Gradient-Based Learning Applied to Docu-ment Recognition [J]. IEEE Conferences on Intelligent Signal Processing.1998,86(11):2278–2324.
[258] Alpaydin E. Combined5×2cv F Test for Comparing Supervised ClassificationLearning Algorithms [J]. Neural Computation.1999,11:1885–1892.
[259] Fukunaga K. Introduction to Statistical Pattern Recognition [M]. New York, USA:Academic Press,1990.
[260] Guan N, Tao D, Luo Z, et al. Non-negative Patch Alignment Framework [J]. IEEETransactions on Neural Networks.2011,22(8):1218–1230.
[261] Berman A, Plemmons R J. Nonnegative Matrices in the Mathematical Sci-ences [M]. New York, USA: Academic Press,1994.
[262] Alefeld G, Schneider N. On Square Roots of M-Matrices [J]. Linear Algebra andIts Applications.1982,42(4):119–132.
[263] Guan N, Tao D, Luo Z, et al. Manifold Regularized Discriminative Non-negativeMatrix Factorization with Fast Gradient Descent [J]. IEEE Transactions on ImageProcessing.2011,20(7):2030–2048.
[264] DietterichT.ApproximateStatisticalTestsforComparingSupervisedClassifiationLearning Algorithms [J]. Neural Computation.1998,10:1895–1923.
[265] Liang Z, Li Y, Zhao T. Projected Gradient Method for Kernel Discriminant Non-negative Matrix Factorization and the Applications [J]. Signal Processing.2010,90(7):2150–2163.
[266] Kim H, Park H. Cancer Class Discovery Using Non-negative Matrix Factoriza-tion Based on Alternating Non-negativity-Constrained Least Squares [J]. LectureNotes in Biology Information.2007,4463:477–487.
[267] Horn R, Johnson C. Matrix Analysis [M]. Cambridge University Press,1985.
[268] Woodbury M A. Inverting Modified Matrices [M]. N. J.: Princeton UniversityPress,1950.
[269] Bertsekas D P. On the Goldstein-Levitin-Polyak Gradient Projection Method [J].IEEE Transactions on Automatic Control.1976,21(2):174–184.
[270] Belhumeour P, Hespanha J, Kriegman D. Eigenfaces vs. Fisherfaces: RecognitionUsing Class Sepcific Linear Projection [J]. IEEE Transactions on Pattern Analysisand Machine Intelligence.1997,19(7):711–720.
[271] Nesterov Y E. A Method of Solving A Convex Programming Problem with Con-vergence Rate O(1/k2)[J]. Soviet Mathematics Doklady.1983,27(2):372–376.
[272] Beck A, Teboulle M. A Fast Iterative Shrinkage-Thresholding Algorithm for Lin-ear Inverse Problems [J]. SIAM Journal of Imaging Sciences.2009,2(1):183.
[273] Ji S, Ye J. An Accelerated Gradient Method for Trace Norm Minimization [C].In The26th Annual International Conference on Machine Learning. New York,USA,2009:457–464.
[274] Zhou T, Tao D. Fast Gradient Clustering [C]. In NIPS2009Workshop on DiscreteOptimization in Machine Learning: Submodularity, Sparsity&Polyhedra.2009.
[275] Nesterov Y. Introductory lectures on convex optimization: A Basic Course [M].Kluwer Academic Publishers,2003.
[276] Duchi J. Adaptive Subgradient Methods for Online Learning and Stochastic Opti-mization [C]. In COLT2010.2010.
[277] Tseng P D O M O W. On Accelerated Proximal Gradient Methods for Convex-Concave Optimization [J]. SIAM Journal on Optimization.2008.
[278] NesterovY.Smooth minimizationof non-smooth functions [J].Mathematical Pro-gramming.2005,103(1):127–152.
[279] Lin X. Dual Averaging Methods for Regularized Stochastic Learning and OnlineOptimization [C]. In NIPS.2009.
[280] Rockafellar R. Convex Analysis [M]. Princetion University Press,1970.
[281] Lewis D, Yang Y, Rose T, et al. RCV1: A New Benchmark Collection for Tex-t Categorization Research [J]. Journal of Machine Learning Research.2004,5:361–397.
[282] Cieri C, Graff D, Liberman M, et al. The TDT-2Text and Speech Corpus [C]. InDARPA Broadcast News Workshop.1999.
[283] Nemirovski A, Juditsky A, Lan G, et al. Robust Stochastic Approximation Ap-proach to Stochastic Programming [J]. SIAM Journal on Optimization.2009,19(4):1574–1609.
[284] Cao S, Liu L. Non-negative Matrix Factorization and its Applications to GeneExpression Data Analysis [J]. Journal of Beijing Normal University (Natural Sci-ence).2007,43(1):30–33.
[285] Bottou L, Bousquet O. The Trade-offs of Large Scale Learning [C]. In Advancesin Neural Information Process Systems.2008:161–168.
[286] Strassen V. The Existence of Probability Measures with Given Marginals [J]. An-nals of Mathematical Statistics.1965,38:423–439.
[287] Vaart A W V D. Asymptotic Statistics [M]. Cambridge University Press,1998.
[288] Fisk D L. Quasi-martingales [J]. Transactions of the American Mathematical So-ciety.1965,120(3):369–389.
[289] DuchiJ,ChandraT,ComTG.EfficientProjectionsontothel1-BallforLearninginHighDimensions[C].InInternationalConferenceonMachineLearning.Helsinki,Finland,2008.
[290] TurkMA,PentlandAP.Facerecognitionusingeigenfaces[C].InIEEEComputerSociety Conference on Computer Vision and Pattern Recognition. Maui, Hawaii,1991:586–591.
[291] Sim T, Baker S, Bsat M. The CMU Pose, Illumination, and ExpressionDatabase [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence.2003,25(12):1615–1618.
[292] Lovasz L, Plummer M. Matching Theory [M]. North Holland: Elsevier,1986.
[293] Duygulu P, Freitas N, Barnard K, et al. Object Recognition as Machine Transla-tion: Learning a Lexicon for a Fxed Image Vocabulary [C]. In The7th EuropeanConference on Computer Vision.2002:97–112.
[294] Grubinger M, Clough P D, Henning M, et al. The IAPR Benchmark: a New Eval-uation Resource for Visual Information Systems [C]. In International Conferenceon Language Resources and Evaluation. Genoa, Italy,2006.
[295] Ahn J H, Kim S K, Oh J H, et al. Multiple nonnegative-matrix factorization ofdynamic pet images [C]. In International Conference on Computer Vision.2004.
[296] Makadia A, Pavlovic V, Kumar S. A New Baseline for Image Annotation [C]. InThe10th European Conference on Computer Vision. Berlin, May2008:316–329.
[297] Barnard K, Duygulu P, Freitas N, et al. Matching Words and Pictures [J]. Journalof Machine Learning Research.2003,3:1107–1135.
[298] Dempster A, Laird N, Rubin D. Maximum Likelihood from Incomplete Data viathe EM Algorithm [J]. Journal of the Royal Statistical Society. Series B.1977,39(1):1–38.
[2] Duda R, Hart P, Stock D. Pattern Classification [M]. Wiley Interscience,2001.
[3] Hotelling H. Analysis of a Complex of Statistical Variables into Principal Compo-nents [J]. Journal of Educational Psychology.1933,24:417–441.
[4] FisherRA.TheUseofMultipleMeasurementsinTaxonomicProblems[J].Annalsof Eugenics.1936,7:179–188.
[5] ComonP.Independentcomponentanalysis,Anewconcept?[J].SignalProcessing.1994,36(3):287–314.
[6] Shawe-Taylor J, Bartlett P, Williamson R, et al. Structural risk minimization overdata-dependent hierarchies [J]. IEEE Transactions on Information Theory.1998,44(5):1926–1940.
[7] Scholkopf B, Smola A, Muller K-R. Nonlinear Component Analysis as a KernelEigenvalue Problem [J]. Neural Computation.1998,10:1299–1319.
[8] Bach F R, Jordan M I. Kernel Independent Component Analysis [J]. Journal ofMachine Learning Research.2002,3:1–48.
[9] Mika S, R tsch G, Weston J, et al. Constructing descriptive and discriminativenonlinear features: Rayleigh coefficients in kernel feature spaces [J]. IEEE Trans-actions on Pattern Analysis and Machine Intelligence.2003,25(5):623–628.
[10] Seung H, Lee D D. The manifold ways of perception [J]. Science.2000,290(5500):2268–2269.
[11] Roweis S, Saul L. Nonlinear Dimensionality Reduction by Locally Linear Embed-ding [J]. Science.2000,29:2323–2326.
[12] Tenenbaum J, Silva V, Langford J. A Global Geometric Framework for NonlinearDimensionality Reduction [J]. Science.2000,290:2319–2323.
[13] BelkinM,NiyogiP.LaplacianEigenmapsandSpectralTechniquesforEmbeddingand Clustering [C]. In Advances in Neural Information Process Systems.2002:585–591.
[14] Roweis S, Saul L. Global coordination of local linear models [C]. In Advances inneural information.2002.
[15] Donoho D, Grimes C. Hessian Eigenmaps: New Locally Linear Embedding Tech-niques for High-dimensional Data [C]. In National Academy of Sciences.2003:5591–5596.
[16] Weinberger K Q, Sha F, Saul L K. Learning a kernel matrix for nonlinear dimen-sionalityreduction[C].InTwenty-firstinternationalconferenceonMachinelearn-ing. New York, New York, USA,2004:106–113.
[17] Zhang Z, Zha H. Principal Manifolds and Nonlinear Dimension Reduction viaLocalTangentSpaceAlignment [J].SIAM Journal ofScientificComputing.2005,26(1):313–338.
[18] Zhang T, Li X, Tao D, et al. Local coordinates alignment and its linearization [C].In Neural Information Processing System.2008:643–652.
[19] He X, Cai D, Yan S. Neighborhood preserving embedding [C]. In ICCV.2005.
[20] Kokiopoulou E, Saad Y. Orthogonal neighborhood preserving projections [J].IEEE transactions on pattern analysis and machine intelligence.2005,29(12):2143–56.
[21] He X F, Niyogi P. Locality Preserving Projections [C]. In Advances in NeuralInformation Process Systems.2003:1–8.
[22] Xu D, Yan S, Tao D, et al. Marginal Fisher Analysis and Its Variants for HumanGait Recognition and Content-Based Image Retrieval [J]. IEEE Transactions onImage Processing.2007,16(11):2811–2821.
[23] Zhang T, Tao D, Yang J. Discriminative Locality Alignment [C]. In The10th Eu-ropean Conference on Computer Vision.2008:725–738.
[24] Simard P, LeCun Y, Denker J. Efficient Pattern Recognition Using a New Trans-formation Distance [C]. In Advances in Neural Information Processing Systems.1992.
[25] Chang H, Yeung D, Xiong Y. Super-resolution through neighbor embedding [C].In International Conference on Computer Vision and Pattern Recognition.2004.
[26] ChanT,ZhangJ.AnImprovedSuper-ResolutionwithManifoldLearningandHis-togram Matching [C]. In International Conference on Pattern Biometric.2006.
[27] Elgammal A, Lee C. Separating style and content on a nonlinear manifold [C]. InInternational Conference on Computer Vision and Pattern Recognition.2004.
[28] Zhang Q, Souvenir R, Pless R. Segmentation Informed by Manifold Learning [C].In EMMCVPR.2005.
[29] Lee D D, Seung H S. Learning the Parts of Objects by Non-negative Matrix Fac-torization.[J]. Nature.1999,401(6755):788–91.
[30] Li S, Hou X, Zhang H, et al. Learning spatially localized, parts-based representa-tion [C]. In IEEE Conference on Computer Vision and Pattern Recognition.2001:207–212.
[31] Zafeiriou S, Tefas A, Buciu I, et al. Exploiting Discriminant Information in Non-negative Matrix Factorization with Application to Frontal Face Verification.[J].IEEE transactions on neural networks.2006,17(3):683–95.
[32] Cai D, He X, Wu X, et al. Non-negative Matrix Factorization on Manifold [C]. InIEEE International Conference on Data Mining. December2008:63–72.
[33] Lee J S, Lee D D, Choi S, et al. Application of Non-negative Matrix Factorizationto Dynamic Positron Emission Tomography [C]. In3rd International ConferenceonIndependentComponentAnalysisandBlindSignalSeparation.2001:629–632.
[34] Donoho D, Stodden V. When Does Non-Negative Matrix Factorization Give aCorrectDecompositionintoParts?[C].InAdvancesinNeuralInformationProcessSystems.2004:1141–1148.
[35] Hoyer P. Non-Negative Matrix Factorization with Sparseness Constraints [C]. InJournal of Machine Learning Research.2004:1457–1469.
[36] Lee D D, Seung H. Algorithms for Non-negative Matrix Factorization [C]. In Ad-vances in Neural Information Process Systems.2001:556–562.
[37] Févotte C, Bertin N, Durrieu J-L. Nonnegative matrix factorization with theItakura-Saito divergence: with application to music analysis.[J]. Neural computa-tion.2009,21(3):793–830.
[38] Cichocki A, Lee H, Kim Y-d, et al. Nonnegative matrix factorization with α-divergence [J]. Pattern Recognition Letters.2008,29(9):1433–1440.
[39] Cichocki A, Cruces S, Amari S-i. Generalized Alpha-Beta Divergences and TheirApplicationtoRobustNonnegativeMatrixFactorization[J].Entropy.2011,13(1):134–170.
[40] Ding C, Xiaofeng H, DSimon H. On the Equivalence of Nonnegative Matrix Fac-torizationandSpectralClustering[C].InSIAMDataMiningConference.Septem-ber2006:606–610.
[41] Ding C, Li T, Peng W. On the equivalence between Non-negative Matrix Factor-ization and Probabilistic Latent Semantic Indexing [J]. Computational Statistics&Data Analysis.2008,52(8):3913–3927.
[42] Vavasis S A. On the complexity of nonnegative matrix factorization [J]. SIAMJournal on Optimization.2009,20(3):1364–1377.
[43] Lin C-J. Projected Gradient Methods for Nonnegative Matrix Factorization [J].Neural computation.2007,19(10):2756–2779.
[44] Kim J, Park H. Toward Faster Nonnegative Matrix Factorization: A New Algo-rithm and Comparisons [J].2008Eighth IEEE International Conference on DataMining.2008:353–362.
[45] Moussaoui S, Brie D, Mohammad-Djafari A, et al. Separation of Non-NegativeMixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sam-pling [J]. IEEE Transactions on Signal Processing.2006,54(11):4133–4145.
[46] Cemgil A T. Bayesian inference for nonnegative matrix factorisation models [J].Computational intelligence and neuroscience.2009,2009:785152.
[47] Kanjani K, Texas A. Parallel Non Negative Matrix Factorization for DocumentClustering [J]. Technical report, Texas A&M University.2007(May).
[48] Bucak S S, Gunsel B. Incremental Subspace Learning via Non-negative MatrixFactorization [J]. Pattern Recognition.2009,42:788–797.
[49] Mairal J, Bach F, Project-team I W, et al. Online Learning for Matrix Factorizationand Sparse Coding [J]. Journal of Machine Learning Research.2010,11:19–60.
[50] Cai D, He X, Wang X. Locality Preserving Nonnegative Matrix Factorization [J].International Joint Conference on Artificial Intelligence.2008:1010–1015.
[51] Owen A B, Perry P O. Bi-cross-validation of the SVD and the non-negative matrixfactorization [J]. Annals of Applied Statistics.2009,3(2):564–594.
[52] Kanagal B, Sindhwani V. Rank Selection in Low-rank Matrix Approximations:A Study of Cross-Validation for NMFs [C]. In Advances in Neural InformationSystems.2010.
[53] Yang J, Yan S, Fu Y, et al. Non-negative Graph Embedding [C]. In IEEE Confer-ence on Computer Vision and Pattern Recognition. June2008:1–8.
[54] Cai D, He X, Member S, et al. Graph Regularized Non-negative Matrix Factor-ization for Data Representation [J]. IEEE Transactions on Pattern Analysis andMachine Intelligence.2011,33(8):1548–1560.
[55] Yan S, Xu D, Zhang B, et al. Graph Embedding and Extensions: A General Frame-workforDimensionalityReduction[J].IEEETransactionsonPatternAnalysisandMachine Intelligence.2007,29(1):40–51.
[56] CaiD,HeX,HanJ.SpectralRegressionforEfficientRegularizedSubspaceLearn-ing [C]. In11th IEEE International Conference on Computer Vision.2007.
[57] Cai D, He X, Han J. SRDA: An Efficient Algorithm for Large-Scale Discrimi-nant Analysis [J]. IEEE Transactions on Knowledge and Data Engineering.2008,20(1):1–8.
[58] Guillamet D, Vitria J, Schiele B. Introducing a weighted non-negative matrix fac-torization for image classification [J]. Pattern Recognition Letters.2003,24(14):2447–2454.
[59] Ho N, Vandooren P. Non-negative matrix factorization with fixed row and columnsums [J]. Linear Algebra and its Applications.2008,429(5-6):1020–1025.
[60] Cybenko G, Crespi V. Learning Hidden Markov Models using Non-Negative Ma-trix Factorization [J]. IEEE Transactions on Information Theory.2011,57(6):3963–3970.
[61] Finesso L, Grassi A, Spreij P. Two-step Nonnegative Matrix Factorization Algo-rithm for the Approximate Realization of Hidden Markov Models [C]. In19thInternational Symposium on Mathematical Theory of Networks and Systems. Bu-dapest, Hungary,2010:369–374.
[62] Lakshminarayanan B, Raich R. Non-negative Matrix Factorization for ParameterEstimation in Hidden Markov Models [C]. In IEEE International Workshop onMachine Learning for Signal Processing. Kittila,2010:89–94.
[63] Yang Z, Zhang H, Yuan Z, et al. Kullback-Leibler Divergence for NonnegativeMatrix Factorization [J]. International Conference on Artificial Neural Networks.2011:250–257.
[64] Cichocki A, Zdunek R, Phan A, et al. Nonnegative Matrix and Tensor Factor-izations: Applications to Exploratory Multi-way Data Analysis and Blind SourceSeparation [M]. Chichester, UK: John Wiley,2009.
[65] Kompass R. A generalized divergence measure for nonnegative matrix factoriza-tion [J]. Neural computation.2007,19(3):780–791.
[66] CichockiA,ZdunekR,AmariSI.Csiszar’sDivergencesforNon-NegativeMatrixFactorization: Family of New Algorithms [J]. Independent Component Analysisand Blind Signal Separation.2006,3889:32–39.
[67] Dhillon I S, Sra S. Generalized Nonnegative Matrix Approximations with Breg-man Divergences [C]. In Advances in Neural Information Process Systems.2005.
[68] Guillamet D, Vitria J. Evaluation of distance metrics for recognition based on non-negative matrix factorization [J]. Pattern Recognition Letters.2003,24(9-10):1599–1605.
[69] Sandler R, Lindenbaum M. Nonnegative Matrix Factorization with Earth Mover’sDistance Metric [C]. In IEEE Conference on Computer Vision and Pattern Recog-nition. January2009:1873–1880.
[70] OlshausenB,FieldD.Sparse CodingwithAnOvercompleteBasisSet:AStrategyEmployed by V1?[J]. Vision Research.1997,37(23):3311–3325.
[71] Hoyer P. Non-negative Sparse Coding [C]. In2th IEEE Workshop on Neural Net-works for Signal Processing.2002:557–565.
[72] Stadlthanner K, Theis F J, Lang E W, et al. Sparse Nonnegative Matrix Factoriza-tion Applied to Microarray Data Sets [J]. Computational Intelligence and Bioin-spired Systems.2005,3512:249–256.
[73] Kim H, Park H. Nonnegative Matrix Factorization Based on Alternating Nonneg-ativity Constrained Least Squares and Active Set Method [J]. SIAM Journal onMatrix Analysis and Applications.2008,30(2):713–730.
[74] Kim J, Park H. Sparse Nonnegative Matrix Factorization for Clustering [J]. CSETechnical Reports, Georgia Institute of Technology.2008.
[75] Kim D, Sra S, Dhillon I S. Fast Newton-type Methods for the Least Squares Non-negative Matrix Approximation Problem [C]. In IEEE International Conferenceon Data Mining.2007:343–354.
[76] Kim H, Park H. Sparse non-negative matrix factorizations via alternating non-negativity-constrainedleastsquaresformicroarraydataanalysis.[J].Bioinformat-ics.2007,23(12):1495–502.
[77] Peharz R, Stark M, Pernkopf F. Sparse nonnegative matrix factorization using l0-constraints [C]. In IEEE International Workshop on Machine Learning for SignalProcessing (MLSP). Kittila,2010:83–88.
[78] Heiler M, Schnorr C. Learning Sparse Representations by Non-Negative MatrixFactorization and Sequential Cone Programming [J]. Journal of Machine LearningResearch.2006,7:1385–1407.
[79] Liu H, Palatucci M, Zhang J. Blockwise Coordinate Descent Procedures for theMulti-task Lasso, with Applications to Neural Semantic Basis Discovery [C]. In26th Annual International Conference on Machine Learning. New York, New Y-ork, USA,2009.
[80] TandonR,SraS.Sparsenonnegativematrixapproximation:newformulationsandalgorithms [J]. Max Planck Institute for Biological Cybernetics, Technical Report.2010.
[81] GillisN,GlineurF.Usingunderapproximationsforsparsenonnegativematrixfac-torization [J]. Pattern Recognition.2010,43(4):1676–1687.
[82] Kim W, Chen B, Kim J, et al. Sparse nonnegative matrix factorization for proteinsequence motif discovery [J]. Expert Systems with Applications.2011,38(10):13198–13207.
[83] Weyrauch B V B, Huang J, Heisele B. Component-based Face Recognition with3DMorphableModels[C].InIEEEWorkshoponFaceProcessinginVideo.Wash-ington, D.C.,2004.
[84] SamariaF,HarterA.ParameterisationofaStochasticModelforHumanFaceIden-tification [C]. In IEEE Workshop on Applications of Computer Vision. SarasotaFL,1994:138–142.
[85] Feng T, Li S Z, Shum H Y, et al. Local non-negative matrix factorization as avisual representation [C]. In2nd International Conference on Development andLearning. ICDL2002.2002:178–183.
[86] GaoY,ChurchG.Improvingmolecularcancerclassdiscoverythroughsparsenon-negative matrix factorization.[J]. Bioinformatics.2005,21(21):3970–5.
[87] Pauca V P, Shahnaz F, Berry M W, et al. Text Mining using Non-Negative MatrixFactorization [C]. In4th SIAM International Conference on Data Mining.2004:452–456.
[88] Schmidt M N, Laurberg H. Nonnegative matrix factorization with Gaussian pro-cess priors [J]. Computational Intelligence and Neuroscience.2008,2008.
[89] Liao S, Lei Z, Li S Z. Nonnegative Matrix Factorization with Gibbs Random Fieldmodeling[J].IEEE12thInternationalConferenceonComputerVisionWorkshop-s.2009:79–86.
[90] Ding C, Li T, Peng W, et al. Orthogonal Nonnegative Matrix Tri-FactorizationsforClustering[C].In12thACMSIGKDDinternationalconferenceonKnowledgediscovery and data mining. New York, New York, USA,2006:126–135.
[91] Li Z, Wu X, Peng H. Nonnegative Matrix Factorization on Orthogonal Sub-space [J]. Pattern Recognition Letters.2010,31(9):905–911.
[92] Choi S. Algorithms for orthogonal nonnegative matrix factorization [C]. In IEEEInternational Joint Conference on Neural Networks. June2008:1828–1832.
[93] Yoo J, Choi S. Non-negative Matrix Factorization with Orthogonality Con-straints [J]. Journal of Computing Science and Engineering.2010,4(2):97–109.
[94] Mirzal A. Converged Algorithms for Orthogonal Nonnegative Matrix Factoriza-tions [J]. arXiv:1010.5290v2.2011.
[95] Lin C-J. On the Convergence of Multiplicative Update Algorithms for Nonnega-tiveMatrixFactorization[J].IEEETransactionsonNeuralNetworks.2007,18(6):1589–1596.
[96] Liu C, He K, Zhou J, et al. Generalized Discriminant Orthogonal Non-negative Matrix [J]. Journal Of Computational Information Systems.2010,6(6):1743–1750.
[97] Zafeiriou S, Tefas A, Pitas I. Discriminant NMF-faces for Frontal Face Verifica-tion [C]. In IEEE Workshop on Machine Learning for Signal Processing.2005:355–359.
[98] Wang Y, Turk M. Fisher non-negative matrix factorization for learning local fea-tures [C]. In Asian Conference on Computer Vision.2004:27–30.
[99] Xue Y, Tong C S, Chen W S. A Modified Non-negative Matrix Factorization Al-gorithmforFaceRecognition[J].18thInternationalConferenceonPatternRecog-nition.2006:495–498.
[100] ZafeiriouS,TefasA,BuciuI,etal.Class-SpecificDiscriminantNon-negativeMa-trix Factorization for Frontal Face Verification [C]. In Lecture Notes in ComputerScience.2005:206–215.
[101] Belkin M. Problems of Learning on Manifolds [D].[S. l.]: University of Chicago,Chicago, IL,2003.
[102] Belkin M, Niyogi P, Sindhwani V. Manifold Regularization: A Geometric Frame-work for Learning from Labeled and Unlabeled Examples [J]. Journal of MachineLearning Research.2006,7:2399–2434.
[103] Bian W, Tao D. Manifold Regularization for SIR with Rate Root-n Conver-gence [C]. In Advances in Neural Information Process Systems.2009:1–8.
[104] An S, Yoo J, Choi S. Manifold-respecting discriminant nonnegative matrix factor-ization [J]. Pattern Recognition Letters.2011,32(6):832–837.
[105] Gu Q, Zhou J. Local Learning Regularized Nonnegative Matrix Factorization [C].In21stInternationalJointConferenceonArtificialIntelligence.2009:1046–1051.
[106] Bottou L, Vapnik V. Local Learning Algorithms [J]. Neural Computation.1992,4(6):888–900.
[107] Wu M, Scholkopf B. A Local Learning Approach for Clustering [C]. In Advancesin Neural Information Processing Systems.2006:1529–1536.
[108] Shen B, Si L. Nonnegative Matrix Factorization Clustering on Multiple Manifold-s [C]. In24th AAAI Conference on Artificial Intelligence.2010:575–580.
[109] Ding C, Li T, Jordan M I. Convex and semi-nonnegative matrix factorization-s.[J].IEEEtransactionsonpatternanalysisandmachineintelligence.2010,32(1):45–55.
[110] Wang H, Nie F, Huang H, et al. Fast Nonnegative Matrix Tri-Factorization forLarge-Scale Data Co-Clustering [C]. In International Joint Conference on Artifi-cial Intelligence.2009:1553–1558.
[111] Guillamet D, Bressan M, Vitria J. A weighted non-negative matrix factorizationfor local representations [C]. In IEEE Computer Society Conference on ComputerVision and Pattern Recognition. CVPR2001.2001:942–947.
[112] Kim Y D, Choi S. Weighted nonnegative matrix factorization [J].2009IEEEInternational Conference on Acoustics, Speech and Signal Processing.2009:1541–1544.
[113] Zhang S, Wang W, Ford J, et al. Learning from Incomplete Ratings Using Non-negative Matrix Factorization [C]. In SIAM International Conference on DataMining.2006:548–552.
[114] GuQ,ZhouJ,DingC.CollaborativeFiltering:WeightedNonnegativeMatrixFac-torization Incorporating User and Item Graphs [C]. In SIAM International Confer-ence on Data Mining.2008:199–210.
[115] Yoo J, Choi S. Weighted Nonnegative Matrix Co-Tri-Factorization for Collabora-tive Prediction [J]. Lecture Notes in Computer Science.2009,5828:396–411.
[116] Lee H, Yoo J, Choi S. Semi-supervised nonnegative matrix factorization [J]. IEEESignal Processing Letters.2010,17(1):4–7.
[117] Laurberg H, Hansen L K. On Affine Non-negative Matrix Factorization [C]. InICASSP.2007:653–656.
[118] Yuan Z, Oja E. Projective Nonnegative Matrix Factorization for Image Compres-sionandFeatureExtraction[C].InInternationalSymposiumonIndependentCom-ponent Analysis and Blind Signal Separation.2004:1–8.
[119] Yang Z, Oja E. Linear and Nonlinear Projective Nonnegative Matrix Factoriza-tion [J]. IEEE transactions on neural networks.2010,21(5):734–49.
[120] Pan B, Lai J, Chen W S. Nonlinear nonnegative matrix factorization based onMercerkernelconstruction[J].PatternRecognition.2011,44(10-11):2800–2810.
[121] Zhang D, Zhou Z H, Chen S. Non-negative Matrix Factorization on Kernels [J].Trends in Artificial Intelligence.2006,4099:404–412.
[122] Buciu I, Nikolaidis N, Pitas I. Nonnegative matrix factorization in polynomial fea-ture space [J]. IEEE Transactions on Neural Networks.2008,19(6):1090–100.
[123] Zafeiriou S, Petrou M. Nonlinear Non-negative Component Analysis Algorithm-s [J]. IEEE Transactions on Image Processing.2010,19(4):1050–1066.
[124] Zhang Z, Li T, Ding C, et al. Binary Matrix Factorization with Applications [C].In IEEE International Conference on Data Mining.2007.
[125] Zdunek R. Data Clustering with Semi-Binary Nonnegative Matrix Factoriza-tion [C]. In Artificial Intelligence and Soft Computing.2008.
[126] Lu H, Vaidya J, Atluri V, et al. Extended Boolean Matrix Decomposition [C]. InIEEE International Conference on Data Mining. December2009:317–326.
[127] KameokaH,OneN,KashinoK,etal.ComplexNMF:Anewsparserepresentationfor acoustic signals [C]. In IEEE International Conference on Acoustics, Speechand Signal Processing. April2009:3437–3440.
[128] Ozerov A, Fevotte C. Multichannel nonnegative matrix factorization in convolu-tive mixtures. With application to blind audio source separation [C]. In IEEE In-ternational Conference on Acoustics, Speech and Signal Processing. April2009:3137–3140.
[129] Ozerov A, Fevotte C. Multichannel Nonnegative Matrix Factorization in Convo-lutive Mixtures for Audio Source Separation [J]. IEEE Transactions on Audio,Speech, and Language Processing.2010,18(3):550–563.
[130] Wang W. Squared Euclidean Distance based Convolutive Non-negative MatrixFactorizationwithMultiplicativeLearningRulesforAudioPatternSeparation[C].In IEEE International Symposium on Signal Processing and Information Technol-ogy.2007:347–352.
[131] Lee H, Choi S. Group Nonnegative Matrix Factorization for EEG Classifica-tion [C]. In International Conference on Machine Learning.2009:320–327.
[132] Lee H, Cichocki A, Choi S. Kernel nonnegative matrix factorization for spectralEEG feature extraction [J]. Neurocomputing.2009,72(13-15):3182–3190.
[133] Gillis N, Plemmons R J. Sparse Nonnegative Matrix Underapproximation and it-s Application to Hyperspectral Image Analysis [C]. In Hyperspectral Image andSignal Processing: Evolution in Remote Sensing.2011.
[134] LaurbergH,ChristensenMG,PlumbleyMD,etal.TheoremsonPositiveData:Onthe Uniqueness of NMF.[J]. Computational intelligence and neuroscience.2008,2008(2):764206.
[135] Welling M, Chemudugunta C, Sutter N. Deterministic Latent Variable Models andtheir Pitfalls [C]. In SIAM International Conference on Data Mining.2008.
[136] Xu W, Liu X, Gong Y. Document clustering based on non-negative matrix fac-torization [C]. In The26th annual international ACM SIGIR conference on Re-search and development in informaion retrieval. New York, New York, USA,2003:267–273.
[137] Gaussier E, Goutte C. Relation between PLSA and NMF and Implications Cat-egories and Subject Descriptors [C]. In28th annual international ACM SpecialInterest Group on Information Retrieval.2005:601–602.
[138] Ding C, Li T, Peng W. Nonnegative Matrix Factorization and Probabilistic LatentSemantic Indexing: Equivalence, Chi-square Statistic, and a Hybrid Method [C].In21st National Conference on Artificial Intelligence.2006:342–347.
[139] LiT,DingC.TheRelationshipsAmongVariousNonnegativeMatrixFactorizationMethods for Clustering [J]. Sixth International Conference on Data Mining.2006:362–371.
[140] Peng W, Li T. On the equivalence between nonnegative tensor factorization andtensorial probabilistic latent semantic analysis [J]. Applied Intelligence.2010.
[141] Wild S, Curry J, Dougherty A. Improving non-negative matrix factorization-s through structured initialization [J]. Pattern Recognition.2004,37(11):2217–2232.
[142] Langville A N, Meyer C D, Albright R. Initializations for the Nonnegative Ma-trix Factorization [C]. In ACM SIGKDD International Conference on KnowledgeDiscovery and Data Mining.2006.
[143] Zheng Z, Yang J, Zhu Y. Initialization enhancer for non-negative matrix factoriza-tion[J].EngineeringApplicationsofArtificialIntelligence.2007,20(1):101–110.
[144] Xue Y, Tong C, Chen Y, et al. Clustering-based initialization for non-negativematrix factorization [J]. Applied Mathematics and Computation.2008,205(2):525–536.
[145] BoutsidisC,GallopoulosE.SVDbasedinitialization:Aheadstartfornonnegativematrix factorization [J]. Pattern Recognition.2008,41(4):1350–1362.
[146] JanecekA,TanY.UsingPopulationBasedAlgorithmsforInitializingNonnegativeMatrix Factorization [J]. Advances in Swarm Intelligence.2011,6729:307–316.
[147] Rezaei M, Boostani R. An Efficient Initialization Method for Nonnegative MatrixFactorization [J]. Journal of Applied Science.2011,11(2):354–359.
[148] Wild S. Seeding Non-Negative Matrix Factorizations with the Spherical K-MeansClustering by [D].[S. l.]: University of Colorado,2003.
[149] Gonzalez E F, Zhang Y. Accelerating the Lee-Seung Algorithm for NonnegativeMatrix Factorization [R].2005.
[150] Gillis N, Glineur F. Accelerated Multiplicative Updates and Hierarchical ALSAlgorithms for Nonnegative Matrix Factorization [J]. Neural computation.2011,23(9):2421–56.
[151] Bertin N, Badeau R, Vincent E. Fast bayesian nmf algorithms enforcing harmonic-ity and temporal continuity in polyphonic music transcription [J]. IEEE Workshopon Applications of Signal Processing to Audio and Acoustics.2009:29–32.
[152] Badeau R, Bertin N, Vincent E. Stability analysis of multiplicative update algo-rithms and application to nonnegative matrix factorization [J]. IEEE transactionson neural networks.2010,21(12):1869–81.
[153] Sra S. Block-Iterative Algorithms for Non-Negative Matrix Approximation [C].In IEEE International Conference on Data Mining.2008:1037–1042.
[154] Berry M, Browne M, Langville A, et al. Algorithms and applications for approxi-mate nonnegative matrix factorization [J]. Computational Statistics&Data Anal-ysis.2007,52(1):155–173.
[155] Cichocki A, Zdunek R, Amari S-i. Hierarchical ALS Algorithms for NonnegativeMatrixand3DTensorFactorization[J].LectureNotesinComputerScience.2007,4666:169–176.
[156] Cichocki A, Phan A H, Caiafa C. Flexible HALS algorithms for sparse non-negative matrix/tensor factorization [C]. In IEEE Workshop on Machine Learningfor Signal Processing. October2008:73–78.
[157] Ho N D, Dooren P V, Blondel V D. Descent Methods for Nonnegative Matrix Fac-torization [J]. Numerical Linear Algebra in Signals, Systems and Control.2011,80:251–293.
[158] Liu H, Zhou Y. Rank-two residue iteration method for nonnegative matrix factor-ization [J]. Neurocomputing.2011.
[159] Biggs M, Ghodsi A, Vavasis S. Nonnegative matrix factorization via rank-onedowndate [J].25th international conference on Machine learning.2008:64–71.
[160] Bonettini S. Inexact Block Coordinate Descent Methods with Application to theNonnegative Matrix Factorization [J]. IMA Journal of Numerical Analysis.2011.
[161] Grippo L, Sciandrone M. On the Convergence of the Block Nonlinear Gauss-Seidel Method Under Convex Constraints [J]. Operations Research Letters.2000,26(3):127–136.
[162] Merrit M, Zhang Y. Interior-Point Gradient Method for Large-Scale Totally Non-negative Least Squares Problems [J]. Journal of Optimization Theory and Appli-cations.2005,126(1):191–202.
[163] Han L, Neumann M, Prasad U. Alternating Projected Barzilai-Borwein Methodsfor Nonnegative Matrix Factorization [J]. Electronic Transactions on NumericalAnalysis.2009,36(06):54–82.
[164] Zdunek R, Cichocki A. Nonnegative matrix factorization with Quasi-Newton op-timization [C]. In Lecture Notes in Artificial Intelligence. June2006:2309–2320.
[165] Zdunek R, Cichocki A. Non-negative Matrix Factorization with Quasi-NewtonOptimization [C]. In The8th International Conference on Artificial Intelligenceand Soft Computing.2006:870–879.
[166] Hsieh C J, Dhillon I S. Fast Coordinate Descent Methods with Variable Selectionfor Non-Negative Matrix Factorization [R].2011.
[167] Schmidt M N, Winther O, Hansen L K. Bayesian Non-negative Matrix Factoriza-tion [J]. Lecture Notes in Artificial Intelligence.2009,5441:540–547.
[168] Wang F, Li P. Efficient Nonnegative Matrix Factorization with Random Projec-tions [C]. In The10th SIAM International Conference on Data Mining.2009:281–292.
[169] Cichocki A, Amari S I, Hori G, et al. Extended SMART Algorithms for Non-negative Matrix Factorization [J]. Lecture Notes in Artificial Intelligence.2006,4029:548–562.
[170] Cichocki A, Zdunek R. Multilayer Nonnegative Matrix Factorization using Pro-jected Gradient Approaches [J]. International Journal Neural Systems.2007,17(6):431–446.
[171] Gillis N, Glineur F. A Multilevel Approach for Nonnegative Matrix Factoriza-tion [J]. CORE Discussion Paper.2010.
[172] Dong C, Zhao H, Wang W. Parallel Nonnegative Matrix Factorization AlgorithmontheDistributedMemoryPlatform[J].InternationalJournalofParallelProgram-ming.2009,38(2):117–137.
[173] Cao B, Shen D, Sun J-t, et al. Detect and Track Latent Factors with Online Non-negative Matrix Factorization [C]. In The20th International Joint Conference onArtificial Intelligence. San Francisco,2007:2689–2694.
[174] Wang F, Tan C, Knoig A C, et al. Efficient Document Clustering via Online Non-negativeMatrixFactorization[C].InIEEEInternationalConferenceonDataMin-ing.2010.
[175] Robila S, Maciak L. Considerations on Parallelizing Nonnegative Matrix Factor-ization for Hyperspectral Data Unmixing [J]. IEEE Geoscience and Remote Sens-ing Letters.2009,6(1):57–61.
[176] LiuC,YangH-c,FanJ,etal.Distributednonnegativematrixfactorizationforweb-scale dyadic data analysis on mapreduce [C]. In19th international conference onWorld wide web. New York, New York, USA,2010.
[177] Sun Z, Li T, Rishe N. Large-Scale Matrix Factorization Using MapReduce [J].IEEE International Conference on Data Mining Workshops.2010:1242–1248.
[178] Gautam B P, Shrestha D, Iaeng M. Document Clustering Through Non-NegativeMatrixFactorization:ACaseStudyofHadoopforComputationalTimeReductionof Large Scale Documents [C]. In26th annual international ACM SIGIR confer-ence on Research and development in informaion retrieval.2010:1–6.
[179] ShahnazF,BerryM,PaucaV,etal.Documentclusteringusingnonnegativematrixfactorization [J]. Information Processing&Management.2006,42(2):373–386.
[180] Park S, An D U, Char B, et al. Document Clustering with Cluster Refinement andNon-negative Matrix Factorization [J]. Lecture Notes in Computer Science.2009,5864:281–288.
[181] Hu C, Zhang B, Yan S, et al. Mining Ratio Rules Via Principal Sparse Non-Negative Matrix Factorization [J]. Fourth IEEE International Conference on DataMining.2004:407–410.
[182] Psorakis I, Roberts S, Ebden M, et al. Overlapping community detection usingBayesian non-negative matrix factorization.[J]. Physical review. E, Statistical,nonlinear, and soft matter physics.2011,83(6-2):066114.
[183] Newman M. Networks: An Introduction [M]. Oxford University Press,2010.
[184] Kersting K, Wahabzada M, Thurau C, et al. Convex NMF on Non-Convex MassivData [J]. Knowledge Discovery and Machine Learning Cone, Technical Report.2005.
[185] Li T, Ding C, Jordan M I. Solving Consensus and Semi-supervised ClusteringProblems Using Nonnegative Matrix Factorization [J]. Seventh IEEE Internation-al Conference on Data Mining.2007,2(1):577–582.
[186] Chen Y, Rege M, Dong M, et al. Non-negative matrix factorization for semi-supervised data clustering [J]. Knowledge and Information Systems.2008,17(3):355–379.
[187] Chen Y, Wang L, Dong M. Non-Negative Matrix Factorization for Semi-supervised Heterogeneous Data Co-clustering [J]. IEEE Transactions on Knowl-edge and Data Engineering.2010,22(10):1459–1474.
[188] Guillamet D, Schiele B, Vitria J. Analyzing non-negative matrix factorization forimage classification [J]. Object recognition supported by user interaction for ser-vice robots.2002,2:116–119.
[189] GuillametD,VitriaJ.Non-negativeMatrixFactorizationforFaceRecognition[J].Lecture Notes in Artificial Intelligence.2002,2504:336–344.
[190] Okun O G. Non-negative Matrix Factorization and Classifiers: Experimental S-tudy [C]. In Fourth IASTED International Conference Proceedings, Visualization,Imaging, and Image Processing.2004.
[191] Kim H, Park H. Discriminant analysis using nonnegative matrix factorization fornonparametric multiclass classification [C]. In IEEE International Conference onGranular Computing.2006:182–187.
[192] Guillamet D, Vitria J. Determining a suitable metric when using non-negative ma-trix factorization [C]. In16th International Conference on Pattern Recognition.2002:128–131.
[193] Kim J, Choi J, Yi J, et al. Effective representation using ICA for face recognitionrobust to local distortion and partial occlusion.[J]. IEEE transactions on patternanalysis and machine intelligence.2005,27(12):1977–81.
[194] Neo H F, Teoh B J, Ngo C L. A Novel Spatially Confined Non-Negative MatrixFactorization for Face Recognition [C]. In IAPR Conference on Machine VisionApplications.2005:502–505.
[195] ZhangD,ChenS,ZhouZ-h.Two-DimensionalNon-negativeMatrixFactorizationfor Face Representation and Recognition [J]. Lecture Notes in Computer Science.2005,3723:350–363.
[196] Buciu I, Nikolaidis N, Pitas I. A comparative study of NMF, DNMF, and LNM-F algorithms applied for face recognition [C]. In The European Association forSignal Processing.2006.
[197] Chen W S, Pan B, Fang B, et al. Incremental Nonnegative Matrix Factorizationfor Face Recognition [J]. Mathematical Problems in Engineering.2008,2008.
[198] Oh H, Lee K, Lee S. Occlusion invariant face recognition using selective localnon-negative matrix factorization basis images [J]. Image and Vision Computing.2008,26(11):1515–1523.
[199] Zhang T, Fang B, Tang Y Y, et al. Topology Preserving Non-negative Matrix Fac-torization for Face Recognition [J]. IEEE transactions on image processing.2008,17(4):574–84.
[200] Yin H, Liu H. Nonnegative matrix factorization with bounded total variationalregularization for face recognition [J]. Pattern Recognition Letters.2010,31(16):2468–2473.
[201] Buciu I, Pitas I. Application of non-negative and local non negative matrix fac-torization to facial expression recognition [J]. International Conference on PatternRecognition,.2004,1:288–291.
[202] Zhi R, Flierl M, Ruan Q, et al. Graph-preserving sparse nonnegative matrix fac-torization with application to facial expression recognition [J]. IEEE transactionson systems, man, and cybernetics. Part B, Cybernetics.2011,41(1):38–52.
[203] Teoh A, Neo H, Ngo D. Sorted locally confined non-negative matrix factorizationin face verification [C]. In International Conference on Communications, Circuitsand Systems.2005.
[204] Yuan L, Mu Z C, Zhang Y. Ear Recognition using Improved Non-Negative MatrixFactorization [C]. In18th International Conference on Pattern Recognition.2006:501–504.
[205] Soukup D, Bajla I. Robust object recognition under partial occlusions using NM-F [J]. Computational intelligence and neuroscience.2008,2008.
[206] Barman P C, Lee S-Y. Document Classification with Unsupervised NonnegativeMatrixFactorizationandSupervisedPercetronLearning[C].In2007InternationalConference on Information Acquisition. July2007:182–186.
[207] Chikhi N F, Rothenburger B, Aussenac-gilles N, et al. Authoritative DocumentsIdentification Based On nonnegative Matrix Factorization [C]. In IEEE Interna-tional Conference on Information Reuse and Integration.2008:262–267.
[208] Berry M W,Gillis N, Glineur F.Document classification using nonnegative matrixfactorization and underapproximation [C]. In IEEE International Symposium onCircuits and Systems. May2009:2782–2785.
[209] BenAbdallah J, Caicedo J C, Gonzalez F a, et al. Multimodal Image AnnotationUsing Non-negative Matrix Factorization [C]. In IEEE/WIC/ACM InternationalConference on Web Intelligence and Intelligent Agent Technology. August2010:128–135.
[210] Okun O, Priisalu H. Nonnegative matrix factorization for pattern recognition [C].In5th International Conference on Visualization, Imaging, and Image Processing.Benidorm, Spain,2005:546–551.
[211] Okun O, Priisalu H. Fast Nonnegative Matrix Factorization and Its Application forProteinFoldRecognition[J].EURASIPJournalonAdvancesinSignalProcessing.2006,2006:1–9.
[212] Li Y, Cichocki A. Non-negative matrix factorization and its application in blindsparse source separation with less sensors than sources [J]. The International Jour-nal for Computation and Mathematics in Electrical and Electronic Engineering.2005,24(2):695–706.
[213] Cichocki A, Zdunek R, Amari S. New Algorithms for Non-Negative Matrix Fac-torization in Applications to Blind Source Separation [C]. In IEEE Internation-al Conference on Acoustics Speed and Signal Processing Proceedings.2006:621–624.
[214] ZhangJ,WeiL,FengX,etal.Patternexpressionnonnegativematrixfactorization:algorithm and applications to blind source separation [J]. Computational intelli-gence and neuroscience.2008,2008:168–177.
[215] Mouri M, Funase A, Cichocki A, et al. Applying Non-negative Matrix Factor-ization for Global Signal Elimination from Electromagnetic Signals [C]. In15thEuropean Signal Processing Conference. Poznan, Poland,2007:2444–2448.
[216] Smaragdis P, Brown J C. Non-Negative Matrix Factorization for Polyphonic Mu-sic Transcription [C]. In IEEE Workshop on Application of Signal Processing toAudio and Acoustics.2003:177–180.
[217] Benetos E, Kotti M, Kotropoulos C. Applying Supervised Classifiers Based onNon-negative Matrix Factorization to Musical Instrument Classification [J]. IEEEInternational Conference on Multimedia and Expo.2006:2105–2108.
[218] Bertin N, Badeau R, Vincent E. Enforcing Harmonicity and Smoothness inBayesian Non-Negative Matrix Factorization Applied to Polyphonic Music Tran-scription [J]. IEEE Transactions on Audio, Speech, and Language Processing.2010,18(3):538–549.
[219] Cheng C C, Hu D J, Saul L K. Nonnegative Matrix Factorizatino for Real TimeMusical Analysis and Sight-Reading Evaluation [C]. In IEEE International Con-ference on Acoustics, Speech and Signal Processing.2008:2017–2020.
[220] Holzapfel A, Stylianou Y. Musical Genre Classification Using Nonnegative Ma-trix Factorization-Based Features [J]. IEEE Transactions on Audio, Speech, andLanguage Processing.2008,16(2):424–434.
[221] Costantini G, Todisco M, Saggio G, et al. Musical Onset Detection by meansof Non-Negative Matrix Factorization [C]. In Latest Trends on Systems.2010:206–209.
[222] Behnke S. Discovering hierarchical speech features using convolutional non-negativematrixfactorization[C].InInternationalJointConferenceonNeuralNet-works.2003:2758–2763.
[223] Sha F, Saul L K. Real-Time Pitch Determination of One or More Voices by Non-negativeMatrixFactorization[J].AdvancesinNeuralInformationProcessingSys-tems.2005,17.
[224] Kellis E, Galanis N, Natsis K, et al. Non-negative Matrix Factorization for NoteOnset Detection of Audio Signals [C]. In16th IEEE Signal Processing SocietyWorkshop on Machine Learning for Signal Processing. August2011:447–452.
[225] Virtanen T. Monaural Sound Source Separation by Nonnegative Matrix Factoriza-tion With Temporal Continuity and Sparseness Criteria [J]. IEEE Transactions onAudio, Speech and Language Processing.2007,15(3):1066–1074.
[226] Stouten V, Demuynck K, Van hamme H. Discovering Phone Patterns in SpokenUtterancesbyNon-NegativeMatrixFactorization[J].IEEESignalProcessingLet-ters.2008,15:131–134.
[227] Ogrady P, Pearlmutter B. Discovering speech phones using convolutive non-negative matrix factorisation with a sparseness constraint [J]. Neurocomputing.2008,72(1-3):88–101.
[228] Schuller B, Weninger F. Discrimination of Speech and Non-linguistic Vocaliza-tions by Non-negative Matrix Factorization [C]. In IEEE International Conferenceon Acoustics Speech and Signal Processing. Dallas, TX,2010:5054–5057.
[229] Weninger F, Schuller B, Batliner A, et al. Recognition of Nonprototypical Emo-tions in Reverberated and Noisy Speech by Nonnegative Matrix Factorization [J].EURASIP Journal on Advances in Signal Processing.2011,2011:1–16.
[230] Miao L, Qi H. Endmember Extraction From Highly Mixed Data Using MinimumVolume Constrained Nonnegative Matrix Factorization [J]. IEEE Transactions onGeoscience and Remote Sensing.2007,45(3):765–777.
[231] Jia S, Qian Y. L1/2Constrained Nonnegative Matrix Factorization for Hyperspec-tral Unmixing [J]. IEEE Transactions on Geoscience and Remote Sensing.2009,47(1):161–173.
[232] Sajda P, Du S, Brown T R, et al. Nonnegative matrix factorization for rapid re-covery of constituent spectra in magnetic resonance chemical shift imaging of thebrain.[J]. IEEE transactions on medical imaging.2004,23(12):1453–65.
[233] HamzaA,BradyD.Reconstructionofreflectancespectrausingrobustnonnegativematrix factorization [J]. IEEE Transactions on Signal Processing.2006,54(9):3637–3642.
[234] Pauca V, Piper J, Plemmons R. Nonnegative matrix factorization for spectral dataanalysis [J]. Linear Algebra and its Applications.2006,416(1):29–47.
[235] Liu W, Yuan K, Ye D. Reducing microarray data via nonnegative matrix factor-ization for visualization and clustering analysis.[J]. Journal of biomedical infor-matics.2008,41(4):602–6.
[236] Wang R, Zhang S, Wang Y, et al. Clustering complex networks and biologicalnetworksbynonnegativematrixfactorizationwithvarioussimilaritymeasures[J].Neurocomputing.2008,72(1-3):134–141.
[237] Fogel P, Young S S, Hawkins D M, et al. Inferential, robust non-negative matrixfactorization analysis of microarray data.[J]. Bioinformatics.2007,23(1):44–9.
[238] Fujiwara T, Ishikawa S, Hoshida Y, et al. Non-Negative Matrix Factorization ofLung Adenocarcinoma Expression Profiles [C]. In Conference on Genome Infor-matics.2005:3–4.
[239] Chen Z, Cichochi A, Rutkowski T M. Constrained Non-negative Matrix Factor-izationMethodsforEEGAnalysisInEarlyDetectionofAlzheimerDisease[C].InIEEEInternationalConferenceonAcoustics,SpeechandSignalProcessing.2006.
[240] Kim S P, Rao Y N, Erdogmus D, et al. Determining Patterns in Neural Activityfor Reaching Movements Using Nonnegative Matrix Factorization [J]. EURASIPJournal on Advances in Signal Processing.2005,2005(19):3113–3121.
[241] Lee J S, Lee D D, Choi S, et al. Non-negative matrix factorization of dynamicimages in nuclear medicine [C]. In IEEE Nuclear Science Symposium ConferenceRecord.2002:2027–2030.
[242] Lohmann G, Volz K G, Ullsperger M. Using non-negative matrix factorization forsingle-trial analysis of fMRI data.[J]. NeuroImage.2007,37(4):1148–60.
[243] Drakakis K, Rickard S, Frein R D, et al. Analysis of Financial Data using Non-Negative Matrix Factorization [C]. In International Mathematical Forum.2008:1853–1870.
[244] Ribeiro B, Silva C, Vieira A, et al. Extracting Discriminative Features Using Non-negative Matrix Factorization in Financial Distress Data [J]. Lecture Notes in Ar-tificial Intelligence.2009,5495:537–547.
[245] Xu B, Lu J, Huang G. A constrained non-negative matrix factorization in infor-mation retrieval [C]. In Fifth IEEE Workshop on Mobile Computing Systems andApplications.2003:273–277.
[246] TsugeS,ShishiboriM.DimensionalityReductionusingNon-negativeMatrixFac-torization for Information Retrieval [C]. In IEEE International Conference on Sys-tems, Man, and Cybernetics.2001:960–965.
[247] Berry M W, Browne M. Email Surveillance Using Non-negative Matrix Factor-ization [J]. Computational and Mathematical Organization Theory.2006,11(3):249–264.
[248] Szupiluk R, Wojewnik P, Zabkowski T. Ensemble methods with non-negative ma-trix factorization for non-payment prevention system [C]. In11th WSEAS Inter-national Conference on SYSTEMS. Agios Nikolaos, Crete Island, Greece,2007:385–388.
[249] Guan X, Wang W, Zhang X. Fast intrusion detection based on a non-negative ma-trixfactorizationmodel[J].JournalofNetworkandComputerApplications.2009,32(1):31–44.
[250] Wachsmuth E, Oram M, Perrett D. Recognition of Objects and Their ComponentParts: Responses of Single Units in the Temporal Cortex of the Macaque [J]. Cere-bral Cortex.1994,4:509–522.
[251] LogothetisN,SheinbergD.VisualObjectRecognition[J].AnnualReviewofNeu-roscience.1996,19:577–621.
[252] Palmer S. Hierarchical Structure in Perceptual Representation [J]. Cognitive Psy-chology.1977,9:441–474.
[253] Zhang T, Tao D, Li X, et al. Patch Alignment for Dimensionality Reduction [J].IEEETransactionsonKnowledgeandDataEngineering.2009,21(9):1299–1313.
[254] Bertsekas D. Nonlinear Programming [M].2nd ed. Belmont: MA: Athena Scien-tific,1999.
[255] Saul L K, Roweis S T. Think Globally, Fit Locally: Unsupervised Learning ofLow Dimensional Manifolds [J]. Journal of Machine Learning Research.2003,4:119–155.
[256] Graham D, Allinson N. Characterising Virtual Eigensignatures for General Pur-pose Face Recognition [M]. Berlin: Springer,1998.
[257] LeCun Y, Bottou L, Yoshua B, et al. Gradient-Based Learning Applied to Docu-ment Recognition [J]. IEEE Conferences on Intelligent Signal Processing.1998,86(11):2278–2324.
[258] Alpaydin E. Combined5×2cv F Test for Comparing Supervised ClassificationLearning Algorithms [J]. Neural Computation.1999,11:1885–1892.
[259] Fukunaga K. Introduction to Statistical Pattern Recognition [M]. New York, USA:Academic Press,1990.
[260] Guan N, Tao D, Luo Z, et al. Non-negative Patch Alignment Framework [J]. IEEETransactions on Neural Networks.2011,22(8):1218–1230.
[261] Berman A, Plemmons R J. Nonnegative Matrices in the Mathematical Sci-ences [M]. New York, USA: Academic Press,1994.
[262] Alefeld G, Schneider N. On Square Roots of M-Matrices [J]. Linear Algebra andIts Applications.1982,42(4):119–132.
[263] Guan N, Tao D, Luo Z, et al. Manifold Regularized Discriminative Non-negativeMatrix Factorization with Fast Gradient Descent [J]. IEEE Transactions on ImageProcessing.2011,20(7):2030–2048.
[264] DietterichT.ApproximateStatisticalTestsforComparingSupervisedClassifiationLearning Algorithms [J]. Neural Computation.1998,10:1895–1923.
[265] Liang Z, Li Y, Zhao T. Projected Gradient Method for Kernel Discriminant Non-negative Matrix Factorization and the Applications [J]. Signal Processing.2010,90(7):2150–2163.
[266] Kim H, Park H. Cancer Class Discovery Using Non-negative Matrix Factoriza-tion Based on Alternating Non-negativity-Constrained Least Squares [J]. LectureNotes in Biology Information.2007,4463:477–487.
[267] Horn R, Johnson C. Matrix Analysis [M]. Cambridge University Press,1985.
[268] Woodbury M A. Inverting Modified Matrices [M]. N. J.: Princeton UniversityPress,1950.
[269] Bertsekas D P. On the Goldstein-Levitin-Polyak Gradient Projection Method [J].IEEE Transactions on Automatic Control.1976,21(2):174–184.
[270] Belhumeour P, Hespanha J, Kriegman D. Eigenfaces vs. Fisherfaces: RecognitionUsing Class Sepcific Linear Projection [J]. IEEE Transactions on Pattern Analysisand Machine Intelligence.1997,19(7):711–720.
[271] Nesterov Y E. A Method of Solving A Convex Programming Problem with Con-vergence Rate O(1/k2)[J]. Soviet Mathematics Doklady.1983,27(2):372–376.
[272] Beck A, Teboulle M. A Fast Iterative Shrinkage-Thresholding Algorithm for Lin-ear Inverse Problems [J]. SIAM Journal of Imaging Sciences.2009,2(1):183.
[273] Ji S, Ye J. An Accelerated Gradient Method for Trace Norm Minimization [C].In The26th Annual International Conference on Machine Learning. New York,USA,2009:457–464.
[274] Zhou T, Tao D. Fast Gradient Clustering [C]. In NIPS2009Workshop on DiscreteOptimization in Machine Learning: Submodularity, Sparsity&Polyhedra.2009.
[275] Nesterov Y. Introductory lectures on convex optimization: A Basic Course [M].Kluwer Academic Publishers,2003.
[276] Duchi J. Adaptive Subgradient Methods for Online Learning and Stochastic Opti-mization [C]. In COLT2010.2010.
[277] Tseng P D O M O W. On Accelerated Proximal Gradient Methods for Convex-Concave Optimization [J]. SIAM Journal on Optimization.2008.
[278] NesterovY.Smooth minimizationof non-smooth functions [J].Mathematical Pro-gramming.2005,103(1):127–152.
[279] Lin X. Dual Averaging Methods for Regularized Stochastic Learning and OnlineOptimization [C]. In NIPS.2009.
[280] Rockafellar R. Convex Analysis [M]. Princetion University Press,1970.
[281] Lewis D, Yang Y, Rose T, et al. RCV1: A New Benchmark Collection for Tex-t Categorization Research [J]. Journal of Machine Learning Research.2004,5:361–397.
[282] Cieri C, Graff D, Liberman M, et al. The TDT-2Text and Speech Corpus [C]. InDARPA Broadcast News Workshop.1999.
[283] Nemirovski A, Juditsky A, Lan G, et al. Robust Stochastic Approximation Ap-proach to Stochastic Programming [J]. SIAM Journal on Optimization.2009,19(4):1574–1609.
[284] Cao S, Liu L. Non-negative Matrix Factorization and its Applications to GeneExpression Data Analysis [J]. Journal of Beijing Normal University (Natural Sci-ence).2007,43(1):30–33.
[285] Bottou L, Bousquet O. The Trade-offs of Large Scale Learning [C]. In Advancesin Neural Information Process Systems.2008:161–168.
[286] Strassen V. The Existence of Probability Measures with Given Marginals [J]. An-nals of Mathematical Statistics.1965,38:423–439.
[287] Vaart A W V D. Asymptotic Statistics [M]. Cambridge University Press,1998.
[288] Fisk D L. Quasi-martingales [J]. Transactions of the American Mathematical So-ciety.1965,120(3):369–389.
[289] DuchiJ,ChandraT,ComTG.EfficientProjectionsontothel1-BallforLearninginHighDimensions[C].InInternationalConferenceonMachineLearning.Helsinki,Finland,2008.
[290] TurkMA,PentlandAP.Facerecognitionusingeigenfaces[C].InIEEEComputerSociety Conference on Computer Vision and Pattern Recognition. Maui, Hawaii,1991:586–591.
[291] Sim T, Baker S, Bsat M. The CMU Pose, Illumination, and ExpressionDatabase [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence.2003,25(12):1615–1618.
[292] Lovasz L, Plummer M. Matching Theory [M]. North Holland: Elsevier,1986.
[293] Duygulu P, Freitas N, Barnard K, et al. Object Recognition as Machine Transla-tion: Learning a Lexicon for a Fxed Image Vocabulary [C]. In The7th EuropeanConference on Computer Vision.2002:97–112.
[294] Grubinger M, Clough P D, Henning M, et al. The IAPR Benchmark: a New Eval-uation Resource for Visual Information Systems [C]. In International Conferenceon Language Resources and Evaluation. Genoa, Italy,2006.
[295] Ahn J H, Kim S K, Oh J H, et al. Multiple nonnegative-matrix factorization ofdynamic pet images [C]. In International Conference on Computer Vision.2004.
[296] Makadia A, Pavlovic V, Kumar S. A New Baseline for Image Annotation [C]. InThe10th European Conference on Computer Vision. Berlin, May2008:316–329.
[297] Barnard K, Duygulu P, Freitas N, et al. Matching Words and Pictures [J]. Journalof Machine Learning Research.2003,3:1107–1135.
[298] Dempster A, Laird N, Rubin D. Maximum Likelihood from Incomplete Data viathe EM Algorithm [J]. Journal of the Royal Statistical Society. Series B.1977,39(1):1–38.