一种基于自适应学习率的推荐优化算法模型
|
摘要
在预测推荐系统中用户和项目构成的高维稀疏矩阵中的缺失值时,通常采用随机梯度下降算法对构造的隐因子(LF)模型进行求解,由于在求解过程中,学习速率始终保持不变,这使得在模型训练过程中模型的性能有所损失。因此,本文将构造一种带有自适应学习率的随机梯度下降算法的LF模型(ADA_LF)来处理推荐系统中的高维稀疏矩阵。采用大型工业数据集对模型进行实验测试,结果表明,采用ADA_SGD算法构建的LF模型在收敛速率、预测精度上都有明显提升,提高了模型的性能。
In the prediction of missing value of recommender system with high-dimensional sparse matrix formed by users and items,Stochastic Gradient Descent algorithm is usually adopted to solve the latent factor(LF) model.However,model performance loss in the process of model training is occurred as a result of constant learning rate in the solution process.Hence,this paper proposes a stochastic gradient descent algorithm model with adaptive learning rate(ADA_SGD) to dispose high-dimensional sparse H of recommender system.Experimental tests of the model on large industrial data sets show that LF model constructed by ADA_SGD algorithm has greatly improved on convergence rate and prediction accuracy.Therefore,the performance of the model is greatly improved.
In the prediction of missing value of recommender system with high-dimensional sparse matrix formed by users and items,Stochastic Gradient Descent algorithm is usually adopted to solve the latent factor(LF) model.However,model performance loss in the process of model training is occurred as a result of constant learning rate in the solution process.Hence,this paper proposes a stochastic gradient descent algorithm model with adaptive learning rate(ADA_SGD) to dispose high-dimensional sparse H of recommender system.Experimental tests of the model on large industrial data sets show that LF model constructed by ADA_SGD algorithm has greatly improved on convergence rate and prediction accuracy.Therefore,the performance of the model is greatly improved.
引文
[1] BAWDEN D,ROBINSON L.The dark side of information:overload,anxiety and other paradoxes and pathologies[J].Journal of Information Science,2009,35(2):180-191.
[2] BORCHERS A,HERLOCKER J,KONSTAN J A,et al.Ganging up on information overload[J].IEEE Computer,1998,31(4):106-108.
[3] ADOMAVICIUS G,TUZHILIN A.Toward the next generation of recommender systems:a survey of the state-of-the-art and possible extensions[J].IEEE Transactions on Knowledge and Data Engineering,2005,17(6):734-749.
[4] 许海玲,吴潇,李晓东,等.互联网推荐系统比较研究[J].软件学报,2009,2(20):350-362.
[5] KOREN Y,BELL R M,VOLINSKY C,et al.Matrix factorization techniques for recommender systems[J].IEEE Computer,2009,42(8):30-37.
[6] ZHANG S,WANG W,FORD J,et al.Learning from incomplete ratings using non-negative matrix factorization[C]// Srivastava J.Siam International Conference on Data Mining,April 20-22,2006,Bethesda,Md,Usa:DBLP,2006:549-553.
[7] CACHEDA F,CARNEIRO V,FERNANDEZ D,et al.Comparison of collaborative filtering algorithms:limitations of current techniques and proposals for scalable,high-performance recommender systems[J].ACM Transactions on The Web,2011,5(1):1-2.
[8] DUCHI J C,HAZAN E,SINGER Y,et al.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(1):2121-2159.
[9] ZEILER M D.ADADELTA:An adaptive learning rate method[EB/OL].(2012-12-22)[2018-08-29],https://arxiv.org/pdf/1212.5701.pdf.
[10] BECK A,TEBOULLE M.A fast iterative shrinkage-thresholding algorithm for linear inverse problems[J].Siam Journal on Imaging Sciences,2009,2(1):183-202.
[11] COMBETTES P L.Signal recovery by proximal forward-backward splitting[J].Multiscale Model Simul,2006,4(4):1168-1200.
[12] KOREN Y.Factorization meets the neighborhood:a multifaceted collaborative filtering model[C]// ACM SIGKDD.International Conference on Knowledge Discovery and Data Mining.Las Vegas,California:ACM,2008:426-434.
[13] PATEREK A.Improving regularized singular value decomposition for collaborative filtering[C]// ACM SIGKDD.Kdd Cup and Workshop 2007.San Jose,California:ACM,2007:39-42.
[14] SALAKHUTDINOV R,MNIH A.Probabilistic matrix factorization[C]//NIPS.International Conference on Neural Information Processing Systems.Hyatt Regency Vancouver,Vancouver:Curran Associates Inc.2007:1257-1264.
[2] BORCHERS A,HERLOCKER J,KONSTAN J A,et al.Ganging up on information overload[J].IEEE Computer,1998,31(4):106-108.
[3] ADOMAVICIUS G,TUZHILIN A.Toward the next generation of recommender systems:a survey of the state-of-the-art and possible extensions[J].IEEE Transactions on Knowledge and Data Engineering,2005,17(6):734-749.
[4] 许海玲,吴潇,李晓东,等.互联网推荐系统比较研究[J].软件学报,2009,2(20):350-362.
[5] KOREN Y,BELL R M,VOLINSKY C,et al.Matrix factorization techniques for recommender systems[J].IEEE Computer,2009,42(8):30-37.
[6] ZHANG S,WANG W,FORD J,et al.Learning from incomplete ratings using non-negative matrix factorization[C]// Srivastava J.Siam International Conference on Data Mining,April 20-22,2006,Bethesda,Md,Usa:DBLP,2006:549-553.
[7] CACHEDA F,CARNEIRO V,FERNANDEZ D,et al.Comparison of collaborative filtering algorithms:limitations of current techniques and proposals for scalable,high-performance recommender systems[J].ACM Transactions on The Web,2011,5(1):1-2.
[8] DUCHI J C,HAZAN E,SINGER Y,et al.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(1):2121-2159.
[9] ZEILER M D.ADADELTA:An adaptive learning rate method[EB/OL].(2012-12-22)[2018-08-29],https://arxiv.org/pdf/1212.5701.pdf.
[10] BECK A,TEBOULLE M.A fast iterative shrinkage-thresholding algorithm for linear inverse problems[J].Siam Journal on Imaging Sciences,2009,2(1):183-202.
[11] COMBETTES P L.Signal recovery by proximal forward-backward splitting[J].Multiscale Model Simul,2006,4(4):1168-1200.
[12] KOREN Y.Factorization meets the neighborhood:a multifaceted collaborative filtering model[C]// ACM SIGKDD.International Conference on Knowledge Discovery and Data Mining.Las Vegas,California:ACM,2008:426-434.
[13] PATEREK A.Improving regularized singular value decomposition for collaborative filtering[C]// ACM SIGKDD.Kdd Cup and Workshop 2007.San Jose,California:ACM,2007:39-42.
[14] SALAKHUTDINOV R,MNIH A.Probabilistic matrix factorization[C]//NIPS.International Conference on Neural Information Processing Systems.Hyatt Regency Vancouver,Vancouver:Curran Associates Inc.2007:1257-1264.