摘要
互联网技术的普及,促使电子商务被越来越多的用户所接受。推荐系统可以与用户进行交互,模拟商店销售人员帮助用户完成购买过程,并能根据用户的兴趣对用户进行个性化的推荐,对提升电子商务网站的用户满意度、增加网站的销售量有重要影响。近年来推荐系统在理论研究和实践应用方面都取得了很大的进展,在电子商务中的应用有着广阔的发展前景,引起了越来越多的企业和学者的关注。
推荐系统的好坏取决于系统所使用的推荐算法,目前主要的推荐算法有:协同过滤、基于内容、基于人口统计、基于知识和上述算法的混合推荐算法。其中协同过滤对推荐对象没有特殊要求,可以广泛地应用于音乐、电影和在线学习等各个领域,在数据密度达到一定程度时表现出非常好的推荐质量,因此在推荐系统领域取得了最大的成功,成为学者研究推荐算法的热点。
论文对传统的基于用户和基于项目的协同过滤算法进行了深入研究,发现传统的协同过滤推荐算法对用户偏爱度问题、用户兴趣度问题和用户信任度问题没有给出合理的解决方案,因此在面对商品数量庞大并且用户评分可信度不高的情况下,不能给出准确和高效的推荐。现有的对协同过滤的改进算法可以在一定程度上解决上述问题,但都存在一定的局限性。基于以上分析,论文提出了分类相似度、用户兴趣度和用户可信度三个概念,并为这三个概念找到了一个较好的结合点,改进了推荐流程,很好地解决了上述几个问题。改进后的算法首先根据分类相似度和用户兴趣度确定向用户进行推荐的类别,缩小预测评分的项目空间;在计算邻居用户时,综合考虑目标用户和邻居用户评分的相似度以及邻居用户在领域内的可信度,因此算法搜索到的邻居用户既与目标用户的兴趣比较相似又比较可信。
论文设计了三部分实验,除采用传统的平均绝对偏差(MAE)和平均推荐产生耗时(MCT)两个评估指标进行评价之外,论文还提出了一个新颖度指标(NOV)对推荐效果进行考查。经过一系列对比实验证明:改进后的算法能够明显地提高推荐的准确度。另外,由于本文算法采用了离线和在线结合的方式产生推荐,并且缩小了预测评分的项目空间,因而推荐效率也有较大提高。本文算法的另一个优点是在用户的平均可信度不高的情况下,依然能够保持非常低的平均绝对偏差,表明该算法有很强的实用性。
The spread of Internet technology makes e-commerce accepted by more and more people, with which people can purchase commodities without walking out the door. Recommendation system can simulate the behavior of shopkeeper to help customer finish the process of buying. What’s more, it can recommend personalized goods for customer according he’s interesting. Therefore, customer will be more satisfied with the buying process, and the sales of online shop will increase. Recently, recommendation system got great achievement both on theory research and application, and it is believed to get a better development in the future, getting more attention from scholars.
The effect of recommend system depends on the technology it adopted including content-based, collaborative filtering, population statistic data based, utility-based, knowledge-based technology and hybrid method. Among these technologies, collaborative filtering algorithm is regarded as the best one for its feature that caring little about attributes of objects, neither strict demand of objects. So it is widely adopted on music recommendation, movie recommendation and online-learning resource recommendation. Collaborative filtering algorithm performs quite well when the density of data is high.
This thesis made a lot research on User-based collaborative filtering algorithm and Item-based collaborative filtering algorithm and found that conventional collaborative filtering algorithm doesn’t give effective answer to User-favor problem, User-interesting problem and User-credibility problem. So it couldn’t perform exactly and efficiently when it meets great number of goods and low average credibility of customer. Current improved algorithms on collaborative filtering can solve some problems mentioned above to some extent, yet remaining some limitation at the same time. Based on the analysis above, this thesis proposes three conceptions: Class similarity, User interesting and User credibility, then combines three conceptions properly to improve the process of recommendation. This improvement can give a good answer to the problems. The proposed algorithm firstly determines some classes for customer according his interests and the class similarity, so the range of items to be predicted is limited. When it searches for neighbors for customer, it integrates similarity between target user and neighbor and the credibility of neighbor on current class together. Consequently, neighbors found in this way are similar and trustable to target user.
Experiments in this thesis contain three parts. Beside the conventional MAE and MCT which is widely used to evaluate the accuracy and efficiency of recommendation algorithm, this thesis also proposes NOV to evaluate the novelty of the algorithm. A serial of experiments proves that the proposed algorithm is more exact than other algorithms. Besides, it is more efficient than others because it combines online calculation and offline calculation together, so that the range of items to be predicted is limited. Another advantage of the proposed algorithm is that the algorithm can still perform well with relatively low MAE when average user credibility is not high while the accuracy of other algorithms becomes bad in this condition.
引文
[1]李勇,徐振宁. Inemet个性化信息服务研究综述[J].计算机工程与应用, 2002, (19):183-187.
[2] DENG Ai-Lin, ZHU Yang-Yong, SHI Bai-Le. A Collaborative Filtering Recommendation Algorithm Based on Item Rating Prediction[J]. Journal of Software, 2003, (9).
[3]蔡自兴,徐光裕,人工智能及应用[M].清华大学出版社, 2004.
[4]徐小琳,胭喜戎等,信息过滤技术和个性化信息服务[J].计算机工程与应用, 2003, (9):182-184.
[5] LAWRENCERD, ALMASIGS, KOTLYARV, etc. Personalization of supermarket product recommendations[R]. IBM Research Report, 2000.
[6] Sehafer J B, Konstan J A, and Riedl J. E-Commerce Recommendation Applications[R]. Data Mining and Knowledge Discovery, 2001.
[7] Herlocker, J. Understanding and Improving Automated Collaborative Filtering Systems[C]. Ph. D. Thesis, Computer Science Dept. , University of Minnesota, 2000:144-145.
[8]杨永健.基于模糊认知图和人工神经网络的个性化推荐算法研究[J]. Journal of Tianjin Vocational Institutes, 2009, (5).
[9]王征,谷安平,刘心松.基于在线客户情绪能量感知的商品推荐算法.吉林大学学报(信息科学版), 2009, (3).
[10]孟宪福,陈莉.基于贝叶斯理论的协同过滤推荐算法[J].计算机应用, 2009, (10).
[11]王晗,夏自谦.基于蚁群算法和浏览路径的推荐算法研究[J].中国科技信息, 2009, (7).
[12]翁小兰,庄永龙.基于项目特征聚类的协同过滤推荐算法[J].计算机应用与软件, 2009, (7).
[13]董祥和,齐莉丽,董荣和.优化的协作过滤推荐算法[J].计算机工程与应用, 2009, (8).
[14] Deerwester S, Dumais S T, Fumas G W,et al. Indexing by latent semantic analysis[J]. Journal of the American Society for Information Science, 1999, (3):102-106.
[15]李伟,王新房,刘妮.基于项目的非邻近序列模式推荐算法[J].计算机工程, 2009, (16).
[16] Bobadilla, J. Serradilla, F. ; Hernando, A. Collaborative filtering adapted to recommender systems of e-learning[C]. Journal of the American Society for Information Science, 2009:261-265.
[17] Zhang Wei. Relational distance-based collaborative filtering for E-Learning[C]. Proceedings of the 2008 International Symposium on Computational Intelligence and Design, 2008:354-357.
[18]刘志勇,刘磊,刘萍萍,杨帆,贾冰.一种基于语义网的个性化学习资源推荐算法[J].吉林大学学报(工学版), 2009, (2).
[19] P. Resnick, J. Varian. Recommendation systems[J]. Mineapplio: Communications of the ACM 1997, (3):56-58.
[20] SchaferJ, Konstan J. Ecommerce Recommendation Applications[R]. Data Mining and Knowledge Diseovery, 2001.
[21]张亚伟,苏一丹.基于移动Agent的分布式个性化推荐系统[J].微计算机信息, 2008(9).
[22]程岩,肖小云,吴洁倩.基于聚类分析的电子商务推荐系统[J].计算机工程与应用, 2005, (2).
[23] DAI Ya-e, GONG Song-jie. Collaborative Filtering Recommendation Based on Fuzzy Clustering in Personalization Services[J]. Computer Engineering and Science, 2009, (4).
[24] Chakraborty, Partha Sarathi. A scalable collaborative filtering based recommender system using incremental clustering[C]. 2009 IEEE International Advance Computing Conference, 2009:1526-1529.
[25] Hu Jian, Zhang Wei. Community collaborative filtering for E-Learning[C]. Proceedings of the 2008 International Conference on Computer and Electrical Engineering, 2008:593-597.
[26] Wu JingHui, Liu Qiang, Luo SiWen. Clustering technology application in e-commerce recommendation system[C]. Proceedings-International Conference on Management of e-Commerce and e-Government, 2008:200-203.
[27]陈冬林,聂规划,刘平峰.基于知识网格的电子商务推荐系统设计[J].计算机应用研究, 2006, (12).
[28]张锋,常会友,衣杨.基于规则的电子商务推荐系统模型和实现[J].计算机集成制造系统, 2004, (8).
[29] Sarwar, B. M. , Karypis, G. , Konstan, A. , and Riedl, J. Application of Dimensionality Reduction in Recommender System-A Case Study[C]. In ACM Web KDD 2000 Workshop, 2000.
[30]郑先荣.基于用户兴趣的协同过滤算法研究[D].中国科学技术大学.硕士学位论文, 2006:41-44.
[31]庄永龙.基于项目特征模型的协同过滤推荐算法[D].武汉理工大学.硕士学位论文, 2008:13-14.
[32] Breese, J. S. , Heckerman, D. , and Kadie, C. , Empirical Analysis of Predietive Algorithms for Collaborative Filtering[C], Proceedings of the 14th conference on Uncertainty in Artificial Intelligence, 1998:43-52.
[33] Sanwar, B. M. ,Karypis, G, Konstan, J. A. ,and Riedl, J. , Item-based Collaborativefiltering Recommendation Algorithms[E], Proeeedings of 10th International world Wide Web Conference, 2001:285-295.
[34]刘胜华,董利红.基于内容过滤的电子商务推荐算法分析[J].知识丛林, 2006, (6).
[35]曹毅,贺卫红.基于内容过滤的电子商务推荐系统研究[J].计算机技术与发展, 2009, (6).
[36]岑咏华,甘利人,丁晟春.基于内容的Web修改化推荐技术研究.图书情报工作,2003, (8).
[37]杨学兵,张俊.决策树算法及核心技术[J].计算机技术与发展, 2007, (1):43-45.
[38] Lee W, Stolfo S J, Mok K W. Data mining in workflow environments: experiences in intrusion detection [C] //ACM SIGKDD International Conference on Knowledge Discovery and Data Mining( KDD-99). USA: 1999:114-124.
[39]张卫星,王茜.基于协同过滤推荐技术的电子商务个性化推荐研究[D].重庆大学.硕士学位论文, 2008:23-24.
[40]林若萍.以从众化机制过滤超载信息之效果研究[D].台湾国立中正大学资讯管理研究所.硕士学位论文, 2002.
[41] DengCai, ZengxiangLu, YandaLi. Collaborative Filtering[J]. Joumal of Computer Science, 2002, (6):l-4.
[42]高凤荣,杜小勇,王珊.一种基于稀疏矩阵划分的个性化推荐算法[J].微电子学与计算机, 2004, (2).
[43]刘平峰,聂规划,陈冬林.电子商务推荐系统研究综述[J].情报杂志, 2007, (9).
[44] Pazzani,M. J. A framework for collaborative, content-based and demographic filtering[C]. Artificial Intelligence Review, 1999:393-408.
[45] GUO Yan-hong, DENG Gui-shi. Hybrid Recommendation Algorithm of Item Cold-start in Collaborative Filtering System[J]. Computer Engineering. 2008, (23).
[46]孙多.基于兴趣度的聚类协同过滤推荐系统的设计[J].安徽大学学报(自然科学版), 2007, (5).
[47]李聪,梁昌勇,董珂.基于项目类别相似性的协同过滤推荐算法[J].合肥工业大学学报(自然科学版), 2008(3).
[48] Zheng Xianrong and Cao Xianbin. Research on lineal gradual forgetting collaborative filtering algorithm[J]. Computer Engineering, 2007, (5):72-82.
[49] G Adomavicius, A Tuzhilin1 Toward the next generation of recommender systems : A survey of the state-of-the-art and possible extensions. IEEE Trans on Knowledge and Data Engineering , 2005, (6):734-749.
[50] ZHANG Fu-guo. Research on Trust based Collaborative Filtering Algorithm for User's Multiple Interests[J]. Journal of Chinese Computer Systems, 2008, (8).
[51]袁薇.搜索引擎系统中个性化机制的研究[J].微电子学与计算机, 2006, (2):68-72.
[52]卢竹兵,唐雁.一种基于信任网络的协同过滤推荐策略[J].西南师范大学学报(自然科学版), 2008(2).
[53]黄德双.神经网络模式识别系统理论[M].电子工业出版社. 1996:83-84.