基于单元的孤立点算法研究及客户忠诚度分析系统构建
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
数据挖掘技术是从上个世纪80年代开始发展起来的一门新技术,其主要的目的就是从大量的、不完全的、有噪声的、模糊的、随机的实际应用数据中,提取隐含在其中的、人们事先不知道的、但又是潜在有用的信息和知识。而孤立点分析是数据挖掘中的重要研究方面之一,其作用就是发现数据中的“小模式”,即数据集中显著不同于其它数据的对象。经过近20年的发展,数据挖掘技术在理论研究上日趋成熟,正不断的扩展其应用范围,当前数据挖掘已用于电信、金融、商业、气象预报、DNA、股票市场、入侵检测和客户分类等许多领域。因此,本文首先研究了基于单元的孤立点发现算法,指出了其存在的缺点,并予以改进;其次,采用该算法并结合其它数据挖掘技术,针对企业中的客户忠诚度分析问题构建了客户忠诚度分析系统,最后,根据海尔客户关系数据,分析了海尔集团的客户忠诚度
     第一,阐述了课题的研究背景及其重要的研究意义;从数据挖掘的理论研究和应用研究方面,对当前数据挖掘的国内与国外的研究动态进行分析;通过对知识发现一般过程的分析,给出了一个典型的数据挖掘系统的整体架构,分析了各模块的主要功能,并对其中采用的数据挖掘的技术作了详细阐述。
     第二,回顾了孤立点发现的研究过程及当前研究动态,介绍了基于距离、基于密度、基于偏离以及高维数据孤立点发现中的主要算法,具体分析了各个算法的主要内容,在此基础上总结比较了各个算法的优劣及其适用范围。
     第三,在基于单元的孤立点发现算法的基础上,提出了一种减少边缘影响的孤立点分析算法。针对算法中边界处孤立点的误判问题,给出了数据空间的单元格划分及数据对象分配方法,定义了数据集边界阈值动态调整函数,提出了基于单元的孤立点挖掘算法的改进算法,在不增加原有算法时间复杂度的前提下,极大地减少了边界处孤立点的误判。并通过实际应用证明了算法的有效性,最终,将该算法用于彩色人脸边缘提取中,取得了极好的应用结果。
     第四,完成了客户忠诚度分析系统。首先给出了客户忠诚度的概念,说明了研究客户忠诚度对企业的重要意义;介绍了系统的主要功能:数据预处理、重点客户发现以及客户忠诚类别划分;详细分析了数据预处理模块所采用的预处理手段和方法;给出重点客户发现和客户忠诚类别划分模块中所采用数据挖掘技术(孤立点分析、聚类分析、分类预测分析),对相应技术中所采用的算法进行了详细的描述;最后介绍了用于结果显示的结果可视化模块中的两种方法:平行坐标和分类图表。
     第五,通过客户忠诚度分析系统,对海尔公司的客户忠诚度进行了分析。主
    
    摘要
    要针对海尔公司的客户忠诚度分析问题,详细阐述了选取和处理客户忠诚度分析数据
    的过程及方法,分析了海尔公司中重点客户发现过程及结果,并结合不同的参数,对
    其结果给出了详细的分析比较,归纳得出了参数变化对重点客户发现的影响规律:此
    外,还运用聚类分析手段,得出了海尔客户数据中的大致类别,并从这些类别中选取
    合适的数据对象组成训练集,采用神经网络预测算法给出了海尔客户关系数据的最终
    的忠诚类别。从而证明了客户忠诚度分析系统的实用性。
     最后,对本文的工作进行了总结和对研究前景的展望。
Data Mining is a new technique developed from 1980s.It aims to extract the implicit, previously unknown, and potentially useful knowledge from voluminous, non-complete, fuzzy, stochastic data. Outliers analysis is a important part of data mining research. Its purpose is to find the "small patterns" from dataset. An outlier is an object that is considerably dissimilar or inconsistent with the remainder of the data. After 20 years of development, on the theory, data mining techniques is becoming more and more consummate and is expanding its application area. Now, data mining has been used in telecom, finance, busyness, weather forecast, DNA, stock market, intrusion detection and customer segmentation etc. So in this paper we first research the algorithm of outlier detection based cell, point out and improve on its shortcomings, then design a system of
    customer loyalty analysis to settle the customer loyalty analysis problem based this algorithm and some other data mining techniques, final, analyze customer loyalty of Haier company based its customer relationship data.
    First, we described the background of research and pointed out its significance. The domestic and foreign situation of data mining research was analyzed from theoretical and applying aspects. After analyzing the general progress of knowledge discovery we gave a classic framework of a data mining system, analyzed main function of every module and expatiate on the technique of data mining.
    Second, The research process and the current situation of outlier detection are reviewed. The algorithms of outlier detection based distance; density, deviation and high dimension are introduced. The content of these algorithms is analyzed. The disadvantages and advantages of these algorithms are compared.
    Third, Based on the algorithm of based-cell outlier detection, an outlier-analysis algorithm to reduce the boundary influence is presented. The data spatial cell partitioning and data object allocating methods based on the problem of boundary outlier misjudgment in cell outlier mining algorithms are discussed. Then a dynamic adjustment function on dataset boundary threshold is defined and an improved algorithm on the cell-based outlier is brought forward. It can greatly reduce the amount of misjudgment on boundary outlier by the algorithm discussed in this paper without increasing the complexity and the calculating time of the original algorithm. The validity of the new algorithm has been verified by some instances. Finally, we used this algorithm in the edge extraction of color
    
    
    
    face images and the effect is satisfying.
    Forth, a customer loyalty analysis system is designed. The definition of customer loyalty is present. The significance to research the customer loyalty is indicated. Then the functions of this system are explained, which include data preprocessing, key customer finding, customer loyalty partitioning. The preprocessing methods of data preprocessing module are discuss. Data mining techniques used in key customer finding module and customer loyalty partitioning module are given out. The algorithms, which we use in these techniques, are depicted. Finally, result visualization module is introduced) which include tow method: parallel coordinates and categorical chart.
    Fifth, the customer loyalty of Haier Company is analyzed by the customer loyalty analysis system. In order to analyze the customer loyalty of Haier Company, the process and way, which we choose and deal the analyzed data, is discussed. The results of the key customer finding is analyzed and compared based different parameters. The rule of parameter change and key customer finding is got. Beside these, the classes of Haier customer data are educed by clustering. Then, the proper data objects are chosen as the train set. The final loyalty classes of customer relationship data are got by the prediction algorithm of neural network. The validity of customer loyalty analysis system is verified.
    Finally, all the results are summarized, and the study prospect is discussed.
引文
[1] P. Adriaans and D. Zantinge, Data Mining. Addion-Wesley :Harlow, England, 1996
    [2] W.H Inomn. Building the Data Warehouse. John Wiley, 1996.
    [3] R.Kimball. The Data Warehouse Toolkit. John Wiley& Sons, New York, 1996.
    [4] S. Chaudhuri and U. Dayal. An owerciew of data warehouse and OLAP technology. ACM SIGMOD Record, 1997.
    [5] Y. kodrato and R. S. Michalski. Machine, An Artificial Intelligence Approach, Vol.3. Morgan Kaufmann,1990.
    [6] P.Langley. Elements of Machine Learning. Morgan Kaufmann, 1996.
    [7] R. S. Michalski, I. Bratko, and M. Kubat. Machine Learning and Data Mining : Methods and Applications. John Wiley & Sons, 1998.
    [8] T. M. Mitchell Machine Learning McGraw Hill, 1997.
    [9] 李焕荣、王树明,一种改进的BP神经网络的预测方法及其研究,系统工程。2000. 9,76-78。
    [10] 张智星、孙春在、水谷英二(日),神经-模糊和软计算,西安交通大学出版社。
    [11] S.M Weiss and C. A. Kulikowski. Computer Systems that Learn: Classification and Prediction Methods from Statistics, Neural Nets Machine Learning, and Expert Systems. Morgan kaufmann, 1991.
    [12] C. M. Bishop. Neural Networks for Pattern Recognition. Oxford, UK:Oxford University Press, 1995.
    [13] G. Cooper and E. Herskovits. A bayesian method for the induction of probability networks from data. Machine Learning., 9:309-347,1992.
    [14] D. Heckerman, D. Geiger, and D. Chickering. Learning Bayesian networks: The combination of knowledge and statistics data. Machine Learning, 20:197,1995.
    [15] C. Elkan. Boosting and naYve bayesian learning. In Technical Report CS97-557, Dept. of Computer Science and Engineering, Univ. Calif. At San Diego, Setp. 1997.
    [16] A. Siberschatz,M. Stonebraker, and J. D. Ullman. Database research: Achievements and opportunities into the 21 st century. Acm SIGMOD Record, 25:52-63, March 1996.
    [17] M. Stonebraker. Readings in Database Systems, 2 ed. Morgan Kaufmann, 1993.
    [18] W. Ziarko. Rough Sets, Fuzzy sets , Fuzzy Sets and Knowledge Discovery. Springer-Verlag, 1994.
    [19] S.M Weiss and N. Indurkhya. Predictive Data Mining. Morgan kaufmann, 1998.
    
    
    [20] M.James. Classification Algorithms. New York: John Wiley&Sons,1985.
    [21] R. Agrawal, S. Ghosh, T. Imielinski, B. Iyer, and A. Swami. An interval classifier for database mining applications. In proc. 18 th int. Conf. Very Large Data Bases, pages 560-573, Vancouver, Canada, August 1992.
    [22] Rajeev Rastogi, Kyuseok Shim: PUBLIC: A Decision Tree Classifier that Integrates Building and Pruning. VLDB 1998: 404-415
    [23] D. E. Brow, V. Corruble, and C. L. Pittard. A comparison of decision tree classers with backpropagation neural networks for multimodal classification problem. Pattern Recognition,26:953-96 1 , 1993.
    [24] J.R. Quinlan C4. 5: Programs for Machine Learning. Morgan Kaufmann, 1993.
    [25] A.Agresti. An Intruduction to Categorical Data Analysis. John Wiley & Sons 1996.
    [26] 冯天谨,神经网络技术,海洋大学出版社。
    [27] L. Kaufman and P.J.Rousseeuw. Finding Groups in Data: An Introduction to Cluster Analysis.New York: John Wiley & Sons, 1990.
    [28] Ralambondrainy, H. (1995) A Conceptual Version of the k-Means Algorithm, Pattern Recognition Letters, 16, pp. 1147-1157.
    [29] Bobrowski, L. and Bezdek, J. C. (1991) c-Means Clustering with the l 1 and l . Norms, IEEE Transactions on Systems, Man and Cybernetics, 21(3) , pp. 545-554.
    [30] M.Ester,H.P.Kriegel,J. Sander, and X.Xu.A density-based algorithm for discovering clusters in large spatial databases. In Proc. 1996 Int. Conf.Knowledge Discovery and Data Mining(KDD'96) , pages 226-231, Portland, OR,Aug.1996.
    [31] M.Ankerst, M.Breunig, H.P.Kriegel, and J.Sander. OPTICS: Ordering points to identify the clustering structure. In Proc. 1999 ACM-SIGMOD Int. Conf Management of Data(SIGMOD'99) ,page 49-60,Philadelphia,PA,.lune. 1999.
    [32] T.Zhang, R.Ramakrishnan, and M.Livny. BIRCH: An efficient data clustering method for very large database .In Proc. 1996 ACM-SIGMOD int. Conf. Management of Data(SIGMOD'96) ,pages 103-114,Montreal,Canada,June 1996.
    [33] S.Guha,R.Rastogi, and K.Shim. Cure An efficient clustering algorithm for large database. In Proc. 1998 ACM-SIGMOD Int. Conf. Management of Data(SIGMOD'98) , pages 73-84,seattle, WA,June 1998.
    [34] Wang, Wei, Jiong Yang, and Richard Muntz. (1997) STING: A Statistical Information Grid Approach to Spatial Data Mining. Proceedings of the 23 rd Very Large Databases Conference (VLDB 1997) . Athens, Greece.
    
    
    [35] T. Kohonen Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43:59-69,1982.
    [36] Raymond T. Ng, Jiawei Han: Efficient and Effective Clustering Methods for Spatial Data Mining. VLDB 1994: 144-155
    [37] D. Hawkins. Identification of Outliers. Chapman and Hall.London, 1980
    [38] V. Barnett and T. Lewis. Outliers in Statistical Data. John Wiley, 3rd edition, 1994.
    [39] E. Knorr and R. Ng. Finding Intensional Knowledge of Distance-based Outliers. VLDB Conference Proceedings, 1999.
    [40] S. Ramaswamy, R. Rastogi, K. Shim. Efficient Algorithms for Mining Outliers from Large Data Sets. ACM SIGMOD Conference Proceedings, 2000.
    [41] E. Knorr and R. Ng. Algorithms for Mining Distance-based Outliers in Large Data Sets. VLDB Conference Proceedings, 1998.
    [42] Edwin M. Knorr, Raymond T. Ng, V. Tucakov: Distance-Based Outliers: Algorithms and Applications. VLDB Journal 8(3-4) : 237-253(2000)
    [43] Sridhar Ramaswamy, Rajeev Rastogi, Kyuseok Shim: Efficient Algorithms for Mining Outliers from Large Data Sets. SIGMOD Conference 2000: 427-438
    [44] M. M. Breunig, H.-P. Kriegel, R. Ng, J. Sander. LOF: Identifying Density-Based Local Outliers. In ACM SIGMOD Conference Proceedings, 2000.
    [45] Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng 1 , J6rg Sander OPTICS-OF: Identifying Local Outliers .In Conf of PKDD1999
    [46] Rakesh Agrawal.Prabhakar Ragaran A Linear Method for Deviation Detection in Large Databases. KDD Conference Proceedings, 1995
    [47] C. C. Aggarwal and P. Yu. Outlier Detection for High Dimensional Data. In Proc. of ACM SIGMOD'2001
    [48] M. Dash. And Liu. And J. Yao. Dimensionality reduction of unsupervised data. In Proc. 9 th IEEE intl. Conf. On Tools with AI(ICTAI'97) , pages 532-539,IEEE Computer Science, 1997.
    [49] U. Fayyad and K.Irani. Multi-interval discretization of continuous-values attributes for classification learning. In proc. 13 th intl. Joint Conf. On Arti_cial Intelligence (IJCAI'93) pages 1022-1027,Morgan Kaufmann Publishers, 1993.
    [50] R. Kerber. Discretization of numeric attributes. In Proc. 9 th Natl. Conf. On Artificial Intelligence (AAAI'92) , pages 123-128, AAAI/MIT press, 1992.
    [51] M. W. Craven and J. W.Shavilk. Using neural networks in data mining. Future Generation Computer system. 13:211-229,1997.
    
    
    [52] J. MacQueen. Some methods for classification and analysis of multivariate observations. Proc. 5th Berkelely symp. Math. Statist, Prob., 1:281-297,1967
    [53] D. A. Keim. Visual techniques for exploring database. In Tutorial Notes,3rd Int. Conf. On Knowledge Discovery and Data Mining (KDD97) ,Newport Bcach.CA, Aug. 1997.
    [54] Ying-Huey Fua, Matthew O. Ward, and Elke A. Rundensteiner. Hierarchical parallel coordinates for exploration of large datasets. In IEEE Visualization 1999 Proceedings, pages 43-50. IEEE Computer Society, 1999

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700