电子交易行情预测算法研究
|
摘要
电子交易已替代传统交易方式广泛应用于股票、期货、技术产权等交易领域，伴随电子交易出现的海量历史数据以及信息的实时更新，为行情分析预测提供了可能。通过对交易过程中积累的大量历史记录进行分析，可为交易者及管理者提供有益的决策参考依据。
知识发现(KDD)与数据挖掘技术是计算机领域一个非常具有活力的研究课题，其研究成果已广泛应用于金融、医疗保健、零售、制造业、工程与科学等行业，现在国外已经把知识发现与数据挖掘技术应用于交易行情分析领域。
本文针对电子交易行情分析与预测四个方面的问题，采用知识发现与数据挖掘技术的概念、方法，综合运用数学等相关学科的技术，通过分析、比较，提出解决办法。并对实现细节进行了描述。其主要研究内容包括：
1．根据历史数据对若干可能的贝叶斯网络拓扑结构进行判优，实现判优算法设计，并利用简化的样本数据进行效果测试。
2．针对电子交易行情数据的动态特征，采用动态三维链表结构实现朴素贝叶斯分类器，并对其分类准确度和效率进行测试与分析。
3．为了能够利用关联规则发现算法对电子交易行情变化进行相关性发现，针对历史行情记录进行事务、事务集的构造算法设计，并对该方法的有效性用股票行情记录进行测试。
4．采用快速傅立叶变换(FFT)方法，实现交易行情信号从时间域向频率域的转换，并在频率域对相似性进行度量，用股票行情记录对该方法的有效性进行测试。
该课题的研究主要是根据河南省技术产权交易系统功能扩展的需要而进行，因此，在方法选择与算法设计上突出了与具体应用相结合的特点。
Instead of the traditional trade mode, Electronic trade is applied to the stock, the future and many other trade fields. Mass market quotation records and the real time market quotation updating that can be used to analyze the market come force with this mode. As a result, useful information can be gotten by analyzing the records.
KDD(Knowledge Discovery in Database) is a new emerging area in the research of artificial intelligence and databases .This technology is used in finance, medical treatment, retail, manufacture, engineering and science. It has already been used in market analysis.
The thesis discusses how to analyze the market quotation records by solving four problems in this aspect. The methods brought forward are based on the conceptions and technologies in KDD and mathematics. The algorithms and data structures are expatiated in detail.
The main work of the thesis:
Firstly, an efficient algorithm and data structure is brought forward to select the optimum Bayesian network model which best represent the dependent relationships of the variables samples.
Secondly, to realize the real time market analysis, a NB (naive Bayesian) classification engine based on a dynamic three-dimensional link table is realized and tested.
Thirdly, the algorithms to construct the records aggregations which can be processed by Apriori algorithm are designed and tested by stock market quotation records.
Fourthly, the market quotation records are transformed from time field to frequency field by FFT algorithm, the comparability is measured in frequency field.
To meet the need of the He Nan province technology property trading system, the characteristic of practice use is concerned in choosing and designing the algorithms.
知识发现(KDD)与数据挖掘技术是计算机领域一个非常具有活力的研究课题，其研究成果已广泛应用于金融、医疗保健、零售、制造业、工程与科学等行业，现在国外已经把知识发现与数据挖掘技术应用于交易行情分析领域。
本文针对电子交易行情分析与预测四个方面的问题，采用知识发现与数据挖掘技术的概念、方法，综合运用数学等相关学科的技术，通过分析、比较，提出解决办法。并对实现细节进行了描述。其主要研究内容包括：
1．根据历史数据对若干可能的贝叶斯网络拓扑结构进行判优，实现判优算法设计，并利用简化的样本数据进行效果测试。
2．针对电子交易行情数据的动态特征，采用动态三维链表结构实现朴素贝叶斯分类器，并对其分类准确度和效率进行测试与分析。
3．为了能够利用关联规则发现算法对电子交易行情变化进行相关性发现，针对历史行情记录进行事务、事务集的构造算法设计，并对该方法的有效性用股票行情记录进行测试。
4．采用快速傅立叶变换(FFT)方法，实现交易行情信号从时间域向频率域的转换，并在频率域对相似性进行度量，用股票行情记录对该方法的有效性进行测试。
该课题的研究主要是根据河南省技术产权交易系统功能扩展的需要而进行，因此，在方法选择与算法设计上突出了与具体应用相结合的特点。
Instead of the traditional trade mode, Electronic trade is applied to the stock, the future and many other trade fields. Mass market quotation records and the real time market quotation updating that can be used to analyze the market come force with this mode. As a result, useful information can be gotten by analyzing the records.
KDD(Knowledge Discovery in Database) is a new emerging area in the research of artificial intelligence and databases .This technology is used in finance, medical treatment, retail, manufacture, engineering and science. It has already been used in market analysis.
The thesis discusses how to analyze the market quotation records by solving four problems in this aspect. The methods brought forward are based on the conceptions and technologies in KDD and mathematics. The algorithms and data structures are expatiated in detail.
The main work of the thesis:
Firstly, an efficient algorithm and data structure is brought forward to select the optimum Bayesian network model which best represent the dependent relationships of the variables samples.
Secondly, to realize the real time market analysis, a NB (naive Bayesian) classification engine based on a dynamic three-dimensional link table is realized and tested.
Thirdly, the algorithms to construct the records aggregations which can be processed by Apriori algorithm are designed and tested by stock market quotation records.
Fourthly, the market quotation records are transformed from time field to frequency field by FFT algorithm, the comparability is measured in frequency field.
To meet the need of the He Nan province technology property trading system, the characteristic of practice use is concerned in choosing and designing the algorithms.
引文
[1] 张志鸿，王桂萍，刘明业，王世卿．多种交易方式并存的实时交易系统中数据的组织访问方法．计算机工程与应用，2002(21)：74～75
[2] Jiawei Han, Micheline Kamber. 数据挖掘概念与技术(第一版)．范明，孟小峰等译．机械工业出版社，2001．
[3] T. M. Mitchell. Machine Learning. New York: McGraw—Hill, 1997.
[4] 杨炳儒．知识工程与知识发现．冶金工业出版社，2000．
[5] 王英博．数据挖掘中数据分类器的设计与实现：[硕士学位论文]，2002．
[6] 杨位钦，顾岚．时间序列分析与动态数据建模(第二版)．北京理工大学出版社，1988年．25l～265
[7] Yao Jingtao, Teng Nicholas, Poh Hean-Lee, Tan Chew Lim. Forecasting and analysis of marketing data using neural networks. Journal of Information Science and Engineering. 1998,14(4):843～862
[8] 马耀华，何瑗等．基于消息的汇率趋势预测的数据挖掘方法．计算机工程与应用，2002，15：250～256
[9] Heckerman D, Shachter R. Decision-Theoretic foundations for causal reasoning. Journal of Artificial Intelligence Research, 1995,3:405～430
[10] Heckerman D, Mandani A, Wellman M. Real-World applications of Bayesian networks. Communication of the ACM, 1995,38(3):38～45
[11] Heckerman D, Geiger D, Chickering D. Learning Bayesian network: the combination of knowledge and statistical data. Machine Learning, 1995, 20(3): 197～243
[12] Heckerman D. Bayesian networks data mining. Data Mining and Knowledge Discovery, 1997,1:79～119
[13] Geriger D, Heckerman D.A charactererization of the Dirichlet Distribution with Application to learning Bayesian networks. In:Besnard P, Hanks S eds. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence. SanFrancisco, CA:Morgan Kaufmann Publishers, Inc, 1995,196～207
[14] Buntine W. Theory refinement on Bayesian network. Proceedings of 7th
Conference on Uncertainty in Artificial Intelligence. Los Angeles, CA: Morgan Kaufmann Publishers, Inc, 1991.52～61
[15] Chickering D. Learning equivalence classes of Bayesian networks structures. In: Horvitz E, Jensen Fed. Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence. SanFrancisco, CA:Morgan Kaufmann Publishers, Inc, 1996.
[16] M. Mehta, R. Agrawal, J. Rissanen. SLIO:A fast scalable classifier for data mining. In:proc. 1996. Int. Conf. Extending Database Technology, Avignon, France, Mar. 1996.
[17] 刘红岩，陈剑等．数据挖掘中的数据分类算法综述．清华大学学报(自然科学版)，2002，42(6)：727～730
[18] 周颜军，王双成，王辉．基于贝叶斯网络的分类器研究．东北师范大学学报(自然科学版)，2003，35(2)：21～27
[19] 盛骤等．概率论与数理统计．高等教育出版社，1989．
[20] AHO, ULLMAN. The Design And Analysi s Of Computer Algorithms. 1998, 6
[21] P. Domingos, M. Pazzani. Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In: Proc. 13th Intl. Conf. Machine Learning, 1996: 105～112
[22] P. Langley, W. Iba, K. Thompson. An analysis of Bayesian network. Proc of the 10th National Conf on Artificial Intelligence San Jose, CA: AAAI Press, 1992. 223～228
[23] Nir Friedman. Bayesian network classifiers. Machine Leaning, 1997, 29(11): 131～163
[24] 赵亮，王培康．关联规则发现：综述．计算机工程与应用，2001，8：94～96
[25] 陆丽娜，陈亚萍，魏恒义等．挖掘关联规则中Apriori算法的研究．小型微型计算机系统，2000，21(9)：940～943
[26] 张梅峰，张建伟，张新敬等．基于Apriori的有效关联规则挖掘算法的研究．计算机工程与应用，2003，19：196～198
[27] 罗可，吴杰．一种基于Apriori的改进算法．计算机工程与应用，2001，20～22
[28] Holt John D, Chung Soon M. Efficient mining of association rules in text databases, In: Proc of the 1999 8th international Conference on Information Knowledge Management(CIKM' 99).Kansas City:ACM, 1999.234～242
[29] 王清毅，张波等．目前数据挖掘算法的评价．小型微型计算机系统，2000 21(1)：75～78
[30] E. O. Brigham. The Fast Fourier Transform. 柳群 译．上海科学技术出版社，1979．
[31] 张彦仲，沈乃汉．快速傅立叶变换及沃尔什变换(第一版)．航空工业出版社，1989．55～91
[32] 王挥．用于预测的贝叶斯网络．东北师范大学学报自然科学版，2002，34(1)：9～14
[33] 王军，周伟达．贝叶斯网络的研究与进展．电子科技，1999，8：6～7
[34] Sarkar S, Murthy I. Constructing Efficient Belief Network Structures with Expert Provided Information. IEEE Trans. On Knowledge and Data Engineering, 1996, 8(2): 195～210
[35] Goldberg David E. Genetic Algorithms in Search, Optimization and Machine Learning. Addison wesley Publishing Company, 1989.
[36] 王君圣，李敏强．基于数据库信息构建贝叶斯网络的GA方法．系统工程与电子技术，2000，22(9)：54～57
[37] 王玮，蔡莲红．贝叶斯网络拓扑结构确定方法的研究．小型微型计算机系统，2002，23(4)：435～437
[38] 慕春棣，戴剑彬等．用于数据挖掘的贝叶斯网络．软件学报，2000，11(5)：660～666
[39] 林士敏，田凤占，陆玉昌．贝叶斯网络的建造及其在数据采掘中的应用．清华大学学报．(自然科学版)，2001，41(1)：49～52
[40] Cooper G, Herskovits E A. Bayseian method for the introduction of probabilistic networks form data. Machine Learning 1992,9(4):309～347
[2] Jiawei Han, Micheline Kamber. 数据挖掘概念与技术(第一版)．范明，孟小峰等译．机械工业出版社，2001．
[3] T. M. Mitchell. Machine Learning. New York: McGraw—Hill, 1997.
[4] 杨炳儒．知识工程与知识发现．冶金工业出版社，2000．
[5] 王英博．数据挖掘中数据分类器的设计与实现：[硕士学位论文]，2002．
[6] 杨位钦，顾岚．时间序列分析与动态数据建模(第二版)．北京理工大学出版社，1988年．25l～265
[7] Yao Jingtao, Teng Nicholas, Poh Hean-Lee, Tan Chew Lim. Forecasting and analysis of marketing data using neural networks. Journal of Information Science and Engineering. 1998,14(4):843～862
[8] 马耀华，何瑗等．基于消息的汇率趋势预测的数据挖掘方法．计算机工程与应用，2002，15：250～256
[9] Heckerman D, Shachter R. Decision-Theoretic foundations for causal reasoning. Journal of Artificial Intelligence Research, 1995,3:405～430
[10] Heckerman D, Mandani A, Wellman M. Real-World applications of Bayesian networks. Communication of the ACM, 1995,38(3):38～45
[11] Heckerman D, Geiger D, Chickering D. Learning Bayesian network: the combination of knowledge and statistical data. Machine Learning, 1995, 20(3): 197～243
[12] Heckerman D. Bayesian networks data mining. Data Mining and Knowledge Discovery, 1997,1:79～119
[13] Geriger D, Heckerman D.A charactererization of the Dirichlet Distribution with Application to learning Bayesian networks. In:Besnard P, Hanks S eds. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence. SanFrancisco, CA:Morgan Kaufmann Publishers, Inc, 1995,196～207
[14] Buntine W. Theory refinement on Bayesian network. Proceedings of 7th
Conference on Uncertainty in Artificial Intelligence. Los Angeles, CA: Morgan Kaufmann Publishers, Inc, 1991.52～61
[15] Chickering D. Learning equivalence classes of Bayesian networks structures. In: Horvitz E, Jensen Fed. Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence. SanFrancisco, CA:Morgan Kaufmann Publishers, Inc, 1996.
[16] M. Mehta, R. Agrawal, J. Rissanen. SLIO:A fast scalable classifier for data mining. In:proc. 1996. Int. Conf. Extending Database Technology, Avignon, France, Mar. 1996.
[17] 刘红岩，陈剑等．数据挖掘中的数据分类算法综述．清华大学学报(自然科学版)，2002，42(6)：727～730
[18] 周颜军，王双成，王辉．基于贝叶斯网络的分类器研究．东北师范大学学报(自然科学版)，2003，35(2)：21～27
[19] 盛骤等．概率论与数理统计．高等教育出版社，1989．
[20] AHO, ULLMAN. The Design And Analysi s Of Computer Algorithms. 1998, 6
[21] P. Domingos, M. Pazzani. Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In: Proc. 13th Intl. Conf. Machine Learning, 1996: 105～112
[22] P. Langley, W. Iba, K. Thompson. An analysis of Bayesian network. Proc of the 10th National Conf on Artificial Intelligence San Jose, CA: AAAI Press, 1992. 223～228
[23] Nir Friedman. Bayesian network classifiers. Machine Leaning, 1997, 29(11): 131～163
[24] 赵亮，王培康．关联规则发现：综述．计算机工程与应用，2001，8：94～96
[25] 陆丽娜，陈亚萍，魏恒义等．挖掘关联规则中Apriori算法的研究．小型微型计算机系统，2000，21(9)：940～943
[26] 张梅峰，张建伟，张新敬等．基于Apriori的有效关联规则挖掘算法的研究．计算机工程与应用，2003，19：196～198
[27] 罗可，吴杰．一种基于Apriori的改进算法．计算机工程与应用，2001，20～22
[28] Holt John D, Chung Soon M. Efficient mining of association rules in text databases, In: Proc of the 1999 8th international Conference on Information Knowledge Management(CIKM' 99).Kansas City:ACM, 1999.234～242
[29] 王清毅，张波等．目前数据挖掘算法的评价．小型微型计算机系统，2000 21(1)：75～78
[30] E. O. Brigham. The Fast Fourier Transform. 柳群 译．上海科学技术出版社，1979．
[31] 张彦仲，沈乃汉．快速傅立叶变换及沃尔什变换(第一版)．航空工业出版社，1989．55～91
[32] 王挥．用于预测的贝叶斯网络．东北师范大学学报自然科学版，2002，34(1)：9～14
[33] 王军，周伟达．贝叶斯网络的研究与进展．电子科技，1999，8：6～7
[34] Sarkar S, Murthy I. Constructing Efficient Belief Network Structures with Expert Provided Information. IEEE Trans. On Knowledge and Data Engineering, 1996, 8(2): 195～210
[35] Goldberg David E. Genetic Algorithms in Search, Optimization and Machine Learning. Addison wesley Publishing Company, 1989.
[36] 王君圣，李敏强．基于数据库信息构建贝叶斯网络的GA方法．系统工程与电子技术，2000，22(9)：54～57
[37] 王玮，蔡莲红．贝叶斯网络拓扑结构确定方法的研究．小型微型计算机系统，2002，23(4)：435～437
[38] 慕春棣，戴剑彬等．用于数据挖掘的贝叶斯网络．软件学报，2000，11(5)：660～666
[39] 林士敏，田凤占，陆玉昌．贝叶斯网络的建造及其在数据采掘中的应用．清华大学学报．(自然科学版)，2001，41(1)：49～52
[40] Cooper G, Herskovits E A. Bayseian method for the introduction of probabilistic networks form data. Machine Learning 1992,9(4):309～347