基于过程神经元网络的时间序列数据挖掘模型及其应用
|
摘要
过程神经元网络(PNN,Process Neural Networks)是传统神经元网络扩展到时间域上的一种新型人工神经网络模型。网络的输入与输出均可为依赖于时间(或过程)变化的连续函数。模型具有良好的非线性性质、强泛化能力和高容错性等特点。针对传统人工神经网络在解决时间序列数据挖掘问题时受到输入同步瞬时限制的问题,论文提出了一类基于过程神经元网络的时间序列数据挖掘模型和算法。这类模型在处理时间序列数据挖掘问题时,能够充分反映时间序列中实际存在的时间累积效应,其模型精度和泛化能力都高于传统人工神经元网络。因此,研究基于过程神经元网络的时间序列数据挖掘模型和算法对于解决时间序列数据挖掘问题具有很重要的意义。
论文首先介绍了时间序列数据挖掘的研究现状,分析了传统人工神经网络解决时间序列数据挖掘问题的局限性,并指出用过程神经元网络解决时间序列数据挖掘问题;结合离散过程神经元网络模型和双隐层过程神经元网络模型,论文提出了一类基于过程神经元网络的时间序列数据挖掘模型,即时间序列过程神经元网络挖掘模型和双隐层时间序列过程神经元网络挖掘模型,并给出了基于离散Walsh函数变换的学习算法,用于确定网络的连接权值和激励阈值。论文最后将基于过程神经元网络的时间序列数据挖掘模型应用于油田地质领域中的水淹层自动判别和沉积微相自动识别等时间序列数据挖掘实际问题。在水淹层自动判别实例中,先后用时间序列过程神经元网络挖掘模型和双隐层时间序列过程神经元网络挖掘模型对大庆油田葡区4口井的测井数据进行训练识别,结果显示双隐层时间序列过程神经元网络的收敛速度和模型精度高于时间序列过程神经元网络。双隐层时间序列过程神经元网络挖掘模型实际上可以看成是时间序列过程神经元网络挖掘模型的改进模型。在沉积微相自动识别实例中,网络学习训练之前,先采用了最小决策算法对标准模式类学习样本进行筛选,以提高网络的学习效率和适应能力。为了对比实验,将经过筛选的学习样本和未经过筛选的学习样本分别输入到相同结构和参数的双隐层时间序列过程神经元网络中,进行学习训练,对比结果显示在网络学习之前先对学习样本进行筛选的确提高了网络的收敛速度和学习效率。应用实例证明了模型和算法的有效性。
Process Neural Networks (PNN) are a type of novel Artificial Neural Networks models, which can be seen as the Artificial Neural Networks in time domain. Both Inputs and outputs of the networks could be continuous functions that are related to time or procedure. The models have good nonlinear properties, powerful generalization ability and strong fault tolerance. Aimed at the problems that traditional Artificial Neural Networks have some disadvantages to solve Data Mining problems of Time Series, a type of Data Mining models and algorithms of Time Series based on Process Neural Networks are proposed in this paper. The models could reflect the temporal accumulation effect of time serie when they are applied to solve Data Mining problems of Time Series, of which the precision and generalization ability are better than those of traditional Artificial Neural Networks. Theirfore, the research on Data Mining models and algorithms of Time Series based on Process Neural Networks is significant to solve Data Mining problems of Time Series.
Firstly, the paper introduces the current research status of Time Series Data Mining problems, analyzes the disadvantages of traditional Artificial Neural Networks to solve Data Mining problems of Time Series, and presents that Process Neural Networks could be used to solve Data Mining problems of Time Series. Secondly, combined Discrete Process Neural Networks with Process Neural Networks with Double Hidden Layers, a type of Data Mining models and algorithms of Time Series based on Process Neural Networks are proposed in this paper, which are Time Series Process Neural Networks and Time Series Process Neural Networks with Double Hidden Layers. The algorithms are based on discrete Walsh conversion, which are used to solve connection weights and activation thresholds of the networks. Lastly, The Data Mining models and algorithms of Time Series based on Process Neural Networks are applied in the oil field to solve the problems of flooding formation automatic identification and sedimentary facies automatic recognition.In the application of flooding formation automatic identification, Time Series Process Neural Networks and Time Series Process Neural Networks with Double Hidden Layers are applied to train and recognize the logging data of four development wells in the area of Xinshugang respectively. The results show that the convergence rate and precision of Time Series Process Neural Networks with Double Hidden Layers are better than those of Time Series Process Neural Networks. In fact, Time Series Process Neural Networks with Double Hidden Layers could be seen as the modified model of Time Series Process Neural Networks. In the application of sedimentary facies automatic recognition, the minimum decision algorithm is used to select learning samples of standard patterns before the training of networks to improve the learning efficiency and adaptability of the networks. In order to comparie with the experiment results, both the learning samples that are selected before and the learning samples that are not selected before are used to train and identify with the same Time Series Process Neural Networks with Double Hidden Layers. The results show that used the learning samples that are selected before could improve the learning efficiency and adaptability of the networks. The results of the applications are proved the effectiveness of models and algorithms.
论文首先介绍了时间序列数据挖掘的研究现状,分析了传统人工神经网络解决时间序列数据挖掘问题的局限性,并指出用过程神经元网络解决时间序列数据挖掘问题;结合离散过程神经元网络模型和双隐层过程神经元网络模型,论文提出了一类基于过程神经元网络的时间序列数据挖掘模型,即时间序列过程神经元网络挖掘模型和双隐层时间序列过程神经元网络挖掘模型,并给出了基于离散Walsh函数变换的学习算法,用于确定网络的连接权值和激励阈值。论文最后将基于过程神经元网络的时间序列数据挖掘模型应用于油田地质领域中的水淹层自动判别和沉积微相自动识别等时间序列数据挖掘实际问题。在水淹层自动判别实例中,先后用时间序列过程神经元网络挖掘模型和双隐层时间序列过程神经元网络挖掘模型对大庆油田葡区4口井的测井数据进行训练识别,结果显示双隐层时间序列过程神经元网络的收敛速度和模型精度高于时间序列过程神经元网络。双隐层时间序列过程神经元网络挖掘模型实际上可以看成是时间序列过程神经元网络挖掘模型的改进模型。在沉积微相自动识别实例中,网络学习训练之前,先采用了最小决策算法对标准模式类学习样本进行筛选,以提高网络的学习效率和适应能力。为了对比实验,将经过筛选的学习样本和未经过筛选的学习样本分别输入到相同结构和参数的双隐层时间序列过程神经元网络中,进行学习训练,对比结果显示在网络学习之前先对学习样本进行筛选的确提高了网络的收敛速度和学习效率。应用实例证明了模型和算法的有效性。
Process Neural Networks (PNN) are a type of novel Artificial Neural Networks models, which can be seen as the Artificial Neural Networks in time domain. Both Inputs and outputs of the networks could be continuous functions that are related to time or procedure. The models have good nonlinear properties, powerful generalization ability and strong fault tolerance. Aimed at the problems that traditional Artificial Neural Networks have some disadvantages to solve Data Mining problems of Time Series, a type of Data Mining models and algorithms of Time Series based on Process Neural Networks are proposed in this paper. The models could reflect the temporal accumulation effect of time serie when they are applied to solve Data Mining problems of Time Series, of which the precision and generalization ability are better than those of traditional Artificial Neural Networks. Theirfore, the research on Data Mining models and algorithms of Time Series based on Process Neural Networks is significant to solve Data Mining problems of Time Series.
Firstly, the paper introduces the current research status of Time Series Data Mining problems, analyzes the disadvantages of traditional Artificial Neural Networks to solve Data Mining problems of Time Series, and presents that Process Neural Networks could be used to solve Data Mining problems of Time Series. Secondly, combined Discrete Process Neural Networks with Process Neural Networks with Double Hidden Layers, a type of Data Mining models and algorithms of Time Series based on Process Neural Networks are proposed in this paper, which are Time Series Process Neural Networks and Time Series Process Neural Networks with Double Hidden Layers. The algorithms are based on discrete Walsh conversion, which are used to solve connection weights and activation thresholds of the networks. Lastly, The Data Mining models and algorithms of Time Series based on Process Neural Networks are applied in the oil field to solve the problems of flooding formation automatic identification and sedimentary facies automatic recognition.In the application of flooding formation automatic identification, Time Series Process Neural Networks and Time Series Process Neural Networks with Double Hidden Layers are applied to train and recognize the logging data of four development wells in the area of Xinshugang respectively. The results show that the convergence rate and precision of Time Series Process Neural Networks with Double Hidden Layers are better than those of Time Series Process Neural Networks. In fact, Time Series Process Neural Networks with Double Hidden Layers could be seen as the modified model of Time Series Process Neural Networks. In the application of sedimentary facies automatic recognition, the minimum decision algorithm is used to select learning samples of standard patterns before the training of networks to improve the learning efficiency and adaptability of the networks. In order to comparie with the experiment results, both the learning samples that are selected before and the learning samples that are not selected before are used to train and identify with the same Time Series Process Neural Networks with Double Hidden Layers. The results show that used the learning samples that are selected before could improve the learning efficiency and adaptability of the networks. The results of the applications are proved the effectiveness of models and algorithms.
引文
[1] Mitra S, Pal S K, Mitra P. Data mining in soft computing framework: A survey. IEEE Transactions on Neural Networks, 2002,13(1):3~14.
[2]吉根林,帅克.数据挖掘技术及其应用[J].南京师大学报. 2000,23(2):25~27.
[3]焦李成,刘芳等.智能数据挖掘与知识发现[M].西安:西安电子科技大学出版社,2006.
[4] Han J, Kamber M. Data mining: Concepts and Techniques. USA: Morgan Kaufmann. 2001.
[5] Friedman J. Data mining and statistics: What is the connection. The 29th Symposium on the Interface, Houston, 1997.
[6] Povinelli R J. Using genetic algorithms to find temporal patterns indicative of time series events [C]. GECCO 2000 Workshop: Data Mining with Evolutionary Algorithms. USA: AAAI Press, 2000: 80-84.
[7] Imielinski T, Mannila H. A database perspective on knowledge discovery. Communications of ACM, 1996, 39:58~64.
[8] Han J., Dong G., Yin Y. Efficient mining of partial periodic patterns in time series databases. In Proc. 1998 Int. Conf. Data. Engineering(ICDE’99), Sydney, Australia, 1998. 106~115.
[9] Grossman S.H. A Fuzzy Approach Towards Inferential Data Mining. Computers and Industrial Engineering,1998, 35(1~2):267~270.
[10] Gyenesei, A. Mining Weighted Association Rules for Fuzzy Quantitative Items. Proceeding of PAKDD Conference, 2000. 416~423.
[11] Chen M S,Han J W, Yu P S. Data mining: An overview from a database perspective. IEEE Trans. On Knowledge and Data Eng., 2001, 8(6):866~884.
[12]宋擒豹,沈均毅.神经网络数据挖掘方法中的数据准备问题[J].计算机工程与应用.2000,36(12):102~104.
[13]郑纬民,黄刚.数据挖掘纵览[M].北京:清华大学出版社.1998.
[14]张保稳.时间序列数据挖掘研究[D].西北工业大学.2002.
[15] Povinelli R. Identifying Temporal Patterns for Characterization and Prediction of Financial Time Series Events. In Proc. International Workshop on Temporal, Spatial and Spatio-Temporal Data Mining, TSDM2000, Lyon, France. Lecture Notes in Artificial Intelligence, 2007.
[16] Povinelli R J, Feng X. Characterization and prediction of welding droplet release using time series data mining[C]. Proceedings of Artificial Neural Networks in Engineering. New York: ASME Press, 2000: 857-862.
[17] Michael T., Rosenstein and Paul R. Concepts from Time Series. In Proceedings of the Fifteenth National Conference on Artificial Intelligence.739-745.
[18] Das G., Lin K., Mannila H., Renganathan G., Smyth P.: Rule discovery from time series, Proceedings of the Fourth International Conference on Knowledge Discovery and DataMining, 1998.
[19] Han J W. Time series abstraction methods: a survey[C]. Proc of GI Jahrestagurg Informatik, Workshop on knowledge Discovery in Databases. Dortmund: 1999:777-786.
[20] D.A.Keim.Visual techniques for exploring databases[C]. In Tutorial Notes, 3rd Int.Conf. on Knowledge Discovery and Data Mining (KDD97), Newport Beach, CA, Aug. 2001.
[21]毛国君,段立娟,王实,石云.数据挖掘原理与算法[M].北京:清华大学出版社,2005.
[22]李军,李雄飞.数据挖掘与知识发现[M].北京:高等教育出版社,2003.
[23]黄书剑.时间数据上的数据挖掘.软件学报[J].2004,15(1):43~46.
[24]刘钊,蒋良孝.基于神经网络的数据挖掘研究[J].计算机工程与应用.2004,3:190.
[25]秦大建,李志蜀.基于神经网络的时间序列组合预测模型研究及应用.计算机应用,2006.
[26]刘豹,胡代平.神经网络在预测中的一些应用研究[J].系统工程学报,1999,14(4).
[27]文新辉,牛明杰.神经网络与预测方法研究[J].预测与控制,1992(4).
[28]沈清.神经网络应用技术[M].长沙:国防科技大学出版社.1993.
[29] Funahashi, K J. On the Approximate Realization of Continuous Mapping by Neural Networks [J]. Neural Networks, 1989,2:183~192.
[30]张广杰.神经网络在油田动态预测方面的应用.石油学报.Vol.18(4):70~75
[31] Kurita, T., A method to determine then number of hidden units of three-layered neural networks by information criteria, The Transactions of the Institute of Electronics, Information and Communication Engineers (in Japanese), J73-D-II,1872-1878.
[32] Werbos.P.J.Beyond regression, New tools for prediction and analysis in the behavioral sciences Theis.Harvard University.1974
[33] Peng Cetal. Multi valued neural network and the knowledge acquisition method by the rough sets for ambiguous recognition problem. Proc. of the IEEE International Conference on Systems, Manand Cybernetics, Beijing, 1996, 736–740.
[34] Su Y M, Leung W M, Xu L. A RPCL-CLP architecture for financial time series forecasting. Proceedings of IEEEE International Conference on Neural Network, 1995, 2: 829-832.
[35] Scholkopf, K Sung, C Burges, etc. Comparing support vector machines with Gaussian kernels to radial basis function classifiers [J].IEEE Trans. Sign. Processing (S1070-9908), 1997, 45(11):2758~2765.
[36] Povinelli R J. Identifying temporal patterns for characterization and prediction of financial time series events [C]. Lecture Notes in Computer Science. London: Springer Verlag, 2001: 46-61.
[37]向国全.前向网络BP算法在数据挖掘中的运用.河南大学学报(自然科学版).Vol.29(3):42~45
[38]徐利治,王仁宏,周蕴时.函数逼近的理论与方法.上海:上海科学技术出版社,1983:36~62.
[39] He Xingui, Liang Jiuzhen. Procedure neural networks, Proceedings of conference on intelligent information proceeding, 16th World Computer Congress 2000, Beijing, China, Publishing House of Electronic Industry. 2000, 143-146.
[40]许少华,陈可为,梁久祯.基于遗传-BP神经网络的沉积微相自动识别[J].大庆石油学院学报,2001, 25(1):51-54.
[41]张立明.人工神经网络的模型及其应用[M].上海:复旦大学出版社,1995
[42]赵海青.神经网络优选组合预测模型在电力负荷预测中的应用.
[43]赵永玲.基于神经网络控制系统的故障诊断研究[D].大庆:大庆石油学院,2003.
[44]韩良浩,王印培.基于人工神经网络的含缺陷受压管道失效模式的识别.化工机械,1997,24(3):154~157.
[45]郑新侠.16Mn管道钢土壤腐蚀速率描述的人工神经网络方法[J].西安石油大学学报(自然科学版),2004,19(1):73~76.
[46]何新贵,许少华.过程神经元网络[M].北京:科学出版社,2007.
[47]许少华,肖红.基于离散Walsh变换的过程神经元网络学习算法[J].大庆石油学院学报,2003,27(4):58~61.
[48]许增福,梁静国,李盼迟,许少华.一种基于Walsh变换的反馈过程神经元网络模型及学习算法[J].信息与控制,2004,33(4):404~407.
[49]许少华,刘扬,何新贵.基于过程神经网络的水淹层自动识别[J].石油学报,2004, 25(4):54-57.
[50]蔺景龙.测井解释中的数学方法.黑龙江科学技术出版社.1994
[51]肖慈峋,娄建立,谭世君.神经网络技术用于测井解释的评述[J].测井技术,1999,23(5):389~392.
[52]王乃举.油田开发测井技术及应用[M].北京:石油工业出版社,1995,103~109.
[53]崔勇,赵澄林.神经网络技术在油田地质领域中的应用.西安石油学院学报(自然科学报).17(4):47~51
[54]罗利.神经网络在测井解释中的应用[J].天然气工业,1997,17(5).
[55]许少华,何新贵,梁久祯.一类正则模糊神经网络及在沉积微相识别中的应用[J].控制与决策,2002,17(3):332~335.
[2]吉根林,帅克.数据挖掘技术及其应用[J].南京师大学报. 2000,23(2):25~27.
[3]焦李成,刘芳等.智能数据挖掘与知识发现[M].西安:西安电子科技大学出版社,2006.
[4] Han J, Kamber M. Data mining: Concepts and Techniques. USA: Morgan Kaufmann. 2001.
[5] Friedman J. Data mining and statistics: What is the connection. The 29th Symposium on the Interface, Houston, 1997.
[6] Povinelli R J. Using genetic algorithms to find temporal patterns indicative of time series events [C]. GECCO 2000 Workshop: Data Mining with Evolutionary Algorithms. USA: AAAI Press, 2000: 80-84.
[7] Imielinski T, Mannila H. A database perspective on knowledge discovery. Communications of ACM, 1996, 39:58~64.
[8] Han J., Dong G., Yin Y. Efficient mining of partial periodic patterns in time series databases. In Proc. 1998 Int. Conf. Data. Engineering(ICDE’99), Sydney, Australia, 1998. 106~115.
[9] Grossman S.H. A Fuzzy Approach Towards Inferential Data Mining. Computers and Industrial Engineering,1998, 35(1~2):267~270.
[10] Gyenesei, A. Mining Weighted Association Rules for Fuzzy Quantitative Items. Proceeding of PAKDD Conference, 2000. 416~423.
[11] Chen M S,Han J W, Yu P S. Data mining: An overview from a database perspective. IEEE Trans. On Knowledge and Data Eng., 2001, 8(6):866~884.
[12]宋擒豹,沈均毅.神经网络数据挖掘方法中的数据准备问题[J].计算机工程与应用.2000,36(12):102~104.
[13]郑纬民,黄刚.数据挖掘纵览[M].北京:清华大学出版社.1998.
[14]张保稳.时间序列数据挖掘研究[D].西北工业大学.2002.
[15] Povinelli R. Identifying Temporal Patterns for Characterization and Prediction of Financial Time Series Events. In Proc. International Workshop on Temporal, Spatial and Spatio-Temporal Data Mining, TSDM2000, Lyon, France. Lecture Notes in Artificial Intelligence, 2007.
[16] Povinelli R J, Feng X. Characterization and prediction of welding droplet release using time series data mining[C]. Proceedings of Artificial Neural Networks in Engineering. New York: ASME Press, 2000: 857-862.
[17] Michael T., Rosenstein and Paul R. Concepts from Time Series. In Proceedings of the Fifteenth National Conference on Artificial Intelligence.739-745.
[18] Das G., Lin K., Mannila H., Renganathan G., Smyth P.: Rule discovery from time series, Proceedings of the Fourth International Conference on Knowledge Discovery and DataMining, 1998.
[19] Han J W. Time series abstraction methods: a survey[C]. Proc of GI Jahrestagurg Informatik, Workshop on knowledge Discovery in Databases. Dortmund: 1999:777-786.
[20] D.A.Keim.Visual techniques for exploring databases[C]. In Tutorial Notes, 3rd Int.Conf. on Knowledge Discovery and Data Mining (KDD97), Newport Beach, CA, Aug. 2001.
[21]毛国君,段立娟,王实,石云.数据挖掘原理与算法[M].北京:清华大学出版社,2005.
[22]李军,李雄飞.数据挖掘与知识发现[M].北京:高等教育出版社,2003.
[23]黄书剑.时间数据上的数据挖掘.软件学报[J].2004,15(1):43~46.
[24]刘钊,蒋良孝.基于神经网络的数据挖掘研究[J].计算机工程与应用.2004,3:190.
[25]秦大建,李志蜀.基于神经网络的时间序列组合预测模型研究及应用.计算机应用,2006.
[26]刘豹,胡代平.神经网络在预测中的一些应用研究[J].系统工程学报,1999,14(4).
[27]文新辉,牛明杰.神经网络与预测方法研究[J].预测与控制,1992(4).
[28]沈清.神经网络应用技术[M].长沙:国防科技大学出版社.1993.
[29] Funahashi, K J. On the Approximate Realization of Continuous Mapping by Neural Networks [J]. Neural Networks, 1989,2:183~192.
[30]张广杰.神经网络在油田动态预测方面的应用.石油学报.Vol.18(4):70~75
[31] Kurita, T., A method to determine then number of hidden units of three-layered neural networks by information criteria, The Transactions of the Institute of Electronics, Information and Communication Engineers (in Japanese), J73-D-II,1872-1878.
[32] Werbos.P.J.Beyond regression, New tools for prediction and analysis in the behavioral sciences Theis.Harvard University.1974
[33] Peng Cetal. Multi valued neural network and the knowledge acquisition method by the rough sets for ambiguous recognition problem. Proc. of the IEEE International Conference on Systems, Manand Cybernetics, Beijing, 1996, 736–740.
[34] Su Y M, Leung W M, Xu L. A RPCL-CLP architecture for financial time series forecasting. Proceedings of IEEEE International Conference on Neural Network, 1995, 2: 829-832.
[35] Scholkopf, K Sung, C Burges, etc. Comparing support vector machines with Gaussian kernels to radial basis function classifiers [J].IEEE Trans. Sign. Processing (S1070-9908), 1997, 45(11):2758~2765.
[36] Povinelli R J. Identifying temporal patterns for characterization and prediction of financial time series events [C]. Lecture Notes in Computer Science. London: Springer Verlag, 2001: 46-61.
[37]向国全.前向网络BP算法在数据挖掘中的运用.河南大学学报(自然科学版).Vol.29(3):42~45
[38]徐利治,王仁宏,周蕴时.函数逼近的理论与方法.上海:上海科学技术出版社,1983:36~62.
[39] He Xingui, Liang Jiuzhen. Procedure neural networks, Proceedings of conference on intelligent information proceeding, 16th World Computer Congress 2000, Beijing, China, Publishing House of Electronic Industry. 2000, 143-146.
[40]许少华,陈可为,梁久祯.基于遗传-BP神经网络的沉积微相自动识别[J].大庆石油学院学报,2001, 25(1):51-54.
[41]张立明.人工神经网络的模型及其应用[M].上海:复旦大学出版社,1995
[42]赵海青.神经网络优选组合预测模型在电力负荷预测中的应用.
[43]赵永玲.基于神经网络控制系统的故障诊断研究[D].大庆:大庆石油学院,2003.
[44]韩良浩,王印培.基于人工神经网络的含缺陷受压管道失效模式的识别.化工机械,1997,24(3):154~157.
[45]郑新侠.16Mn管道钢土壤腐蚀速率描述的人工神经网络方法[J].西安石油大学学报(自然科学版),2004,19(1):73~76.
[46]何新贵,许少华.过程神经元网络[M].北京:科学出版社,2007.
[47]许少华,肖红.基于离散Walsh变换的过程神经元网络学习算法[J].大庆石油学院学报,2003,27(4):58~61.
[48]许增福,梁静国,李盼迟,许少华.一种基于Walsh变换的反馈过程神经元网络模型及学习算法[J].信息与控制,2004,33(4):404~407.
[49]许少华,刘扬,何新贵.基于过程神经网络的水淹层自动识别[J].石油学报,2004, 25(4):54-57.
[50]蔺景龙.测井解释中的数学方法.黑龙江科学技术出版社.1994
[51]肖慈峋,娄建立,谭世君.神经网络技术用于测井解释的评述[J].测井技术,1999,23(5):389~392.
[52]王乃举.油田开发测井技术及应用[M].北京:石油工业出版社,1995,103~109.
[53]崔勇,赵澄林.神经网络技术在油田地质领域中的应用.西安石油学院学报(自然科学报).17(4):47~51
[54]罗利.神经网络在测井解释中的应用[J].天然气工业,1997,17(5).
[55]许少华,何新贵,梁久祯.一类正则模糊神经网络及在沉积微相识别中的应用[J].控制与决策,2002,17(3):332~335.