An anomaly detection approach for multiple monitoring data series based on latent correlation probabilistic model

详细信息    查看全文

• 作者：Jianwei Ding ; Yingbo Liu ; Li Zhang ; Jianmin Wang ; Yonghong Liu
• 关键词：Anomaly detection ; Abnormal pattern ; Multiple monitoring data series ; Latent correlation
• 刊名：Applied Intelligence
• 出版年：2016
• 出版时间：March 2016
• 年：2016
• 卷：44
• 期：2
• 页码：340-361
• 全文大小：1,834 KB
• 参考文献：1.Aggarwal CC, Philip SY (2008) Outlier detection with uncertain data. In: SDM. SIAM, pp 483–493
  2.Atzori L, Iera A, Morabito G (2010) The internet of things: A survey. Comput Netw 54(15):2787–2805CrossRef MATH
  3.Bilmes JA, et al. (1998) A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Int Comput Sci Inst 4(510):126
  4.Chan PK, Mahoney MV (2005) Modeling multiple time series for anomaly detection. In: Data Mining, Fifth IEEE International Conference on, pages 8–pp. IEEE
  5.Das K, Schneider J (2007) Detecting anomalous records in categorical datasets. In: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 220–229. ACM
  6.Dutton WH (2013) The internet of things. Available at SSRN
  7.Fawcett T (2006) An introduction to roc analysis. Pattern Recogn Lett 27(8):861–874CrossRef MathSciNet
  8.Fujimaki R, Nakata T, Tsukahara H, Sato A, Yamanishi K (2009) Mining abnormal patterns from heterogeneous time-series with irrelevant features for fault event detection. Stat Anal Data Min 2(1):1–17CrossRef MathSciNet MATH
  9.Ghoting A, Parthasarathy S, Otey ME (2008) Fast mining of distance-based outliers in high-dimensional datasets. Data Min Knowl Disc 16(3):349–364CrossRef MathSciNet
  10.Heard NA, Weston DJ, Platanioti K, Hand DJ, et al. (2010) Bayesian anomaly detection methods for social networks. Ann Appl Stat 4(2):645–662CrossRef MathSciNet MATH
  11.Hecht-Nielsen R (1989) Theory of the backpropagation neural network. In: Neural Networks, 1989. IJCNN., International Joint Conference on, pp 593–605. IEEE
  12.Hu X, Hu S, Zhang X, Zhang H, Luo L (2014) Anomaly detection based on local nearest neighbor distance descriptor in crowded scenes. Sci World J:2014
  13.Izakian H, Pedrycz W (2013) Anomaly detection in time series data using a fuzzy c-means clustering. In: IFSA World Congress and NAFIPS Annual Meeting (IFSA/NAFIPS), 2013 Joint, pp 1513–1518. IEEE
  14.Janakiram D, Adi Mallikarjuna Reddy V, Phani Kumar AVU (2006) Outlier detection in wireless sensor networks using bayesian belief networks. In: Communication System Software and Middleware, 2006. Comsware 2006. First International Conference on, pp 1–6. IEEE
  15.Keogh E, Chakrabarti K, Pazzani M, Mehrotra S (2001) Locally adaptive dimensionality reduction for indexing large time series databases. ACM SIGMOD Rec 30(2):151–162CrossRef
  16.Keogh E, Kasetty S (2003) On the need for time series data mining benchmarks: a survey and empirical demonstration. Data Min Knowl Disc 7(4):349–371CrossRef MathSciNet
  17.Kou Y, Lu C-T, Chen D (2006) Spatial weighted outlier detection. In: SDM, pp 614–618. SIAM
  18.Liu D, Lung C-H, Lambadaris I, Seddigh N (2013) Network traffic anomaly detection using clustering techniques and performance comparison. In: Electrical and Computer Engineering (CCECE), 2013 26th Annual IEEE Canadian Conference on, pp 1–4. IEEE
  19.Ma J, Perkins S (2003) Online novelty detection on temporal sequences. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 613–618. ACM
  20.Mabu S, Chen C, Lu N, Shimada K, Hirasawa K (2011) An intrusion-detection model based on fuzzy class-association-rule mining using genetic network programming. IEEE Trans Syst Man Cybern Part C Appl Rev 41(1):130–139CrossRef
  21.Mahoney MV, Chan PK (2002) Learning nonstationary models of normal network traffic for detecting novel attacks. In: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 376–385. ACM
  22.Marx ML, Larsen RJ (2006) Introduction to mathematical statistics and its applications. Pearson/Prentice Hall
  23.Mascaro S, Nicholso AE, Korb KB (2014) Anomaly detection in vessel tracks using bayesian networks. Int J Approx Reason 55(1):84–98CrossRef
  24.Moshtaghi M, Havens TC, Bezdek JC, Park L, Leckie C, Rajasegarar S, Keller JM, Palaniswami M (2011) Clustering ellipses for anomaly detection. Pattern Recogn 44(1):55–69CrossRef MATH
  25.Otey ME, Ghoting A, Parthasarathy S (2006) Fast distributed outlier detection in mixed-attribute data sets. Data Min Knowl Disc 12(2-3):203–228CrossRef MathSciNet
  26.Papadimitriou S, Sun J, Faloutsos C (2005) Streaming pattern discovery in multiple time-series. In: Proceedings of the 31st international conference on Very large data bases, pp 697–708. VLDB Endowment
  27.Peterson LE (2009) K-nearest neighbor. Scholarpedia 4(2):1883CrossRef
  28.Pokrajac D, Lazarevic A, Latecki LJ (2007) Incremental local outlier detection for data streams. In: Computational Intelligence and Data Mining, 2007. CIDM 2007. IEEE Symposium on, pp 504–515. IEEE
  29.Povinelli RJ, Johnson MT, Lindgren AC, Ye J (2004) Time series classification using gaussian mixture models of reconstructed phase spaces. IEEE Trans Knowl Data Eng 16(6):779–783CrossRef
  30.Qin M, Hwang K (2004) Frequent episode rules for internet anomaly detection. In: Network Computing and Applications, 2004.(NCA 2004). Proceedings. Third IEEE International Symposium on, pp 161–168. IEEE
  31.Schlechtingen M, Santos IF (2011) Comparative analysis of neural network and regression based condition monitoring approaches for wind turbine fault detection. Mech Syst Signal Process 25(5):1849–1875CrossRef
  32.Sheikhan M, Jadidi Z (2014) Flow-based anomaly detection in high-speed links using modified gsa-optimized neural network. Neural Comput Appl 24(3-4):599–611CrossRef
  33.Sotiris VA, Tse PW, Pecht MG (2010) Anomaly detection through a bayesian support vector machine. IEEE Trans Reliab 59(2):277–286CrossRef
  34.Su M-Y (2011) Real-time anomaly detection systems for denial-of-service attacks by weighted k-nearest-neighbor classifiers. Expert Syst Appl 38(4):3492–3498CrossRef
  35.Tandon G, Chan PK (2007) Weighting versus pruning in rule validation for detecting network and host anomalies. In: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 697–706. ACM
  36.Thatte G, Mitra U, Heidemann J (2011) Parametric methods for anomaly detection in aggregate traffic. IEEE/ACM Trans Networking (TON) 19(2):512–525CrossRef
  37.Tsai C-F, Lin C-Y (2010) A triangle area based nearest neighbors approach to intrusion detection. Pattern Recogn 43(1):222–229CrossRef MathSciNet MATH
  38.Turner R, Fernandes E (2013) Optimizing back propagation parameters for anomaly detection. Eur J Eng Innov 10(1):10–19
  39.Wang X, Mueen A, Ding H, Trajcevski G, Scheuermann P, Keogh E (2013) Experimental comparison of representation methods and distance measures for time series data. Data Min Knowl Disc 26(2):275–309CrossRef MathSciNet
  40.Wu M, Jermaine C (2006) Outlier detection by sampling with accuracy guarantees. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 767–772. ACM
  41.Yamanishi K, Takeuchi J-I, Williams G, Milne P (2004) On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms. Data Min Knowl Disc 8(3):275–300CrossRef MathSciNet
  42.Yi B-K, Faloutsos C (2000) Fast time sequence indexing for arbitrary lp norms. In: proceedings of the 26st Intl Conference on Very Large Databases. VLDB
  43.Yu JX, Qian W, Lu H, Zhou A (2006) Finding centric local outliers in categorical/numerical spaces. Knowl Inf Syst 9(3):309–338CrossRef
  44.Yu R, He X, Liu Y (2014) Glad: group anomaly detection in social media analysis. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 372–381. ACM
  45.Zhang C, Weng N, Chang J, Zhou A (2009) Detecting abnormal trend evolution over multiple data streams. In: Advances in Data and Web Management, pp 285–296. Springer
  46.Zhang J, Wang H (2006) Detecting outlying subspaces for high-dimensional data: the new task, algorithms, and performance. Knowl Inf Syst 10(3):333–355CrossRef
  
• 作者单位：Jianwei Ding (1) (2) (4)
  Yingbo Liu (2) (4)
  Li Zhang (2) (4)
  Jianmin Wang (2) (4)
  Yonghong Liu (3)
  
  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China
  2. Institute of Information System & Engineering, School of Software, Tsinghua University, Beijing, China
  4. East Main Building, School of Software, Tsinghua University, 100084, Beijing, China
  3. General Research Institute of SANY Group, Changsha, China
  
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Mechanical Engineering
  Manufacturing, Machines and Tools
  
• 出版者：Springer Netherlands
• ISSN：1573-7497

文摘

Condition monitoring systems are widely used to monitor the working condition of equipment, generating a vast amount and variety of monitoring data in the process. The main task of surveillance focuses on detecting anomalies in these routinely collected monitoring data, intended to help detect possible faults in the equipment. However, with the rapid increase in the volume of monitoring data, it is a nontrivial task to scan all the monitoring data to detect anomalies. In this paper, we propose an approach called latent correlation-based anomaly detection (LCAD) that efficiently and effectively detects potential anomalies from a large number of correlative isomerous monitoring data series. Instead of focusing on one or more isomorphic monitoring data series, LCAD identifies anomalies by modeling the latent correlation among multiple correlative isomerous monitoring data series, using a probabilistic distribution model called the latent correlation probabilistic model, which helps to detect anomalies according to their relations with the model. Experimental results on real-world data sets show that when dealing with a large number of correlative isomerous monitoring data series, LCAD yields better performances than existing anomaly detection approaches.