基于深度信念网络的高维传感器数据异常检测算法
|
摘要
越来越多的物联网数据呈现高维度特征,针对目前传感器数据异常检测算法对高维数据在线检测的困难,提出一种基于深度信念网络的高维传感器数据异常检测算法。首先利用深度信念网络对高维数据进行特征提取,降低原始数据维度,再对降维后的数据进行异常检测。在检测过程中将QSSVM(Quarter-Sphere Support Vector Machine)与滑动窗口模型相结合,实现了在线式的异常检测。通过在四组真实传感器数据上的大量实验,与先前的异常检测算法做了对比,实验结果表明,新算法相对于OCSVM(One-Class Support Vector Machine)仅利用原有算法50%的计算时间,将检测准确度提高了约20%。
The greater number of Internet of Things data begin to represent a high dimensional feature. Currently,anomaly detection algorithm of sensor data is difficult to detect the high dimensional data online. Targeting this,a kind of anomaly detection algorithm of high-dimension sensor data is put forward on the basis of Deep Belief Network(DBN). First of all,DBN is used to extract the features of high dimensional data to lower the dimension of original data. And then we can do the anomaly detection on the data of dimension reduction. During the detection process,the online anomaly detection can be achieved by combining the Quarter-Sphere Support Vector Machine(QSSVM)with the sliding window model. By doing a lot of experiments on real sensor data of four groups,we contrasted them with the previous anomaly detection algorithm. The experimental result shows that compared with the One-Class Support Vector Machine(OCSVM),the new algorithm,using only half the time of original algorithm,improved about 20% the detection accuracy of the original.
The greater number of Internet of Things data begin to represent a high dimensional feature. Currently,anomaly detection algorithm of sensor data is difficult to detect the high dimensional data online. Targeting this,a kind of anomaly detection algorithm of high-dimension sensor data is put forward on the basis of Deep Belief Network(DBN). First of all,DBN is used to extract the features of high dimensional data to lower the dimension of original data. And then we can do the anomaly detection on the data of dimension reduction. During the detection process,the online anomaly detection can be achieved by combining the Quarter-Sphere Support Vector Machine(QSSVM)with the sliding window model. By doing a lot of experiments on real sensor data of four groups,we contrasted them with the previous anomaly detection algorithm. The experimental result shows that compared with the One-Class Support Vector Machine(OCSVM),the new algorithm,using only half the time of original algorithm,improved about 20% the detection accuracy of the original.
引文
[1] Mcdonald D,Sanchez S,Madria S,et al.A Survey of Methods for Finding Outliers in Wireless Sensor Networks[J].Journal of Network & Systems Management,2015,23(1):163-182.
[2] Shahid N,Naqvi I H,Qaisar S B.Characteristics and Classification of Outlier Detection Techniques for Wireless Sensor Networks in Harsh Environments:A Survey[J].Artificial Intelligence Review,2015,43(2):193-228.
[3] Sheng B,Li Q,Mao W,et al.Outlier Detection in Sensor Networks[C]//Acm International Symposium on Mobile Ad Hoc Networking & Computing.ACM,2007.
[4] 潘渊洋,李光辉,徐勇军.基于DBSCAN的环境传感器网络异常数据检测方法[J].计算机应用与软件,2012(11):69-72.
[5] Lee D W,Kim J H.High Reliable In-Network Data Verification in Wireless Sensor Networks[J].Wireless Personal Communications,2010,54(3):501-519.
[6] 毕冉,李建中.无线传感器网络中能量高效的Top-k监测算法[J].计算机研究与发展,2014,51(11):2361-2373.
[7] 胡石,李光辉,冯海林.基于Top-k(σ)的无线传感器网络异常数据检测算法[J].南京大学学报(自然科学版),2016,52(2):261-269.
[8] Angiulli F,Fassetti F.Detecting Distance-Based Outliers in Streams of Data[C]//Sixteenth Acm Conference on Conference on Information & Knowledge Management.ACM,2007.
[9] Shahid N,Naqvi I H,Qaisar S B.One-Class Support Vector Machines:Analysis of Outlier Detection for Wireless Sensor Networks in Harsh Environments[J].Artificial Intelligence Review,2015,43(4):515-563.
[10] Krishnamachari B,Iyengar S.Distributed Bayesian Algorithms for Fault-Tolerant Event Region Detection in Wireless Sensor Networks[M].IEEE Computer Society,2004.
[11] 基于K-means聚类的WSN0异常数据检测算法[J].计算机工程,2015,41(7):124-128.
[12] Samparthi V S K,Verma H K.Outlier Detection of Data in Wireless Sensor Networks Using Kernel Density Estimation[J].International Journal of Computer Applications,2010,5(6):28-32.
[13] Jun M C,Jeong H,Kuo C C J.Distributed Spatio-Temporal Outlier Detection in Sensor Networks[C]//Defense & Security.International Society for Optics and Photonics,2005.
[14] Tax D M J,Duin R P W.Support Vector Data Description[J].Machine Learning,2004,54(1):45-66.
[15] Davis J J,Clark A J.Data Preprocessing for Anomaly Based Network Intrusion Detection:A Review[J].Computers & Security,2011,30(6):353-375.
[16] Shahid N,Naqvi I H,Bin Qaisar S.Quarter-Sphere SVM:Attribute and Spatio-Temporal Correlations Based Outlier & Event Detection in Wireless Sensor Networks[C]//Wireless Communications and Networking Conference(WCNC),2012 IEEE.IEEE,2012:2048-2053.
[17] 李昊奇,应娜,郭春生,等.基于深度信念网络和线性单分类SVM的高维异常检测[J].电信科学,2018(1):34-42.
[18] Erfani S M,Rajasegarar S,Karunasekera S,et al.High-Dimensional and Large-Scale Anomaly Detection Using a Linear One-Class SVM with Deep Learning[J].Pattern Recognition,2016,58(C):121-134.
[19] 潘志松,陈斌,缪志敏,等.Overview of Study on One-Class Classifiers[J].电子学报,2009,37(11):2496-2503.
[20] Tax D M J,Duin R P W.Support Vector Data Description[J].Machine Learning,2004,54(1):45-66.
[21] Scholkopf B.Estimating the Support of a High-Dimensional Distribution[J].Neural Computation,2014,13(7):1443-1471.
[22] Triwijoyo B K,Pradipto Y D.Detection of Hypertension Retinopathy Using Deep Learning and Boltzmann Machines[C]//Journal of Physics Conference Series.2017:012039.
[23] 徐丽坤,刘晓东,向小翠.基于深度信念网络的遥感影像识别与分类[J].地质科技情报,2017(4):249-254.金鹏(1995-),男,安徽农业大学信息与计算机学院在读硕士研究生,主要研究方向为物联网网络测量,传感器网络异常检测等方面的研究;夏晓峰(1986-),男,高级工程师,北京邮电大学计算机科学工学硕士,目前就职于国家开发银行信息科技局,曾发表多篇EI检索论文,主要工作和研究方向为金融行业网络架构设计及新技术应用等方面;乔焰(1984-),女,副教授,北京邮电大学工学博士,新加坡南洋理工大学博士后,德国德累斯顿工业大学访问学者,目前就职于安徽农业大学信息与计算机学院,曾发表SCI/EI检索论文20多篇,主持国家自然科学基金一项,国家重点实验室开放课题基金两项,主要研究方向为物联网网络测量,传感器网络异常检测等方面的研究;崔信红(1994年1月-),女,安徽农业大学信息与计算机学院在读硕士研究生,主要研究方向为物联网网络测量,传感器网络异常检测等方面的研究。
[2] Shahid N,Naqvi I H,Qaisar S B.Characteristics and Classification of Outlier Detection Techniques for Wireless Sensor Networks in Harsh Environments:A Survey[J].Artificial Intelligence Review,2015,43(2):193-228.
[3] Sheng B,Li Q,Mao W,et al.Outlier Detection in Sensor Networks[C]//Acm International Symposium on Mobile Ad Hoc Networking & Computing.ACM,2007.
[4] 潘渊洋,李光辉,徐勇军.基于DBSCAN的环境传感器网络异常数据检测方法[J].计算机应用与软件,2012(11):69-72.
[5] Lee D W,Kim J H.High Reliable In-Network Data Verification in Wireless Sensor Networks[J].Wireless Personal Communications,2010,54(3):501-519.
[6] 毕冉,李建中.无线传感器网络中能量高效的Top-k监测算法[J].计算机研究与发展,2014,51(11):2361-2373.
[7] 胡石,李光辉,冯海林.基于Top-k(σ)的无线传感器网络异常数据检测算法[J].南京大学学报(自然科学版),2016,52(2):261-269.
[8] Angiulli F,Fassetti F.Detecting Distance-Based Outliers in Streams of Data[C]//Sixteenth Acm Conference on Conference on Information & Knowledge Management.ACM,2007.
[9] Shahid N,Naqvi I H,Qaisar S B.One-Class Support Vector Machines:Analysis of Outlier Detection for Wireless Sensor Networks in Harsh Environments[J].Artificial Intelligence Review,2015,43(4):515-563.
[10] Krishnamachari B,Iyengar S.Distributed Bayesian Algorithms for Fault-Tolerant Event Region Detection in Wireless Sensor Networks[M].IEEE Computer Society,2004.
[11] 基于K-means聚类的WSN0异常数据检测算法[J].计算机工程,2015,41(7):124-128.
[12] Samparthi V S K,Verma H K.Outlier Detection of Data in Wireless Sensor Networks Using Kernel Density Estimation[J].International Journal of Computer Applications,2010,5(6):28-32.
[13] Jun M C,Jeong H,Kuo C C J.Distributed Spatio-Temporal Outlier Detection in Sensor Networks[C]//Defense & Security.International Society for Optics and Photonics,2005.
[14] Tax D M J,Duin R P W.Support Vector Data Description[J].Machine Learning,2004,54(1):45-66.
[15] Davis J J,Clark A J.Data Preprocessing for Anomaly Based Network Intrusion Detection:A Review[J].Computers & Security,2011,30(6):353-375.
[16] Shahid N,Naqvi I H,Bin Qaisar S.Quarter-Sphere SVM:Attribute and Spatio-Temporal Correlations Based Outlier & Event Detection in Wireless Sensor Networks[C]//Wireless Communications and Networking Conference(WCNC),2012 IEEE.IEEE,2012:2048-2053.
[17] 李昊奇,应娜,郭春生,等.基于深度信念网络和线性单分类SVM的高维异常检测[J].电信科学,2018(1):34-42.
[18] Erfani S M,Rajasegarar S,Karunasekera S,et al.High-Dimensional and Large-Scale Anomaly Detection Using a Linear One-Class SVM with Deep Learning[J].Pattern Recognition,2016,58(C):121-134.
[19] 潘志松,陈斌,缪志敏,等.Overview of Study on One-Class Classifiers[J].电子学报,2009,37(11):2496-2503.
[20] Tax D M J,Duin R P W.Support Vector Data Description[J].Machine Learning,2004,54(1):45-66.
[21] Scholkopf B.Estimating the Support of a High-Dimensional Distribution[J].Neural Computation,2014,13(7):1443-1471.
[22] Triwijoyo B K,Pradipto Y D.Detection of Hypertension Retinopathy Using Deep Learning and Boltzmann Machines[C]//Journal of Physics Conference Series.2017:012039.
[23] 徐丽坤,刘晓东,向小翠.基于深度信念网络的遥感影像识别与分类[J].地质科技情报,2017(4):249-254.金鹏(1995-),男,安徽农业大学信息与计算机学院在读硕士研究生,主要研究方向为物联网网络测量,传感器网络异常检测等方面的研究;夏晓峰(1986-),男,高级工程师,北京邮电大学计算机科学工学硕士,目前就职于国家开发银行信息科技局,曾发表多篇EI检索论文,主要工作和研究方向为金融行业网络架构设计及新技术应用等方面;乔焰(1984-),女,副教授,北京邮电大学工学博士,新加坡南洋理工大学博士后,德国德累斯顿工业大学访问学者,目前就职于安徽农业大学信息与计算机学院,曾发表SCI/EI检索论文20多篇,主持国家自然科学基金一项,国家重点实验室开放课题基金两项,主要研究方向为物联网网络测量,传感器网络异常检测等方面的研究;崔信红(1994年1月-),女,安徽农业大学信息与计算机学院在读硕士研究生,主要研究方向为物联网网络测量,传感器网络异常检测等方面的研究。
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |