基于LDA模型的海量APT通信日志特征研究
|
摘要
为实现高级持续性威胁(APT)的通信检测,提出一种对服务器端和主机端日志数据的检测方法。通过建立IP地址数据库,采用DBSCAN聚类算法对海量日志数据进行收集和处理得到异常通信日志。利用高级持续性威胁14种通信特征的隐含狄利克雷分布(LDA)建模对异常通信日志进行检测。实验结果表明,与潜在语义分析和概率潜在语义分析检测模型相比,LDA建模提高了APT通信检测的效率和准确度。
In order to realize the communication detection of the Advanced Persistent Threat(APT),this paper presents a detection method for server-side and host-side log data.It makes the establishment of IP address database and uses DBSCAN clustering algorithm to collect and deal with the massive log data to get abnormal communication log.The abnormal communication log is detected by using Latent Dirichlet Distribution(LDA) modeling of the 14 communication features of APT.Experimental results show that LDA modeling improves the efficiency and accuracy of APT communication detection compared with Latent Semantic Analysis(LSA) and Probabilistic Latent Semantic Analysis(PLSA) detection models.
In order to realize the communication detection of the Advanced Persistent Threat(APT),this paper presents a detection method for server-side and host-side log data.It makes the establishment of IP address database and uses DBSCAN clustering algorithm to collect and deal with the massive log data to get abnormal communication log.The abnormal communication log is detected by using Latent Dirichlet Distribution(LDA) modeling of the 14 communication features of APT.Experimental results show that LDA modeling improves the efficiency and accuracy of APT communication detection compared with Latent Semantic Analysis(LSA) and Probabilistic Latent Semantic Analysis(PLSA) detection models.
引文
[1]许佳,周丹平,顾海东.APT攻击及其检测技术综述[J].保密科学技术,2014(1):34-40.
[2]王宇,韩伟杰.APT攻击特征分析与对策研究[J].保密科学技术,2013(12):32-43.
[3]云晓春.国家互联网网络安全宏观态势综述[J].保密科学技术,2012(1):6-8.
[4]刘昕.大数据背景下的APT攻击检测与防御[J].电子测试,2014(1):80-81.
[5]Yen T F,Oprea A,Onarlioglu K,et al.Beehive:Largescale Log Analysis for Detecting Suspicious Activity in Enterprise Networks[C]//Proceedings of the 29th Annual Computer Security Applications Conference.New York,USA:ACM Press,2013:199-208.
[6]戴阳阳,李朝锋,徐华.初始点优化与参数自适应的密度聚类算法[J].计算机工程,2016,42(1):203-209.
[7]陈燕.对两个经典聚类算法的分析[J].现代电子技术,2007,30(17):174-176.
[8]冯少荣,肖文俊.DBSCAN聚类算法的研究与改进[J].中国矿业大学学报,2008,37(1):105-111.
[9]刘萍,郑凯伦,邹德安.基于LDA模型的科研合作推荐研究[J].情报理论与实践,2015,38(9):79-85.
[10]施乾坤.基于LDA模型的文本主题挖掘和文本静态可视化的研究[D].南宁:广西大学,2013.
[11]刘伟峰,杨爱兰.基于BIC准则和Gibbs采样的有限混合模型无监督学习算法[J].电子学报,2011,39(3):134-139.
[12]石晶,胡明,石鑫,等.基于LDA模型的文本分割[J].计算机学报,2008,31(10):1865-1873.
[13]王鹏,高铖,陈晓美.基于LDA模型的文本聚类研究[J].情报科学,2015(1):63-68.
[14]俞辉.基于LSA和PLSA的多文档自动文摘[J].计算机工程与科学,2009,31(9):108-111.
[15]牛贺.基于PLSA模型的推荐算法研究与实现[D].沈阳:东北大学,2012.
[2]王宇,韩伟杰.APT攻击特征分析与对策研究[J].保密科学技术,2013(12):32-43.
[3]云晓春.国家互联网网络安全宏观态势综述[J].保密科学技术,2012(1):6-8.
[4]刘昕.大数据背景下的APT攻击检测与防御[J].电子测试,2014(1):80-81.
[5]Yen T F,Oprea A,Onarlioglu K,et al.Beehive:Largescale Log Analysis for Detecting Suspicious Activity in Enterprise Networks[C]//Proceedings of the 29th Annual Computer Security Applications Conference.New York,USA:ACM Press,2013:199-208.
[6]戴阳阳,李朝锋,徐华.初始点优化与参数自适应的密度聚类算法[J].计算机工程,2016,42(1):203-209.
[7]陈燕.对两个经典聚类算法的分析[J].现代电子技术,2007,30(17):174-176.
[8]冯少荣,肖文俊.DBSCAN聚类算法的研究与改进[J].中国矿业大学学报,2008,37(1):105-111.
[9]刘萍,郑凯伦,邹德安.基于LDA模型的科研合作推荐研究[J].情报理论与实践,2015,38(9):79-85.
[10]施乾坤.基于LDA模型的文本主题挖掘和文本静态可视化的研究[D].南宁:广西大学,2013.
[11]刘伟峰,杨爱兰.基于BIC准则和Gibbs采样的有限混合模型无监督学习算法[J].电子学报,2011,39(3):134-139.
[12]石晶,胡明,石鑫,等.基于LDA模型的文本分割[J].计算机学报,2008,31(10):1865-1873.
[13]王鹏,高铖,陈晓美.基于LDA模型的文本聚类研究[J].情报科学,2015(1):63-68.
[14]俞辉.基于LSA和PLSA的多文档自动文摘[J].计算机工程与科学,2009,31(9):108-111.
[15]牛贺.基于PLSA模型的推荐算法研究与实现[D].沈阳:东北大学,2012.