基于孤立点异常度的Web攻击数据挖掘算法
|
摘要
随着Web2.0技术的迅猛发展,网络安全变得更加突出。通过Web日志数据挖掘检测恶意攻击行为已经成为网络信息安全领域研究的一项重要技术,目前市场上的Web日志分析系统都是基于特征匹配来实现攻击行为检测的,虽然检测率较高,但对于新出现的或者尚未发现攻击特征的攻击类型无法识别。因此,深入研究从海量日志中挖掘恶意攻击行为技术有很强的理论意义与应用价值。论文深入研究了聚类算法中的距离定义以及异常度的定义,提出了一种有约束聚类的分簇方法,对Web日志中的HTTP请求进行分簇,最后利用统计学的思想,提出了一种近似正太分布的检测模型,并给出了基于孤立点异常度的Web攻击数据挖掘算法。通过实验验证表明,该算法能有效发现Web日志中的攻击数据,提高了检测率并降低了误报率。
With the rapid development of web2.0 technology,Web log analysis has become an important technology in network information security. Most web log analysis systems on the market are based on feature matching technology to achieve aggres-sive behavior detection. Although the feature matching technology has a high detection rate,it is difficult to detect the new type of at-tacks and the aggressive behavior which is not in the feature library. Therefore,the study of the digging aggressive behavior frommassive web log has great practical significance and application value. This thesis studied the definition of isolated points and its sig-nificance for the attack on data mining,and studied the definition of distance and abnormality in clustering algorithms. Thesis pro-posed a constrained clustering method to classify the HTTP parameters. At last,with the help of the statistical thinking,thesis pro-posed a similar Normal Distribution model,and a mining algorithm based on the degree of abnormal outlier. After experimental veri-fication,this algorithm has already completed the task of aggressive data mining and has a higher detection rate and a lower false alarm rate as well.
With the rapid development of web2.0 technology,Web log analysis has become an important technology in network information security. Most web log analysis systems on the market are based on feature matching technology to achieve aggres-sive behavior detection. Although the feature matching technology has a high detection rate,it is difficult to detect the new type of at-tacks and the aggressive behavior which is not in the feature library. Therefore,the study of the digging aggressive behavior frommassive web log has great practical significance and application value. This thesis studied the definition of isolated points and its sig-nificance for the attack on data mining,and studied the definition of distance and abnormality in clustering algorithms. Thesis pro-posed a constrained clustering method to classify the HTTP parameters. At last,with the help of the statistical thinking,thesis pro-posed a similar Normal Distribution model,and a mining algorithm based on the degree of abnormal outlier. After experimental veri-fication,this algorithm has already completed the task of aggressive data mining and has a higher detection rate and a lower false alarm rate as well.
引文
[1]Spiliopoulou M,Faulstich L C.一种网络利用挖掘[C]//网络与数据库国际研究会,瓦伦西亚,西班牙.1998.
[2]Suneetha K R,Krishnamoorti R.基于关联规则改进的Web日志挖掘算法[J].计算机应用,2011,29(2):101-104.
[3]陈维,阮海红.网络环境下的信息检索与数据挖掘技术[J].现代情报,2009,29(5):144-146.
[4]邢东山,沈钧毅,宋擒豹.从Web日志中挖掘用户浏览偏爱路径[J].计算机学报,2004,26(11):1518-1523.
[5]宋擒豹,沈钧毅.Web日志的高效多能挖掘算法[J].计算机研究与发展,2001,38(3):328-333.
[6]Wang J,Liu Y,Lai SY.Web日志挖掘数据预处理研究[J].计算机与信息技术,2007(6):004.
[7]Khasawneh N,Chan C C.数据挖掘中一种有效的基于用户与实体的Web日志数据预处理方法[C]//2006IEEE/WIC/ACM网络智能国际会议.IEEE计算机学会,2006:325-328.
[8]Li Y,Feng B,Mao Q.数据挖掘中关于路径实现技术[C]//计算机科学与技术,2008.ISCSCT'08.IEEE国际研讨会,2008,1:554-559.
[9]汤效琴,戴汝源.数据挖掘中聚类分析的技术方法[J].微计算机信息,2003,19(1):3-4.
[10]Hawkins D M.异常值识别[M].伦敦:查普曼和霍尔,1980.
[11]Atkinson A C,Weisberg S.一种基于模拟退火策略的适应最小二乘法与最小中值法的多种异常识别方法[J].数学与数学应用协会,1991(33):7.
[12]韦佳,彭宏,林毅申.基于改进距离的孤立点检测方法术[J].华南理工大学学报,2008,36(9):23-27.
[2]Suneetha K R,Krishnamoorti R.基于关联规则改进的Web日志挖掘算法[J].计算机应用,2011,29(2):101-104.
[3]陈维,阮海红.网络环境下的信息检索与数据挖掘技术[J].现代情报,2009,29(5):144-146.
[4]邢东山,沈钧毅,宋擒豹.从Web日志中挖掘用户浏览偏爱路径[J].计算机学报,2004,26(11):1518-1523.
[5]宋擒豹,沈钧毅.Web日志的高效多能挖掘算法[J].计算机研究与发展,2001,38(3):328-333.
[6]Wang J,Liu Y,Lai SY.Web日志挖掘数据预处理研究[J].计算机与信息技术,2007(6):004.
[7]Khasawneh N,Chan C C.数据挖掘中一种有效的基于用户与实体的Web日志数据预处理方法[C]//2006IEEE/WIC/ACM网络智能国际会议.IEEE计算机学会,2006:325-328.
[8]Li Y,Feng B,Mao Q.数据挖掘中关于路径实现技术[C]//计算机科学与技术,2008.ISCSCT'08.IEEE国际研讨会,2008,1:554-559.
[9]汤效琴,戴汝源.数据挖掘中聚类分析的技术方法[J].微计算机信息,2003,19(1):3-4.
[10]Hawkins D M.异常值识别[M].伦敦:查普曼和霍尔,1980.
[11]Atkinson A C,Weisberg S.一种基于模拟退火策略的适应最小二乘法与最小中值法的多种异常识别方法[J].数学与数学应用协会,1991(33):7.
[12]韦佳,彭宏,林毅申.基于改进距离的孤立点检测方法术[J].华南理工大学学报,2008,36(9):23-27.