基于请求分类的Web QoS控制策略模型研究
|
摘要
随着计算机网络和多媒体技术的迅猛发展,Internet的服务模式正由传统的数据通信与信息浏览向电子交易与服务转变。由于HTTP请求的增长过快,导致许多的网站的Web服务器面临超载的问题,主要表现是响应时间过长甚至拒绝服务。Web服务请求的响应时间主要由两个因素决定,网络传输的质量和Web服务器的处理性能。近年来,网络传输的服务质量(QoS)技术研究已经比较成熟,然而,仅仅靠网络QoS机制并不能完全解决端到端的QoS控制问题。例如,在服务器过载的情况下,即使网络传输质量能够得到很好的保证,仍然有可能被拒绝服务,或者响应时间大大延长而导致“拒绝服务”的效果。由此可见,由于服务器的超载问题,Web服务器已经在某种程度上成为实现端到端服务质量控制的瓶颈。因此,Web服务器系统作为核心设施,必须同样具备建立和支持QoS的机制与策略,对不同的用户或者HTTP请求提供区分服务和性能保证,为用户提供满意的服务性能,已经成为一个迫切需要解决的问题。
针对上述问题,论文就其中的关键性问题展开了工作,重点研究了请求分类策略和队列调度控制策略等等,本论文研究的主要内容如下:
(1)研究了当前流行的Web服务质量控制的体系结构,分析了其核心技术以及国际上主要的Web服务质量的商用产品,这部分对把握最新的发展趋势以及对Web服务质量研究具有指导意义。
(2)提出了一种基于请求内容和Session级别的请求分类策略。一种同时兼顾Session、Web内容和用户权限的请求分类思想,并给出了兼顾Session和内容同时考虑公平性的请求分类性能函数。通过对不同的请求内容和Session级别可以识别用户的级别,对各类用户实施不同的基于请求内容和Session级别的访问控制策略,使得级别较高的用户获得更多的系统资源,得到更好的服务和响应,从而达到对用户实行分类服务的目的。
(3)提出了一种自适应优先级调度策略APSP(Adaptive PriorityScheduling Policy)。该策略的基本思想是在系统相对空闲时,将不同级别队列中的所有请求交给服务器进行处理;在系统繁忙时,为等待在不同队列中的客户请求设置不同的调度次序,挑选优先级最高的请求交给服务器进行处理。
(4)提出了一种基于最小覆盖集的属性约简算法。在对相关理论充分研究的基础上通过构造一种改进的相关矩阵,将属性约简问题简化为一个最小覆盖集的问题。
(5)利用粗糙集理论中的粗糙近似关系来挖掘Web日志文件中的用户访问模式,通过对用户访问模式的聚类分析后将不同的页面组合根据其支持度的不同确定不同的级别,并以此作为对Web用户请求分类的基础。
With the development of the computer network and multimedia technologies, the Internet is undergoing substantial changes from a communication and browsing infrastructure to a medium for conducting business and selling services. Most of the Web applications are the HTTP requests. Many web sites had to face the problem of the server overloading. The main symptom is that users have to wait a very long time for the HTTP response and even the HTTP requests will be rejected. The responding time is generally influenced by two elements: the quantity of the service of network's transfer layer and the performance of the web server. Recently years, the research has been carried out actively in this field. However, the Internet QoS mechanism cannot completely solve the end to end control problems. For example, in condition of the server overloading, if any measure was not taken about the QoS control in the applications layer,although the QoS of the network transferring is ensured very well, the request may also be rejected or the responding time be delayed so long that leads to "reject request" result. So we could conclude that the overload of the web server is becoming the botleneck of the QoS control. The Web server systems as core establishment must have the mechanisms and strategies to set up and sustain QoS, provide service differentiation and performance assurance to different users or HTTP request.
Due to the above problems, in this article we do some key research in the field. We mainly focus on the strategy of classified services, the Qos control on the web servers, etc. The contents of the article are as following:
(1) We study the mainstream technique and the architecture of Web QoS system, analyzing their kernel techniques. This work helps us master the latest developing trend of this field.
(2) In this article, a Web QoS control strategy that is proposed is based on the session、the content of the request and the user's privilege. By the session、content of the request and the user's privilege, the customers can be classified into different classes. Different access control strategies can be imposed to the different classes customers. Higher-class customers can get more system resource, so that then can enjoy better service and faster HTTP response.
(3) An Adaptive Priority Scheduling Policy is proposed to change the order of server process request. On the one hand, when the system was idle relatively, all requests which will be handled by the Web servers are from different queues. On the other hand, when the system was buzy relatively, the Web servers will set the scheduling order for the different queues, and then select the high-priority request to the Web server for processing.
(4) An attribute reduction algorithm of rough set based on the minimal covering set is proposed. The attribute reduction in rough set is simplified the problem of the minimal covering set by constructing the relation matrix of the knowledge system.
(5) A rough approximation-based clustering to cluster Web transactions form Web access logs is proposed. Based on the clustering analysis of the user's pattern, the different web pages will be combinated by their supportance. Using this approach, users can effectively mine web log records to discover web page access patterns. This is the basic theory of the claasification of user's request.
针对上述问题,论文就其中的关键性问题展开了工作,重点研究了请求分类策略和队列调度控制策略等等,本论文研究的主要内容如下:
(1)研究了当前流行的Web服务质量控制的体系结构,分析了其核心技术以及国际上主要的Web服务质量的商用产品,这部分对把握最新的发展趋势以及对Web服务质量研究具有指导意义。
(2)提出了一种基于请求内容和Session级别的请求分类策略。一种同时兼顾Session、Web内容和用户权限的请求分类思想,并给出了兼顾Session和内容同时考虑公平性的请求分类性能函数。通过对不同的请求内容和Session级别可以识别用户的级别,对各类用户实施不同的基于请求内容和Session级别的访问控制策略,使得级别较高的用户获得更多的系统资源,得到更好的服务和响应,从而达到对用户实行分类服务的目的。
(3)提出了一种自适应优先级调度策略APSP(Adaptive PriorityScheduling Policy)。该策略的基本思想是在系统相对空闲时,将不同级别队列中的所有请求交给服务器进行处理;在系统繁忙时,为等待在不同队列中的客户请求设置不同的调度次序,挑选优先级最高的请求交给服务器进行处理。
(4)提出了一种基于最小覆盖集的属性约简算法。在对相关理论充分研究的基础上通过构造一种改进的相关矩阵,将属性约简问题简化为一个最小覆盖集的问题。
(5)利用粗糙集理论中的粗糙近似关系来挖掘Web日志文件中的用户访问模式,通过对用户访问模式的聚类分析后将不同的页面组合根据其支持度的不同确定不同的级别,并以此作为对Web用户请求分类的基础。
With the development of the computer network and multimedia technologies, the Internet is undergoing substantial changes from a communication and browsing infrastructure to a medium for conducting business and selling services. Most of the Web applications are the HTTP requests. Many web sites had to face the problem of the server overloading. The main symptom is that users have to wait a very long time for the HTTP response and even the HTTP requests will be rejected. The responding time is generally influenced by two elements: the quantity of the service of network's transfer layer and the performance of the web server. Recently years, the research has been carried out actively in this field. However, the Internet QoS mechanism cannot completely solve the end to end control problems. For example, in condition of the server overloading, if any measure was not taken about the QoS control in the applications layer,although the QoS of the network transferring is ensured very well, the request may also be rejected or the responding time be delayed so long that leads to "reject request" result. So we could conclude that the overload of the web server is becoming the botleneck of the QoS control. The Web server systems as core establishment must have the mechanisms and strategies to set up and sustain QoS, provide service differentiation and performance assurance to different users or HTTP request.
Due to the above problems, in this article we do some key research in the field. We mainly focus on the strategy of classified services, the Qos control on the web servers, etc. The contents of the article are as following:
(1) We study the mainstream technique and the architecture of Web QoS system, analyzing their kernel techniques. This work helps us master the latest developing trend of this field.
(2) In this article, a Web QoS control strategy that is proposed is based on the session、the content of the request and the user's privilege. By the session、content of the request and the user's privilege, the customers can be classified into different classes. Different access control strategies can be imposed to the different classes customers. Higher-class customers can get more system resource, so that then can enjoy better service and faster HTTP response.
(3) An Adaptive Priority Scheduling Policy is proposed to change the order of server process request. On the one hand, when the system was idle relatively, all requests which will be handled by the Web servers are from different queues. On the other hand, when the system was buzy relatively, the Web servers will set the scheduling order for the different queues, and then select the high-priority request to the Web server for processing.
(4) An attribute reduction algorithm of rough set based on the minimal covering set is proposed. The attribute reduction in rough set is simplified the problem of the minimal covering set by constructing the relation matrix of the knowledge system.
(5) A rough approximation-based clustering to cluster Web transactions form Web access logs is proposed. Based on the clustering analysis of the user's pattern, the different web pages will be combinated by their supportance. Using this approach, users can effectively mine web log records to discover web page access patterns. This is the basic theory of the claasification of user's request.
引文
[1]单志广,林闯等.Web QoS控制研究综述.计算机学报,2004,27(2),pages:145-156.
[2]IETF Working Group on Integrated Services.http://www.Ietf.org/html.charters/intser -vcharter.html.
[3]IETF Working Group on Differentiated Services.http://www.Ietf.org/html.charters/diffserv -charter.html.
[4]Bhatti N.,Friedrich R.Web server support for tiered services.IEEE Network,1999,pages:64-71.
[5]Banga G.Operating system support for server applications[Ph.D.dissertation].Rice University,Houston,Texas,1999.
[6]Bhoj P.,Ramanathan S.,Singhal S..Web2K:Bringing QoS to Web servers.Hewlett-Packard Laboratories:Technical Report HPL22000261,2000.
[7]Cardellini V.,Casalicchio E.,Colajanni M.Mechanisms for quality of service in Web clusters.Computer networks,2001,37,pages:761-771.
[8]李双庆.Web服务器集群技术研究.[博士学位论文].重庆大学,重庆,2003.
[9]Cardellini V.,Colajanni M.,Yu P.S.Geographic load balancing for scalable distributed Web systems.In:Proceedings of the Eighth International Symposium on Modeling,Analysis and Simulation of Computer and Telecommunication Systems,IEEE Computer Society,San Francisco,CA,USA,2000,pages:20-27.
[10]单志广.Web服务质量(QoS)控制的策略、模型及性能分析.[博士学位论文].北京科技大学,北京,2002.
[11]Andrew S.Tanenbaum.计算机网络.北京:清华大学出版社,2004,pages:338-342.
[12]Lu C.,Abdelzaher T.,Stankovic J.,Son S.A feedback control approach for guaranteeing relative delays in Web servers.In:Proceedings of IEEE Real Time Technology and Applications,Symposium,TaiPei,2001,pages:51-62.
[13]Tarek Abdelzaher,Kang G.Shin,Nina Bhatti.Performance Guarantees for Web Server End-System:A Control-Theoretical Approach.IEEE Transactions on Parallel and Distributed Systems.2002,13(1),pages:80-96.
[14]J.Almedia,M.Dabu,A.Manikntty,and P.Cao.Providing differentiated levels of service in web content hosting.In First Workshop on Internet Server Performance,Madison,Wisconisin,June 1998.
[15]V.Cardellini,E.Casalicchico,M.Colajanni,and M.Mambelli.Web Switch Support for Differentiated Services.Proc.Workshop Performance and Architecture of Web Server(PAWS),June 2001.
[16]X.Chen and P.Mohapatra.Providing differentiated services from an Internet Server.IEEE lntemet Conference on Computer Communications and Networks,Boston,MA,October 1999.
[17]Vikram Kanodia,Edward W.Knightly.Ensuring Latency Targets in Multiclass Web Servers.IEEE Transactions on Parallel and Distributed System,October 2002.
[18]T.Abdelzaher,N.Bhatti.Web server QoS management by adaptive content delivery.In:Proceeding of the 7th International Workshop on QoS,1999.
[19]杨武,李波,李双庆.一种基于会话的Web QoS控制策略.计算机科学,2003,32(5),pages:48-50.
[20]张鹏.基于公平性的DiffServ服务质量模型的研究.[硕士论文].东北大学,沈阳,2005.
[21]曹林奇,肖晓强,金士荛.DPSP_一种基于内容的客户请求调度策略.计算机研究与发展,2002,39(2),pages:142-147.
[22]朱立才,黄津津.基于内容交换的Web QoS.计算机应用,2005,25(12),pages:2966-2670.
[23]尹琦.一种改善Web QoS区分服务性能的控制策略.计算机工程与应用,2006,3,pages:143-145.
[24]黄锐.自适应QoS的研究与应用.[硕士学位论文].四川大学,成都,2003.
[25]王彬蔚.网络中的服务质量(QoS)的研究.[硕士学位论文].电子科技大学,成都,2002.
[26]李波.Web QoS技术研究.[硕士学位论文].重庆大学,2004.
[27]Pawalk Z.Rough Set.International Journal of Computer Sciences,1982(11):341-356
[28]Pawalk Z.Rough sets.Theoretical Aspects of Reasoning about Data.Boston:Kluwer Academic Publishers,1991.
[29]曾黄麟.智能计算.重庆:重庆大学出版社,2004年6月,pages:10-124.
[30]C H Papadimitriou,K Steiglitz.Combinatorial Optimization Algorithms and Complexity[M].1982.
[31]Liu Shao-hui,Sheng Qiu-jian,Wu Bin etc.Research on efficient algorithms for rough set methods.Chinese Journal of Computer,2003,26(5),pages:524-529.
[32]苗夺谦,王珏.粗糙集理论中概念与运算的信息表示.软件学报,1999(2),pages:113-116.
[33]陈彩云,李治国.关于属性约简和集合覆盖问题的探讨.计算机工程与应用,2004,(2),pages:44-47.
[34]陈瑞兵,黄文奇.一种求解集合覆盖问题的启发式算法.计算机科学,2007,34(4),pages:133-136.
[35]何明,冯博琴,马兆丰等.一种基于Rough集理论的属性约简启发式算法.小型微型计算机系统,2005,26(3),pages:356-359.
[36]陈奇伟.电子商务系统中基于ASRNET的Web QoS控制策略.[硕士学位论文].北京科技大学,北京,2005.
[37]田昌鹏.基于Web日志分析的Web QoS研究.计算机科学,2007,34(6),pages:78-80.
[38]于国防,王耀才.基于内容自适应的Web QoS提升技术研究.计算机工程与应用,2006,12,pages:156-158.
[39]刘玉军,卢泽新,钟读杭.Windows系统网络QoS分析.计算机工程,2000,26(9),p ages:38-40.
[40]刘莹,焦丽梅,孙凝晖.基于ServerScope的TCP/IP负载发生器的研究.小型微型计算机系统,2005,26(5),pages:850-854.
[41]徐雷鸣,庞博,赵耀.NS与网络模拟.北京:人民邮电出版社,2003,11,pages:1-180.
[2]IETF Working Group on Integrated Services.http://www.Ietf.org/html.charters/intser -vcharter.html.
[3]IETF Working Group on Differentiated Services.http://www.Ietf.org/html.charters/diffserv -charter.html.
[4]Bhatti N.,Friedrich R.Web server support for tiered services.IEEE Network,1999,pages:64-71.
[5]Banga G.Operating system support for server applications[Ph.D.dissertation].Rice University,Houston,Texas,1999.
[6]Bhoj P.,Ramanathan S.,Singhal S..Web2K:Bringing QoS to Web servers.Hewlett-Packard Laboratories:Technical Report HPL22000261,2000.
[7]Cardellini V.,Casalicchio E.,Colajanni M.Mechanisms for quality of service in Web clusters.Computer networks,2001,37,pages:761-771.
[8]李双庆.Web服务器集群技术研究.[博士学位论文].重庆大学,重庆,2003.
[9]Cardellini V.,Colajanni M.,Yu P.S.Geographic load balancing for scalable distributed Web systems.In:Proceedings of the Eighth International Symposium on Modeling,Analysis and Simulation of Computer and Telecommunication Systems,IEEE Computer Society,San Francisco,CA,USA,2000,pages:20-27.
[10]单志广.Web服务质量(QoS)控制的策略、模型及性能分析.[博士学位论文].北京科技大学,北京,2002.
[11]Andrew S.Tanenbaum.计算机网络.北京:清华大学出版社,2004,pages:338-342.
[12]Lu C.,Abdelzaher T.,Stankovic J.,Son S.A feedback control approach for guaranteeing relative delays in Web servers.In:Proceedings of IEEE Real Time Technology and Applications,Symposium,TaiPei,2001,pages:51-62.
[13]Tarek Abdelzaher,Kang G.Shin,Nina Bhatti.Performance Guarantees for Web Server End-System:A Control-Theoretical Approach.IEEE Transactions on Parallel and Distributed Systems.2002,13(1),pages:80-96.
[14]J.Almedia,M.Dabu,A.Manikntty,and P.Cao.Providing differentiated levels of service in web content hosting.In First Workshop on Internet Server Performance,Madison,Wisconisin,June 1998.
[15]V.Cardellini,E.Casalicchico,M.Colajanni,and M.Mambelli.Web Switch Support for Differentiated Services.Proc.Workshop Performance and Architecture of Web Server(PAWS),June 2001.
[16]X.Chen and P.Mohapatra.Providing differentiated services from an Internet Server.IEEE lntemet Conference on Computer Communications and Networks,Boston,MA,October 1999.
[17]Vikram Kanodia,Edward W.Knightly.Ensuring Latency Targets in Multiclass Web Servers.IEEE Transactions on Parallel and Distributed System,October 2002.
[18]T.Abdelzaher,N.Bhatti.Web server QoS management by adaptive content delivery.In:Proceeding of the 7th International Workshop on QoS,1999.
[19]杨武,李波,李双庆.一种基于会话的Web QoS控制策略.计算机科学,2003,32(5),pages:48-50.
[20]张鹏.基于公平性的DiffServ服务质量模型的研究.[硕士论文].东北大学,沈阳,2005.
[21]曹林奇,肖晓强,金士荛.DPSP_一种基于内容的客户请求调度策略.计算机研究与发展,2002,39(2),pages:142-147.
[22]朱立才,黄津津.基于内容交换的Web QoS.计算机应用,2005,25(12),pages:2966-2670.
[23]尹琦.一种改善Web QoS区分服务性能的控制策略.计算机工程与应用,2006,3,pages:143-145.
[24]黄锐.自适应QoS的研究与应用.[硕士学位论文].四川大学,成都,2003.
[25]王彬蔚.网络中的服务质量(QoS)的研究.[硕士学位论文].电子科技大学,成都,2002.
[26]李波.Web QoS技术研究.[硕士学位论文].重庆大学,2004.
[27]Pawalk Z.Rough Set.International Journal of Computer Sciences,1982(11):341-356
[28]Pawalk Z.Rough sets.Theoretical Aspects of Reasoning about Data.Boston:Kluwer Academic Publishers,1991.
[29]曾黄麟.智能计算.重庆:重庆大学出版社,2004年6月,pages:10-124.
[30]C H Papadimitriou,K Steiglitz.Combinatorial Optimization Algorithms and Complexity[M].1982.
[31]Liu Shao-hui,Sheng Qiu-jian,Wu Bin etc.Research on efficient algorithms for rough set methods.Chinese Journal of Computer,2003,26(5),pages:524-529.
[32]苗夺谦,王珏.粗糙集理论中概念与运算的信息表示.软件学报,1999(2),pages:113-116.
[33]陈彩云,李治国.关于属性约简和集合覆盖问题的探讨.计算机工程与应用,2004,(2),pages:44-47.
[34]陈瑞兵,黄文奇.一种求解集合覆盖问题的启发式算法.计算机科学,2007,34(4),pages:133-136.
[35]何明,冯博琴,马兆丰等.一种基于Rough集理论的属性约简启发式算法.小型微型计算机系统,2005,26(3),pages:356-359.
[36]陈奇伟.电子商务系统中基于ASRNET的Web QoS控制策略.[硕士学位论文].北京科技大学,北京,2005.
[37]田昌鹏.基于Web日志分析的Web QoS研究.计算机科学,2007,34(6),pages:78-80.
[38]于国防,王耀才.基于内容自适应的Web QoS提升技术研究.计算机工程与应用,2006,12,pages:156-158.
[39]刘玉军,卢泽新,钟读杭.Windows系统网络QoS分析.计算机工程,2000,26(9),p ages:38-40.
[40]刘莹,焦丽梅,孙凝晖.基于ServerScope的TCP/IP负载发生器的研究.小型微型计算机系统,2005,26(5),pages:850-854.
[41]徐雷鸣,庞博,赵耀.NS与网络模拟.北京:人民邮电出版社,2003,11,pages:1-180.