A multi-agent reinforcement learning approach to dynamic service composition

详细信息    查看全文

• 作者：Hongbing Wang ; ^{hbw@seu.edu.cn" class="auth_mail" title="E-mail the corresponding author} ; Xiaojun Wang ^{XiaojunWang@seu.edu.cn" class="auth_mail" title="E-mail the corresponding author} ; Xingguo Hu ^{Xingguohu@seu.edu.cn" class="auth_mail" title="E-mail the corresponding author} ; Xingzhi Zhang ^{Xingzhizhang@seu.edu.cn" class="auth_mail" title="E-mail the corresponding author} ; Mingzhu Gu ^{Mingzhugu@seu.edu.cn" class="auth_mail" title="E-mail the corresponding author}
• 关键词：Web service composition ; Reinforcement learning ; Multi-agent
• 刊名：Information Sciences
• 出版年：2016
• 出版时间：1 October 2016
• 年：2016
• 卷：363
• 期：Complete
• 页码：96-119
• 全文大小：1725 K

文摘

As a promising implementation of software systems, service composition has attracted significant attention and research, which constructs complex and value-added applications by composing existing single services to reduce the deployment time and cost. However, as the services on the Internet and the external environment are frequently changeable, these demand the service composition must be adaptive and dynamic to address these changes automatically. Therefore, this paper describes a multi-agent reinforcement learning model for the dynamic optimization of web service composition. In this model, agent can utilize reinforcement learning algorithms to interact with environment in real time to compute optimal composition strategy dynamically, and multi-agents mechanism can keep higher effectiveness in contrast to single-agent reinforcement learning. We propose a distributed Q-learning algorithm, which decompose the task into many sub-tasks and make every agent focus on own sub-task, to accelerate the convergence rate. In addition, we also introduce experience sharing strategy to improve the efficiency. As a result, these methods allow composite service to dynamically adjust itself to fit a varying environment, where the properties of the component services continue changing. Finally, a series of comparable experiments with traditional Q-learning algorithm demonstrate that our algorithms have certain validity, higher efficiency and obvious advantages.