An Efficient Approach of Processing Multiple Continuous Queries

详细信息    查看全文

• 作者：Wen Liu ; Yan-Ming Shen ; Peng Wang
• 关键词：data stream ; streams aggregation ; query sharing ; continuous query
• 刊名：Journal of Computer Science and Technology
• 出版年：2016
• 出版时间：November 2016
• 年：2016
• 卷：31
• 期：6
• 页码：1212-1227
• 全文大小：481 KB
• 刊物类别：Computer Science
• 刊物主题：Computer Science, general
  Software Engineering
  Theory of Computation
  Data Structures, Cryptology and Information Theory
  Artificial Intelligence and Robotics
  Information Systems Applications and The Internet
  Chinese Library of Science
  
• 出版者：Springer Boston
• ISSN：1860-4749
• 卷排序：31

文摘

As stream data is being more frequently collected and analyzed, stream processing systems are faced with more design challenges. One challenge is to perform continuous window aggregation, which involves intensive computation. When there are a large number of aggregation queries, the system may suffer from scalability problems. The queries are usually similar and only differ in window specifications. In this paper, we propose collaborative aggregation which promotes aggregate sharing among the windows so that repeated aggregate operations can be avoided. Different from the previous approaches in which the aggregate sharing is restricted by the window pace, we generalize the aggregation over multiple values as a series of reductions. Therefore, the results generated by each reduction step can be shared. The sharing process is formalized in the feed semantics and we present the compose-and-declare framework to determine the data sharing logic at a very low cost. Experimental results show that our approach offers an order of magnitude performance improvement to the state-of-the-art results and has a small memory footprint.