Column Access-aware In-stream Data Cache with Stream Processing Framework

详细信息    查看全文

• 作者：Kun Ma ; Bo Yang
• 关键词：Big data ; Stream processing ; NoSQL ; Stream computing ; Data cache ; Access frequency
• 刊名：Journal of Signal Processing Systems
• 出版年：2017
• 出版时间：March 2017
• 年：2017
• 卷：86
• 期：2-3
• 页码：191-205
• 全文大小：
• 刊物类别：Engineering
• 刊物主题：Signal,Image and Speech Processing; Circuits and Systems; Electrical Engineering; Image Processing and Computer Vision; Pattern Recognition; Computer Imaging, Vision, Pattern Recognition and Graphics;
• 出版者：Springer US
• ISSN：1939-8115
• 卷排序：86

文摘

In recent years, researches focus on addressing the query bottleneck issue of big data, e.g. NoSQL databases, MapReduce and big data processing framework. Although NoSQL databases have many advantages on On-Line Analytical Processing (OLAP), it is a big project to migrate Relational Database Management System (RDBMS) to NoSQL. Therefore, the optimization of RDBMS is still important. In this paper, we construct Column Access-aware In-stream Data Cache (CAIDC) for relational databases, which is an integral part of RDBMS and in-memory cache. Furthermore, a live synchronization approach from physical RDBMS to in-memory data cache using stream processing framework is proposed. On one hand, CAIDC provides low latency while supporting log-based trigger in the presence of updates to maintain data consistency because of stream processing framework. On the other hand, CAIDC translates the frequently accessed data to column-oriented in-memory cache by the column access frequency to ensure heavy hitter queries. Finally, experimental results show that this approach is supporting a wide range of applications with big data.