Caladrius: A Performance Modelling Service for Distributed Stream Processing Systems
作者： Faria Kalim , Thomas Cooper , Huijun Wu , Yao Li , Ning Wang
关键词:
摘要: Real-time stream processing has become increasingly important in recent years and led to the development of a multitude systems. Given varying job workloads that characterize processing, these systems need be tuned adjusted maintain performance targets face variation incoming traffic. Current auto-scaling adopt series trials approach job's expected due lack modelling tools. We find general traffic trends most jobs lend themselves well prediction. Based on this premise, we built system called Caladrius forecasts future load predicts its after proposed change parallelism operators. Experimental results show is able estimate throughput CPU under given scaling configuration.
sci-hub.se 参考文章(21)
Jennifer Widom, Gurmeet Singh Manku, Chris Olston, Rajeev Motwani, Mayur Datar, Brian Babcock, Justin Rosenstein, Shivnath Babu, Arvind Arasu, Rohit Varma, Query Processing, Approximation, and Resource Management in a Data Stream Management System. conference on innovative data systems research. ,(2003)
Thomas Heinze, Zbigniew Jerzak, Gregor Hackenbroich, Christof Fetzer, Latency-aware elastic scaling for distributed data stream processing systems distributed event-based systems. pp. 13- 22 ,(2014) , 10.1145/2611286.2611294
Sanjeev Kulkarni, Nikunj Bhagat, Maosong Fu, Vikas Kedigehalli, Christopher Kellogg, Sailesh Mittal, Jignesh M. Patel, Karthik Ramasamy, Siddarth Taneja, Twitter Heron: Stream Processing at Scale international conference on management of data. pp. 239- 250 ,(2015) , 10.1145/2723372.2742788
Thomas Heinze, Valerio Pappalardo, Zbigniew Jerzak, Christof Fetzer, Auto-scaling techniques for elastic data stream processing international conference on data engineering. pp. 296- 302 ,(2014) , 10.1109/ICDEW.2014.6818344
Jielong Xu, Zhenhua Chen, Jian Tang, Sen Su, T-Storm: Traffic-Aware Online Scheduling in Storm international conference on distributed computing systems. pp. 535- 544 ,(2014) , 10.1109/ICDCS.2014.61
Bugra Gedik, Scott Schneider, Martin Hirzel, Kun-Lung Wu, Elastic Scaling for Data Stream Processing IEEE Transactions on Parallel and Distributed Systems. ,vol. 25, pp. 1447- 1463 ,(2014) , 10.1109/TPDS.2013.295
Anurag S. Maskey, Nesime Tatbul, Wolfgang Lindner, Esther Ryvkina, Alexander Rasin, Mitch Cherniack, Stan Zdonik, Ying Xing, Daniel J. Abadi, Magdalena Balazinska, Yanif Ahmad, Jeong-Hyon Hwang, The Design of the Borealis Stream Processing Engine conference on innovative data systems research. pp. 277- 289 ,(2005)
Javad Ghaderi, Sanjay Shakkottai, Rayadurgam Srikant, Scheduling Storms and Streams in the Cloud measurement and modeling of computer systems. ,vol. 43, pp. 439- 440 ,(2015) , 10.1145/2745844.2745882
Y. Liang, Real-Time VBR Video Traffic Prediction for Dynamic Bandwidth Allocation IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews). ,vol. 34, pp. 32- 47 ,(2004) , 10.1109/TSMCC.2003.818492
Yuanqiu Luo, Nirwan Ansari, Limited sharing with traffic prediction for dynamic bandwidth allocation and QoS provisioning over Ethernet passive optical networks Journal of Optical Networking. ,vol. 4, pp. 561- 572 ,(2005) , 10.1364/JON.4.000561