Predicting the Stability of Large-scale Distributed Stream Processing Systems on the Cloud

Tri Minh Truong, Aaron Harwood, Richard O. Sinnott

2017

Abstract

Large-scale topology-based stream processing systems are non-trivial to build and deploy. They require understanding of the performance, cost of deployment and considerations of potential downtime. Our work considers stability as a primary characteristic of these systems. By stability, we mean that unstable systems exhibit large-spikes in latency and can drop throughput frequently or unpredictably. Such instabilities can be due to variations of workloads or underlying hardware platforms that are often difficult to predict. To understand and tackle this for large-scale stream processing systems, we apply queueing theory and simulate the results through a series of experiments on the Cloud.

Paper Citation

in Harvard Style

Truong T., Harwood A. and Sinnott R. (2017). Predicting the Stability of Large-scale Distributed Stream Processing Systems on the Cloud . In Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-243-1, pages 603-610. DOI: 10.5220/0006357606030610

in Bibtex Style

@conference{closer17,
author={Tri Minh Truong and Aaron Harwood and Richard O. Sinnott},
title={Predicting the Stability of Large-scale Distributed Stream Processing Systems on the Cloud},
booktitle={Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2017},
pages={603-610},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006357606030610},
isbn={978-989-758-243-1},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Predicting the Stability of Large-scale Distributed Stream Processing Systems on the Cloud
SN - 978-989-758-243-1
AU - Truong T.
AU - Harwood A.
AU - Sinnott R.
PY - 2017
SP - 603
EP - 610
DO - 10.5220/0006357606030610