A Container-centric Methodology for Benchmarking Workflow Management Systems

Vincenzo Ferme, Ana Ivanchikj, Cesare Pautasso, Marigianna Skouradaki, Frank Leymann

2016

Abstract

Trusted benchmarks should provide reproducible results obtained following a transparent and well-defined process. In this paper, we show how Containers, originally developed to ease the automated deployment of Cloud application components, can be used in the context of a benchmarking methodology. The proposed methodology focuses on Workflow Management Systems (WfMSs), a critical service orchestration middleware, which can be characterized by its architectural complexity, for which Docker Containers offer a highly suitable approach. The contributions of our work are: 1) a new benchmarking approach taking full advantage of containerization technologies; and 2) the formalization of the interaction process with the WfMS vendors described clearly in a written agreement. Thus, we take advantage of emerging Cloud technologies to address technical challenges, ensuring the performance measurements can be trusted. We also make the benchmarking process transparent, automated, and repeatable so that WfMS vendors can join the benchmarking effort.

Paper Citation

in Harvard Style

Ferme V., Ivanchikj A., Pautasso C., Skouradaki M. and Leymann F. (2016). A Container-centric Methodology for Benchmarking Workflow Management Systems . In Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 2: CLOSER, ISBN 978-989-758-182-3, pages 74-84. DOI: 10.5220/0005908400740084

in Bibtex Style

@conference{closer16,
author={Vincenzo Ferme and Ana Ivanchikj and Cesare Pautasso and Marigianna Skouradaki and Frank Leymann},
title={A Container-centric Methodology for Benchmarking Workflow Management Systems},
booktitle={Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 2: CLOSER,},
year={2016},
pages={74-84},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005908400740084},
isbn={978-989-758-182-3},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 2: CLOSER,
TI - A Container-centric Methodology for Benchmarking Workflow Management Systems
SN - 978-989-758-182-3
AU - Ferme V.
AU - Ivanchikj A.
AU - Pautasso C.
AU - Skouradaki M.
AU - Leymann F.
PY - 2016
SP - 74
EP - 84
DO - 10.5220/0005908400740084