EME: An Automated, Elastic and Efficient Prototype for Provisioning Hadoop Clusters On-demand

Feras M. Awaysheh, Tomás F. Pena, José C. Cabaleiro

2017

Abstract

Aiming at enhancing the MapReduce-based applications Quality of Service (QoS), many frameworks suggest a scale-out approach, statically adding new nodes to the cluster. Such frameworks are still expensive to acquire and does not consider the optimal usage of available resources in a dynamic manner. This paper introduces a prototype to address with this issue, by extending MapReduce resource manager with dynamic provisioning and low-cost resources capacity uplift on-demand. We propose an Enhanced Mapreduce Environment (EME), to support heterogeneous environments by extending Apache Hadoop to an opportunistically containerized environment, which enhances system throughput by adding underused resources to a local or cloud based cluster. The main architectural elements of this framework are presented, as well as the requirements, challenges, and opportunities of a first prototype.

Paper Citation

in Harvard Style

Awaysheh F., Pena T. and Cabaleiro J. (2017). EME: An Automated, Elastic and Efficient Prototype for Provisioning Hadoop Clusters On-demand . In Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-243-1, pages 737-742. DOI: 10.5220/0006379607370742

in Bibtex Style

@conference{closer17,
author={Feras M. Awaysheh and Tomás F. Pena and José C. Cabaleiro},
title={EME: An Automated, Elastic and Efficient Prototype for Provisioning Hadoop Clusters On-demand},
booktitle={Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2017},
pages={737-742},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006379607370742},
isbn={978-989-758-243-1},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - EME: An Automated, Elastic and Efficient Prototype for Provisioning Hadoop Clusters On-demand
SN - 978-989-758-243-1
AU - Awaysheh F.
AU - Pena T.
AU - Cabaleiro J.
PY - 2017
SP - 737
EP - 742
DO - 10.5220/0006379607370742