A Data-Aware MultiWorkflow Cluster Scheduler

César Acevedo, Porfidio Hernandez, Antonio Espinosa, Víctor Méndez

2016

Abstract

Previous scheduling research work is based on the analysis of the computational time of application workflows. Current use of clusters deals with the execution of multiworkflows that may share applications and input files. In order to reduce the makespan of such multiworkflows adequate data allocation policies should be applied to reduce input data latency. We propose a scheduling strategy for multiworkflows that considers the data location of shared input files in different locations of the storage system of the cluster. For that, we first merge all workflows in a study and evaluate the global design pattern obtained. Then, we apply a classic list scheduling heuristic considering the location of the input files in the storage system to reduce the communication overhead of the applications. We have evaluated our proposal with an initial set of experimental environments showing promising results of up to 20% makespan improvement.

Paper Citation

in Harvard Style

Acevedo C., Hernandez P., Espinosa A. and Méndez V. (2016). A Data-Aware MultiWorkflow Cluster Scheduler . In Proceedings of the 1st International Conference on Complex Information Systems - Volume 1: COMPLEXIS, ISBN 978-989-758-181-6, pages 95-102. DOI: 10.5220/0005932000950102

in Bibtex Style

@conference{complexis16,
author={César Acevedo and Porfidio Hernandez and Antonio Espinosa and Víctor Méndez},
title={A Data-Aware MultiWorkflow Cluster Scheduler},
booktitle={Proceedings of the 1st International Conference on Complex Information Systems - Volume 1: COMPLEXIS,},
year={2016},
pages={95-102},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005932000950102},
isbn={978-989-758-181-6},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Complex Information Systems - Volume 1: COMPLEXIS,
TI - A Data-Aware MultiWorkflow Cluster Scheduler
SN - 978-989-758-181-6
AU - Acevedo C.
AU - Hernandez P.
AU - Espinosa A.
AU - Méndez V.
PY - 2016
SP - 95
EP - 102
DO - 10.5220/0005932000950102