Distributed File System Based on Erasure Coding for I/O-Intensive Applications

Dimitri Pertin, Sylvain David, Pierre Evenou, Benoît Parrein, Nicolas Normand

2014

Abstract

Distributed storage systems take advantage of the network, storage and computational resources to provide a scalable infrastructure. But in such large system, failures are frequent and expected. Data replication is the common technique to provide fault-tolerance but suffers from its important storage consumption. Erasure coding is an alternative that offers the same data protection but reduces significantly the storage consumption. As it entails additional workload, current storage providers limit its use for longterm storage. We present the Mojette Transform (MT), an erasure code whose computations rely on fast XOR operations. The MT is part of RozoFS, a distributed file system that provides a global namespace relying on a cluster of storage nodes. This work is part of our ongoing effort to prove that erasure coding is not necessarily a bottleneck for intense I/O applications. In order to validate our approach, we consider a case study involving a storage cluster of RozoFS that supports video editing as an I/O intensive application.

Paper Citation

in Harvard Style

Pertin D., David S., Evenou P., Parrein B. and Normand N. (2014). Distributed File System Based on Erasure Coding for I/O-Intensive Applications . In Proceedings of the 4th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-019-2, pages 451-456. DOI: 10.5220/0004960604510456

in Bibtex Style

@conference{closer14,
author={Dimitri Pertin and Sylvain David and Pierre Evenou and Benoît Parrein and Nicolas Normand},
title={Distributed File System Based on Erasure Coding for I/O-Intensive Applications},
booktitle={Proceedings of the 4th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2014},
pages={451-456},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004960604510456},
isbn={978-989-758-019-2},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Distributed File System Based on Erasure Coding for I/O-Intensive Applications
SN - 978-989-758-019-2
AU - Pertin D.
AU - David S.
AU - Evenou P.
AU - Parrein B.
AU - Normand N.
PY - 2014
SP - 451
EP - 456
DO - 10.5220/0004960604510456