Towards High Performance Big Data Processing by Making Use of Non-volatile Memory

Shuichi Oikawa

2015

Abstract

Cloud computing environments for big data processing require high performance storage. There are emerging high performance memory storage technologies, such as next generation non-volatile (NV) memory and battery backed NV-DIMM. While their performance is much higher than the current block storage devices, such as SSDs and HDDs, they provides only limited capacity. Such limited capacity makes it difficult for memory storage to be adapted as mass storage, and their uses in cloud computing environments have been severely limited. This paper proposes a method that combines memory storage with block storage. It makes use of memory storage as cache of block storage in order to remove the capacity limitation of memory storage. The proposed method inherits the high performance of memory storage and also the large capacity of block storage. Therefore, memory storage can be transparently used as a part of mass storage while its low overhead access can accelerate storage performance. The proposed method was implemented as a device driver of the Linux kernel. Its performance evaluation shows that it outperforms a bare SSD drive and achieves better performance on the Hadoop and database environments.

Paper Citation

in Harvard Style

Oikawa S. (2015). Towards High Performance Big Data Processing by Making Use of Non-volatile Memory . In Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-104-5, pages 529-534. DOI: 10.5220/0005463605290534

in Bibtex Style

@conference{closer15,
author={Shuichi Oikawa},
title={Towards High Performance Big Data Processing by Making Use of Non-volatile Memory},
booktitle={Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2015},
pages={529-534},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005463605290534},
isbn={978-989-758-104-5},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Towards High Performance Big Data Processing by Making Use of Non-volatile Memory
SN - 978-989-758-104-5
AU - Oikawa S.
PY - 2015
SP - 529
EP - 534
DO - 10.5220/0005463605290534