File Download

  • Find it @ UNIST can give you direct access to the published full text of this article. (UNISTARs only)

Views & Downloads

Detailed Information

Cited time in webofscience Cited time in scopus
Metadata Downloads

Performance Analysis of MapReduce with In-Memory Caching in HDFS

Author(s)
Yoo, Tae-kyung
Advisor
Choi, Young-ri
Issued Date
2015-02
URI
https://scholarworks.unist.ac.kr/handle/201301/71894 http://unist.dcollection.net/jsp/common/DcLoOrgPer.jsp?sItemId=000001923314
Abstract
In this paper, we study the effects of HDFS in-memory caching on various MapReduce applications. We first evaluate the performance of seven MapReduce applications to understand different resource usage patterns. We then modify the centralized cache management system in HDFS such that individual blocks of a file can be cached. Using the modified system in HDFS, we compare the performance of MapReduce applications with in-memory caching to that without in-memory caching for workloads of a single MapReduce application and multiple MapReduce applications. In the experiments, the same workload was executed multiple times to see the effects of in-memory caching. Our experimental results show that the in-memory cache system can be beneficial to workloads of multiple I/O-intensive MapReduce applications, but the in-memory cache system cannot improve the performance of non-I/O- intensive MapReduce applications, possibly degrading the performance due to the overhead of in-memory caching.
Publisher
Ulsan National Institute of Science and Technology (UNIST)
Degree
Master
Major
Department of Computer Engineering

qrcode

Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.