BROWSE

ITEM VIEW & DOWNLOAD

Performance Analysis of MapReduce with In-Memory Caching in HDFS

Cited 0 times inthomson ciCited 0 times inthomson ci
Title
Performance Analysis of MapReduce with In-Memory Caching in HDFS
Author
Yoo, Tae-kyung
Advisor
Choi, Young-Ri
Keywords
MapReduce; Hadoop; Big data; In-memory caching
Issue Date
2015-02
Publisher
Graduate School of UNIST
Abstract
In this paper, we study the effects of HDFS in-memory caching on various MapReduce applications. We first evaluate the performance of seven MapReduce applications to understand different resource usage patterns. We then modify the centralized cache management system in HDFS such that individual blocks of a file can be cached. Using the modified system in HDFS, we compare the performance of MapReduce applications with in-memory caching to that without in-memory caching for workloads of a single MapReduce application and multiple MapReduce applications. In the experiments, the same workload was executed multiple times to see the effects of in-memory caching. Our experimental results show that the in-memory cache system can be beneficial to workloads of multiple I/O-intensive MapReduce applications, but the in-memory cache system cannot improve the performance of non-I/O- intensive MapReduce applications, possibly degrading the performance due to the overhead of in-memory caching.
Description
Department of Computer Engineering
URI
Go to Link
Appears in Collections:
CSE_Theses_Master
Files in This Item:
Performance Analysis of MapReduce with In-Memory Caching in HDFS.pdf Download

find_unist can give you direct access to the published full text of this article. (UNISTARs only)

Show full item record

qrcode

  • mendeley

    citeulike

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

MENU