CATS: cache-aware task scheduling for Hadoop-based systems
- Authors
- Lim, Byungnam; Kim, Jong Wook; Chung, Yon Dohn
- Issue Date
- 12월-2017
- Publisher
- SPRINGER
- Keywords
- Task scheduling; Distributed systems; Hadoop; In-memory
- Citation
- CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, v.20, no.4, pp.3691 - 3705
- Indexed
- SCIE
SCOPUS
- Journal Title
- CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS
- Volume
- 20
- Number
- 4
- Start Page
- 3691
- End Page
- 3705
- URI
- https://scholar.korea.ac.kr/handle/2021.sw.korea/81433
- DOI
- 10.1007/s10586-017-0920-6
- ISSN
- 1386-7857
- Abstract
- Today with the explosion of big data, data-intensive cluster computing systems have driven to a new data processing paradigm. As Hadoop, one of the most famous data processing frameworks, achieves high performance by running multiple tasks in parallel across nodes in large clusters, task scheduling is considered as one of the most important factors affecting the overall performance. In modern operating systems, caching is used to improve local disk access times, providing data from the main memory without disk accesses. This option, however, is poorly utilized by existing task scheduling methods of Hadoop-based systems, mainly due to the inability of tracking cached data in shared-nothing distributed environments. In this paper, we propose a cache-aware task scheduling method, cache-aware task scheduling (CATS), for Hadoop-based systems which is able to exploit the operating system's buffer cache and assign tasks to nodes in consideration of the cached data. Through comprehensive experiments, we show that the proposed cache-aware scheduling improves the overall job execution time for various workload types and data sizes.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Graduate School > Department of Computer Science and Engineering > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.