SSFile: A novel column-store for efficient data analysis in Hadoop-based distributed systems
- Authors
- Son, Jihoon; Ryu, Hyoseok; Yi, Sungmin; Chung, Yon Dohn
- Issue Date
- 20-9월-2015
- Publisher
- ELSEVIER SCIENCE INC
- Keywords
- Column-store; Hadoop; HDFS; Relational data analysis; Distributed systems; Server clusters
- Citation
- INFORMATION SCIENCES, v.316, pp.68 - 86
- Indexed
- SCIE
SCOPUS
- Journal Title
- INFORMATION SCIENCES
- Volume
- 316
- Start Page
- 68
- End Page
- 86
- URI
- https://scholar.korea.ac.kr/handle/2021.sw.korea/92458
- DOI
- 10.1016/j.ins.2015.04.014
- ISSN
- 0020-0255
- Abstract
- Recently, large-scale relational data analysis has gained much attention. Several Hadoop-based distributed systems have been proposed for scalable relational data analysis. Because the column-store approach is very suitable for analytic queries, many studies on column-oriented storage and query processing for Hadoop-based distributed systems have been conducted. However, two problems have arisen in existing studies, the first of which is that only a small amount of data is processed per task during distributed processing. Each task reads only the necessary data using the columnar structure. Because the task initialization in Hadoop requires a large overhead, it is inefficient that each task processes a small amount of data. The second problem is the lack of support for techniques that optimize columnar execution. Although many such techniques have been proposed for efficient columnar query execution, existing column-store methods for Hadoop-based distributed systems cannot support them efficiently. In this paper, we propose a novel column-store method called SSFile for Hadoop-based distributed systems. SSFile increases the actual amount of data processed per task and supports representative columnar execution techniques for efficient query processing. Through extensive experiments, we show that SSFile significantly improves the performance of distributed processing. (C) 2015 Elsevier Inc. All rights reserved.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Graduate School > Department of Computer Science and Engineering > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.