TensorLightning: A Traffic-Efficient Distributed Deep Learning on Commodity Spark Clusters
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Seil | - |
dc.contributor.author | Kim, Hanjoo | - |
dc.contributor.author | Park, Jaehong | - |
dc.contributor.author | Jang, Jaehee | - |
dc.contributor.author | Jeong, Chang-Sung | - |
dc.contributor.author | Yoon, Sungroh | - |
dc.date.accessioned | 2021-09-02T21:16:10Z | - |
dc.date.available | 2021-09-02T21:16:10Z | - |
dc.date.created | 2021-06-16 | - |
dc.date.issued | 2018 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholar.korea.ac.kr/handle/2021.sw.korea/80954 | - |
dc.description.abstract | With the recent success of deep learning, the amount of data and computation continues to grow daily. Hence a distributed deep learning system that shares the training workload has been researched extensively. Although a scale-out distributed environment using commodity servers is widely used, not only is there a limit due to synchronous operation and communication traffic but also combining deep neural network (DNN) training with existing clusters often demands additional hardware and migration between different cluster frameworks or libraries, which is highly inefficient. Therefore, we propose TensorLightning which integrates the widely used data pipeline of Apache Spark with powerful deep learning libraries, Caffe and TensorFlow. TensorLightning embraces a brand-new parameter aggregation algorithm and parallel asynchronous parameter managing schemes to relieve communication discrepancies and overhead. We redesign the elastic averaging stochastic gradient descent algorithm with pruned and sparse form parameters. Our approach provides the fast and flexible DNN training with high accessibility. We evaluated our proposed framework with convolutional neural network and recurrent neural network models; the framework reduces network traffic by 67% with faster convergence. | - |
dc.language | English | - |
dc.language.iso | en | - |
dc.publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | - |
dc.title | TensorLightning: A Traffic-Efficient Distributed Deep Learning on Commodity Spark Clusters | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Jeong, Chang-Sung | - |
dc.identifier.doi | 10.1109/ACCESS.2018.2842103 | - |
dc.identifier.scopusid | 2-s2.0-85047817358 | - |
dc.identifier.wosid | 000435552100001 | - |
dc.identifier.bibliographicCitation | IEEE ACCESS, v.6, pp.27671 - 27680 | - |
dc.relation.isPartOf | IEEE ACCESS | - |
dc.citation.title | IEEE ACCESS | - |
dc.citation.volume | 6 | - |
dc.citation.startPage | 27671 | - |
dc.citation.endPage | 27680 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.subject.keywordAuthor | TensorLightning | - |
dc.subject.keywordAuthor | deep learning | - |
dc.subject.keywordAuthor | Apache Spark | - |
dc.subject.keywordAuthor | distributed system | - |
dc.subject.keywordAuthor | commodity servers | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
(02841) 서울특별시 성북구 안암로 14502-3290-1114
COPYRIGHT © 2021 Korea University. All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.