A survey on parallel training algorithms for deep neural networks
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Yook, Dongsuk | - |
dc.contributor.author | Lee, Hyowon | - |
dc.contributor.author | Yoo, In-Chul | - |
dc.date.accessioned | 2021-08-31T16:15:08Z | - |
dc.date.available | 2021-08-31T16:15:08Z | - |
dc.date.created | 2021-06-18 | - |
dc.date.issued | 2020 | - |
dc.identifier.issn | 1225-4428 | - |
dc.identifier.uri | https://scholar.korea.ac.kr/handle/2021.sw.korea/59067 | - |
dc.description.abstract | Since a large amount of training data is typically needed to train Deep Neural Networks (DNNs), a parallel training approach is required to train the DNNs. The Stochastic Gradient Descent (SGD) algorithm is one of the most widely used methods to train the DNNs. However, since the SGD is an inherently sequential process, it requires some sort of approximation schemes to parallelize the SGD algorithm. In this paper, we review various efforts on parallelizing the SGD algorithm, and analyze the computational overhead, communication overhead, and the effects of the approximations. | - |
dc.language | Korean | - |
dc.language.iso | ko | - |
dc.publisher | ACOUSTICAL SOC KOREA | - |
dc.title | A survey on parallel training algorithms for deep neural networks | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Yook, Dongsuk | - |
dc.contributor.affiliatedAuthor | Yoo, In-Chul | - |
dc.identifier.doi | 10.7776/ASK.2020.39.6.505 | - |
dc.identifier.scopusid | 2-s2.0-85099519344 | - |
dc.identifier.wosid | 000600289500001 | - |
dc.identifier.bibliographicCitation | JOURNAL OF THE ACOUSTICAL SOCIETY OF KOREA, v.39, no.6, pp.505 - 514 | - |
dc.relation.isPartOf | JOURNAL OF THE ACOUSTICAL SOCIETY OF KOREA | - |
dc.citation.title | JOURNAL OF THE ACOUSTICAL SOCIETY OF KOREA | - |
dc.citation.volume | 39 | - |
dc.citation.number | 6 | - |
dc.citation.startPage | 505 | - |
dc.citation.endPage | 514 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.identifier.kciid | ART002649582 | - |
dc.description.journalClass | 1 | - |
dc.description.journalRegisteredClass | scopus | - |
dc.description.journalRegisteredClass | kci | - |
dc.relation.journalResearchArea | Acoustics | - |
dc.relation.journalWebOfScienceCategory | Acoustics | - |
dc.subject.keywordAuthor | Deep Neural Network (DNN) | - |
dc.subject.keywordAuthor | Deep learning | - |
dc.subject.keywordAuthor | Stochastic Gradient Descent (SGD) | - |
dc.subject.keywordAuthor | Parallel processing | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
(02841) 서울특별시 성북구 안암로 14502-3290-1114
COPYRIGHT © 2021 Korea University. All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.