Proactive Congestion Avoidance for Distributed Deep Learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kang, Minkoo | - |
dc.contributor.author | Yang, Gyeongsik | - |
dc.contributor.author | Yoo, Yeonho | - |
dc.contributor.author | Yoo, Chuck | - |
dc.date.accessioned | 2021-08-30T04:36:06Z | - |
dc.date.available | 2021-08-30T04:36:06Z | - |
dc.date.created | 2021-06-19 | - |
dc.date.issued | 2021-01 | - |
dc.identifier.issn | 1424-8220 | - |
dc.identifier.uri | https://scholar.korea.ac.kr/handle/2021.sw.korea/50240 | - |
dc.description.abstract | This paper presents "Proactive Congestion Notification" (PCN), a congestion-avoidance technique for distributed deep learning (DDL). DDL is widely used to scale out and accelerate deep neural network training. In DDL, each worker trains a copy of the deep learning model with different training inputs and synchronizes the model gradients at the end of each iteration. However, it is well known that the network communication for synchronizing model parameters is the main bottleneck in DDL. Our key observation is that the DDL architecture makes each worker generate burst traffic every iteration, which causes network congestion and in turn degrades the throughput of DDL traffic. Based on this observation, the key idea behind PCN is to prevent potential congestion by proactively regulating the switch queue length before DDL burst traffic arrives at the switch, which prepares the switches for handling incoming DDL bursts. In our evaluation, PCN improves the throughput of DDL traffic by 72% on average. | - |
dc.language | English | - |
dc.language.iso | en | - |
dc.publisher | MDPI | - |
dc.title | Proactive Congestion Avoidance for Distributed Deep Learning | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Yang, Gyeongsik | - |
dc.contributor.affiliatedAuthor | Yoo, Chuck | - |
dc.identifier.doi | 10.3390/s21010174 | - |
dc.identifier.scopusid | 2-s2.0-85098545242 | - |
dc.identifier.wosid | 000606236500001 | - |
dc.identifier.bibliographicCitation | SENSORS, v.21, no.1, pp.1 - 18 | - |
dc.relation.isPartOf | SENSORS | - |
dc.citation.title | SENSORS | - |
dc.citation.volume | 21 | - |
dc.citation.number | 1 | - |
dc.citation.startPage | 1 | - |
dc.citation.endPage | 18 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Chemistry | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Instruments & Instrumentation | - |
dc.relation.journalWebOfScienceCategory | Chemistry, Analytical | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Instruments & Instrumentation | - |
dc.subject.keywordAuthor | distributed deep learning | - |
dc.subject.keywordAuthor | P4 | - |
dc.subject.keywordAuthor | congestion avoidance | - |
dc.subject.keywordAuthor | deep learning | - |
dc.subject.keywordAuthor | network congestion | - |
dc.subject.keywordAuthor | proactive congestion notification | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
(02841) 서울특별시 성북구 안암로 14502-3290-1114
COPYRIGHT © 2021 Korea University. All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.