Learning to Balance Local Losses via Meta-Learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Yoa, Seungdong | - |
dc.contributor.author | Jeon, Minkyu | - |
dc.contributor.author | Oh, Youngjin | - |
dc.contributor.author | Kim, Hyunwoo J. | - |
dc.date.accessioned | 2022-03-12T01:41:24Z | - |
dc.date.available | 2022-03-12T01:41:24Z | - |
dc.date.created | 2022-01-20 | - |
dc.date.issued | 2021 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholar.korea.ac.kr/handle/2021.sw.korea/138666 | - |
dc.description.abstract | The standard training for deep neural networks relies on a global and fixed loss function. For more effective training, dynamic loss functions have been recently proposed. However, the dynamic global loss function is not flexible to differentially train layers in complex deep neural networks. In this paper, we propose a general framework that learns to adaptively train each layer of deep neural networks via meta-learning. Our framework leverages the local error signals from layers and identifies which layer needs to be trained more at every iteration. Also, the proposed method improves the local loss function with our minibatch-wise dropout and cross-validation loop to alleviate meta-overfitting. The experiments show that our method achieved competitive performance compared to state-of-the-art methods on popular benchmark datasets for image classification: CIFAR-10 and CIFAR-100. Surprisingly, our method enables training deep neural networks without skip-connections using dynamically weighted local loss functions. | - |
dc.language | English | - |
dc.language.iso | en | - |
dc.publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | - |
dc.title | Learning to Balance Local Losses via Meta-Learning | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Kim, Hyunwoo J. | - |
dc.identifier.doi | 10.1109/ACCESS.2021.3113934 | - |
dc.identifier.scopusid | 2-s2.0-85115668921 | - |
dc.identifier.wosid | 000701212300001 | - |
dc.identifier.bibliographicCitation | IEEE ACCESS, v.9, pp.130834 - 130844 | - |
dc.relation.isPartOf | IEEE ACCESS | - |
dc.citation.title | IEEE ACCESS | - |
dc.citation.volume | 9 | - |
dc.citation.startPage | 130834 | - |
dc.citation.endPage | 130844 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.subject.keywordAuthor | Deep learning | - |
dc.subject.keywordAuthor | Deep learning | - |
dc.subject.keywordAuthor | Licenses | - |
dc.subject.keywordAuthor | Loss measurement | - |
dc.subject.keywordAuthor | Neural networks | - |
dc.subject.keywordAuthor | Standards | - |
dc.subject.keywordAuthor | Task analysis | - |
dc.subject.keywordAuthor | Training | - |
dc.subject.keywordAuthor | image classification | - |
dc.subject.keywordAuthor | machine learning | - |
dc.subject.keywordAuthor | meta-learning | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
(02841) 서울특별시 성북구 안암로 14502-3290-1114
COPYRIGHT © 2021 Korea University. All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.