BioBERT: a pre-trained biomedical language representation model for biomedical text mining
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Jinhyuk | - |
dc.contributor.author | Yoon, Wonjin | - |
dc.contributor.author | Kim, Sungdong | - |
dc.contributor.author | Kim, Donghyeon | - |
dc.contributor.author | Kim, Sunkyu | - |
dc.contributor.author | So, Chan Ho | - |
dc.contributor.author | Kang, Jaewoo | - |
dc.date.accessioned | 2021-08-31T10:12:35Z | - |
dc.date.available | 2021-08-31T10:12:35Z | - |
dc.date.created | 2021-06-19 | - |
dc.date.issued | 2020-02-15 | - |
dc.identifier.issn | 1367-4803 | - |
dc.identifier.uri | https://scholar.korea.ac.kr/handle/2021.sw.korea/57643 | - |
dc.description.abstract | Motivation: Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora. Results: We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. | - |
dc.language | English | - |
dc.language.iso | en | - |
dc.publisher | OXFORD UNIV PRESS | - |
dc.subject | RECOGNITION | - |
dc.subject | CORPUS | - |
dc.title | BioBERT: a pre-trained biomedical language representation model for biomedical text mining | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Kang, Jaewoo | - |
dc.identifier.doi | 10.1093/bioinformatics/btz682 | - |
dc.identifier.scopusid | 2-s2.0-85080840963 | - |
dc.identifier.wosid | 000518528800035 | - |
dc.identifier.bibliographicCitation | BIOINFORMATICS, v.36, no.4, pp.1234 - 1240 | - |
dc.relation.isPartOf | BIOINFORMATICS | - |
dc.citation.title | BIOINFORMATICS | - |
dc.citation.volume | 36 | - |
dc.citation.number | 4 | - |
dc.citation.startPage | 1234 | - |
dc.citation.endPage | 1240 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Biochemistry & Molecular Biology | - |
dc.relation.journalResearchArea | Biotechnology & Applied Microbiology | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Mathematical & Computational Biology | - |
dc.relation.journalResearchArea | Mathematics | - |
dc.relation.journalWebOfScienceCategory | Biochemical Research Methods | - |
dc.relation.journalWebOfScienceCategory | Biotechnology & Applied Microbiology | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Interdisciplinary Applications | - |
dc.relation.journalWebOfScienceCategory | Mathematical & Computational Biology | - |
dc.relation.journalWebOfScienceCategory | Statistics & Probability | - |
dc.subject.keywordPlus | RECOGNITION | - |
dc.subject.keywordPlus | CORPUS | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
(02841) 서울특별시 성북구 안암로 14502-3290-1114
COPYRIGHT © 2021 Korea University. All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.