ConBERT: A Concatenation of Bidirectional Transformers for Standardization of Operative Reports from Electronic Medical Records

Abstract

This operative report documents the details of a surgery. Standardization of the medical terminology for the operative report written in free text is significant for performing medical research and establishing insurance systems by accurately sharing information on treatment. However, standardization of operative reports is a labor-intensive task that has a risk of induced errors. We have proposed a concatenation of bidirectional encoder representations from transformers (ConBERT) model for predicting the International Classification of Disease-9 code using the operative report and diagnosis recorded in free text to standardize the operative report automatically. We compared the pre-trained models of BERT and character BERT and created a new model by concatenating the combinations of each model. The proposed ConBERT model showed a micro AP score of 0.7672, F1 score of 0.7415, and AUC of 0.9842. In addition, we developed a web-based application to demonstrate the performance of our model and make it publicly accessible.