Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model

초록

Machine learning (ML) approaches for predicting future citations of patents using textual information are valuable for early screening of potential breakthrough technologies. However, their practicality is often limited by the opaque nature of ML models and hierarchical structures of patent documents. This study proposes a patent-specific hierarchical attention network (PatenHAN) designed to enhance interpretability at the technological factor level using patent claim information. Central to this approach are (1) a patent-specific pre-trained language model (PLM) that effectively captures the semantics of patent claims, (2) a hierarchical network architecture that reflects the structure of patent claims representing the technological factors and scope of the invention, and (3) a claim-wise self-attention mechanism that enables the interpretation of individual claims' influence on predictions by revealing pivotal claims with high attention scores. A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of the proposed approach in early screening for potential breakthrough technologies, achieving enhanced interpretability and notable performance, including an accuracy of 91.7 % and a Matthews correlation coefficient of 0.293. Additional analyses using various PLMs and claim types confirm the robust performance of PatenHAN and provide practical insights into its implementation. The proposed approach is expected to serve as a useful complementary tool for the early screening of potential breakthrough technologies, facilitating expert-machine collaborations.

키워드