Automatic Stop Word Generation for Mining Software Artifact Using Topic Model with Pointwise Mutual Information
- Authors
- Lee, Jung-Been; Lee, Taek; In, Hoh Peter
- Issue Date
- 9월-2019
- Publisher
- IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG
- Keywords
- text mining; software artifact; stop words; topic modeling; Pointwise Mutual Information (PMI)
- Citation
- IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, v.E102D, no.9, pp.1761 - 1772
- Indexed
- SCIE
SCOPUS
- Journal Title
- IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS
- Volume
- E102D
- Number
- 9
- Start Page
- 1761
- End Page
- 1772
- URI
- https://scholar.korea.ac.kr/handle/2021.sw.korea/63057
- DOI
- 10.1587/transinf.2018EDP7390
- ISSN
- 1745-1361
- Abstract
- Mining software artifacts is a useful way to understand the source code of software projects. Topic modeling in particular has been widely used to discover meaningful information from software artifacts. However, software artifacts are unstructured and contain a mix of textual types within the natural text. These software artifact characteristics worsen the performance of topic modeling. Among several natural language preprocessing tasks, removing stop words to reduce meaningless and uninteresting terms is an efficient way to improve the quality of topic models. Although many approaches are used to generate effective stop words, the lists are outdated or too general to apply to mining software artifacts. In addition, the performance of the topic model is sensitive to the datasets used in the training for each approach. To resolve these problems, we propose an automatic stop word generation approach for topic models of software artifacts. By measuring topic coherence among words in the topic using Pointwise Mutual Information (PMI), we added words with a low PMI score to our stop words list for every topic modeling loop. Through our experiment, we proved that our stop words list results in a higher performance of the topic model than lists from other approaches.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Graduate School > Department of Computer Science and Engineering > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.