Detailed Information

Cited 0 time in webofscience Cited 0 time in scopus
Metadata Downloads

Characterizing water quality and quantity profiles with poor quality data in a machine learning algorithm

Authors
Kim, ZhonghyunJeong, HeewonShin, SoraJung, JinhoKim, Joon HaKi, Seo Jin
Issue Date
4월-2020
Publisher
DESALINATION PUBL
Keywords
Non-linear data analysis; Dimensionality reduction; Water quality data; Pollution hotspots; Incomplete data; Self-organizing map
Citation
DESALINATION AND WATER TREATMENT, v.182, pp.127 - 134
Indexed
SCIE
SCOPUS
Journal Title
DESALINATION AND WATER TREATMENT
Volume
182
Start Page
127
End Page
134
URI
https://scholar.korea.ac.kr/handle/2021.sw.korea/56855
DOI
10.5004/dwt.2020.25481
ISSN
1944-3994
Abstract
Statistical analyses are often subject to misinterpretation due to poor data quality which is inaccurate, incomplete, or unavailable. This study describes how incomplete data diminishes the screening accuracy of water pollution hotspots using a self-organizing map (SOM), a popular algorithm in reducing the dimension of complex data in a nonlinear fashion. A full data set consisting of 12 water quality and quantity parameters monitored monthly over 3 years at the Yeongsan River in Korea was provided to SOM as a reference input. For purposes of comparison, SOM was further allowed to accept three incomplete data sets in terms of variable availability as well as data loss for single and multiple parameters and different pollution levels. We found that data loss of either single or multiple parameters exceeding 15% of the entire data set led to significant changes in spatial and temporal patterns of the original data. However, the variables intentionally unavailable in the given data set affected the screening performance of water pollution hotspots in SOM, to a less obvious extent, as long as the percentage of missing data fell below 10%. The same applied to data loss with three pollution levels, from high through moderate to low concentrations of one important variable. Therefore, we recommend the use of multiple approaches that couple dimensionality reduction algorithms with reasonable imputation methods for the data set with a high percentage (e.g. above 15%) of missing values.
Files in This Item
There are no files associated with this item.
Appears in
Collections
College of Life Sciences and Biotechnology > Division of Environmental Science and Ecological Engineering > 1. Journal Articles

qrcode

Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Researcher JUNG, Jin ho photo

JUNG, Jin ho
생명과학대학 (환경생태공학부)
Read more

Altmetrics

Total Views & Downloads

BROWSE