A new measure for assessing the efficiency of hydrological data-driven forecasting models

Abstract

There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the naive model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966-0.713 at correlation coefficients of 0.977-0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943-0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1% to 37.8% according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to naive models in data-driven forecasting.