Systematic evaluation of deep Q-networks for multi-chiller system reinforcement learning control

초록

The building sector accounts for a significant share of global greenhouse gas emissions, with HVAC systems, particularly those employing multi-chiller configurations, serving as major energy consumers. This study systematically evaluates the performance of Deep Q-Networks (DQN) for multi-chiller control through co-simulation with EnergyPlus. The analysis covers four aspects: hyperparameter optimization, baseline DQN performance, evaluation of control timesteps, and assessment under diverse climate scenarios. An optimal set of hyper-parameters (learning rate 0.01, discount factor 0.8, and exploration decay 0.9999) enabled stable convergence and measurable energy savings. Compared to the conventional Uniform Load (UL) method, the DQN approach achieved up to 7.1% reduction by operating chillers in efficient part-load ratio ranges. In addition, despite unexpected constraints, the model autonomously learned from data to achieve stable and efficient control, delivering additional savings compared to UL and SL, maximizing COP under low-load conditions, and substantially reducing inefficient operating hours. Shortening the control timestep from 60 to 10 min yielded an additional 1% of savings but increased computation time by over 300%, highlighting the trade-off between precision and cost. Across climate zones, the framework delivered savings ranging from 2.0% to 17.4%, demonstrating better performance under low-load conditions. Moreover, model trained in 4A climate zone generalized effectively to others, with performance differences within 11.8% of locally trained models, demonstrating strong transferability. Overall, this study illustrates the performance of DQN and underscores its key characteristics across different scenarios, providing insights into effective parameter choices and achievable performance. Ultimately, it establishes a practical evaluation framework that advances reinforcement learning design and supports its real-world deployment in HVAC energy control.

키워드