Tunnel ventilation controller design using an RLS-based natural actor-critic algorithm

Abstract

Tunnel ventilation systems provide drivers with a comfortable and safe driving environment by generating sufficient airflow and by diluting the concentration of noxious contaminants below an acceptable level. For that purpose, tunnel ventilation systems contain mechanical equipment such as jet-fans, blowers and dust collectors. These machines consume large amount of energy, therefore, it is necessary to have an efficient operating algorithm for tunnel ventilation in terms of energy savings and safe driving. In this paper, a new reinforcement learning (RL) method is applied as the control algorithm. In the process of formulating the reward of the tunnel ventilation system, which is a performance index to be maximized in the RL methodology, the following two objectives are of great interest: maintaining an adequate level of pollutants and minimizing power consumption. The RL control algorithm adopted in this research is based on an actor-critic architecture and natural gradient method. Due to its ability to achieve the truly steepest direction of gradients, the natural gradient method can be a promising route to improving the efficacy of the actor module. Also, the recursive least-squares (RLS) method is employed to the critic module in order to improve the efficiency by which data is used. Using actual data collected from an existing tunnel ventilation system, extensive simulation studies were performed. It was confirmed that the suggested algorithm achieved the desired control goals and, when compared to previously developed RL-based control algorithms, improved the performance considerably.