Energy efficient operation optimization of building air-conditioners via simulator-assisted asynchronous reinforcement learning

At present, buildings are the major energy consumers in most developed and developing countries. Energy efficiency is one of the primary objectives of today's building projects, especially in the context of carbon-peak and carbon-neutral. In buildings, air-conditioning systems account for nearly half of total building energy consumption. Optimal control and operation can significantly improve air-conditioning's energy efficiency but the key challenge is the complex dynamics of air-conditioning systems. This study focuses the energy efficient operation of air-conditioners in office buildings, and proposes a simulation-assisted reinforcement learning (RL) method to develop energy efficient operation strategies. This study uses a whole building energy simulation model to build the environmental simulator for the air-conditioning system, and uses asynchronous RL algorithm to learn energy efficient operation strategies. Energy saving and Setpoint Not Met is used as the optimization objective as well as the criteria for evaluating the performance of the RL operation strategies. A validation simulator with varied weather conditions is also built to validate the robustness of reinforcement learning. The results show that, compared to common rule-based control strategy, 16.1% energy saving with better thermal comfort can be achieved by the RL operation strategy. In addition, the results also show the RL operation strategy has a certain level of robustness.