Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning

Zhiang Zhang, Adrian Chong, Yuqi Pan, Chenlu Zhang, Khee Poh Lam

Research output: Journal PublicationArticlepeer-review

165 Citations (Scopus)


Whole building energy model (BEM) is a physics-based modeling method for building energy simulation. It has been widely used in the building industry for code compliance, building design optimization, retrofit analysis, and other uses. Recent research also indicates its strong potential for the control of heating, ventilation and air-conditioning (HVAC) systems. However, its high-order nature and slow computational speed limit its practical application in real-time HVAC optimal control. Therefore, this study proposes a practical control framework (named BEM-DRL) that is based on deep reinforcement learning. The framework is implemented and assessed in a novel radiant heating system in an existing office building as a case study. The complete implementation process is presented in this study, including: building energy modeling for the novel heating system, multi-objective BEM calibration using the Bayesian method and the Genetic Algorithm, deep reinforcement learning training and simulation results evaluation, and control deployment. By analyzing the real-life control deployment data, it is found that BEM-DRL achieves 16.7% heating demand reduction with more than 95% probability compared to the old rule-based control. However, the framework still faces the practical challenges including building energy modeling of novel HVAC systems and multi-objective model calibration. Systematic study is also needed for the design of deep reinforcement learning training to provide a guideline for practitioners.

Original languageEnglish
Pages (from-to)472-490
Number of pages19
JournalEnergy and Buildings
Publication statusPublished - 15 Sept 2019
Externally publishedYes


  • Deep reinforcement learning
  • Energy efficiency
  • HVAC
  • Optimal control
  • Whole building energy model

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Building and Construction
  • Mechanical Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning'. Together they form a unique fingerprint.

Cite this