Multi-agent Reinforcement Learning applied to Heating, Ventilation, and Air Conditioning in a Building Energy Management System

Abstract

[eng] The EU aims to be climate-neutral by 2050, focusing on promoting renewable sources and energy efficiency. As of 2021 it is required that all the new buildings consume very low net energy (Nearly Zero-Energy Building, NZEB). In order to support this, improvement over HVAC systems control and predictive models for thermal conditions are pointed out as key factors. Building Energy Management Systems (BEMS) are an implementation of such systems that are gaining interest from the authorities. This master thesis presents the case of study of a new office building in Aarhus, Denmark, where the BEMS will be tested, and focuses on the design and implementation of a proposed controller for the heating and ventilation, using two abstractions of the building—called dev and test—on a building energy simulator, Energyplus. The contribution of this thesis is two-fold. First, it presents a state-of-the-art integration between Energyplus and a standard interface used in Reinforcement Learning problems, OpenAI Gym. Second, it develops a high-level decentralized controller using Multi-agent Reinforcement Learning (MARL) to actuate individual room setpoint temperatures and fans mass airflows. The system is trained using the previous integrated simulation tool, and can be deployed to the framed building. Comparison to a baseline rule-based controller shows it is possible to achieve both energy savings and improved thermal comfort, with an acceptable air quality, and that there is a Pareto frontier of optimal choices in the trade-off between these conflicting goals. It is also observed that the trained controllers on the dev building abstraction are able to perform well on the test building too, meaning they can adapt to different building configurations.