The real building buffer is extracted from the readings of student labs in one the school buildings. The amount of datapoints in the buffers ranges from 170~260K, depends on the number of rooms involved and missing values. We obtain data of an entire year, from the beginning of July 2017 to the end of June 2018 of 15 rooms across 3 floors (for most of the rooms, however, you might find the start and end dates of the entire dataset might be earlier and later. Due to missing and corrupted data, if we want to collect a similar amount of data for each room, the time period might be different from room to room.). Since the rooms on the same side of a floor often share similar thermal dynamics, we thus create batch data for each floor to ensure that the replay buffer reflects each variable air volume (VAV)’s thermal dynamics precisely.
State: We use the following attributes for the RL process to evaluate the policy: indoor air temperature, actual supply airflow, outside air temperature, and humidity. These states include the features needed for thermal comfort estimation and those that represent the responses of actions as RL states.
Action: We control two important parameters, namely, zone air temperature setpoint and actual supply airflow setpoint.
Reward: We monitor the thermal states of the space as well as the thermal comfort index predicted by a regression model, and then make control decisions with the actions selected by the BRL model. Our reward function penalizes high HVAC energy use and discourages a large absolute value of the thermal comfort index, which indicates discomfort to occupants.
Use case
Automated Control of HVAC (Heating-Ventilation-Air Conditioning) for a University Building
| Example observations | T_in: indoor temperature (°C) T_out: outdoor temperature (°C) T_supply: supply air temperature (°C) RH_in: indoor relative humidity (%) Airflow_actual: measured airflow (CFM) Occupancy_flag: (0/1) Heater_valve_position: (%) Reheat_valve_position: (%) Timestamp: encoded sin/cos (hour, day) |
| Example actions | Airflow_setpoint ∈ [200, 1000] CFM ZoneTemp_setpoint ∈ [20 °C, 24 °C] |
| Example reward | − α*Energy(t) − β*max(|T_in − 22 °C| − 1 °C, 0) − γ*SafetyViolations • α ~ 0.002 (kWh → €) • β ~ 2 °C⁻¹ • γ ~ 5 (strong penalty if setpoint exceeds safety limits) |