A Flexible Distributed Building Simulator for Federated Reinforcement Learning
A Flexible Distributed Building Simulator for Federated Reinforcement Learning
复制标题
DOI：
10.1109/coins54846.2022.9855000
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
Shugo Fujimura;Koki Fujita;Yuwei Sun;H. Esaki;H. Ochiai
中科院分区：
文献类型：
--
作者：
Shugo Fujimura;Koki Fujita;Yuwei Sun;H. Esaki;H. Ochiai
近年来，为了实现可持续发展的社会，利用强化学习进行建筑控制的研究正在进行。此类建筑物控制方法通常使用从建筑物中的传感器和其他来源获取的数据，但此类数据可能包含人员流动等敏感信息，并且在云中收集这些数据进行模型训练可能会在隐私和安全方面存在问题。通过联邦学习，可以在保护这些数据的同时共享和训练大量建筑物的模型。然而，在实际运行真实建筑物时，几乎不可能使用联邦学习进行模型训练实验。有一些关于建筑模拟器的研究，但没有一个是为同时操作大量建筑而设计的，就像联邦学习的情况一样。因此，我们提出了一种用于联合强化学习的分布式建筑模拟器。该模拟器既可扩展又灵活，允许用户进行扩展到多台机器的实验，而无需编写复杂的代码进行分布式处理。在本文中，我们描述了该模拟器的设计和实现，并证明它可以以可扩展的方式对大量建筑物（至少1024个建筑物）进行实验。
Recently, researches on building control using reinforcement learning are underway to realize a sustainable society. Such building control methods typically use data acquired from sensors and other sources in the building, but such data may contain sensitive information such as human flow, and collecting these in the Cloud for model training can be problematic in terms of privacy and security. With federated learning, it is possible to share and train models for large numbers of buildings while protecting such data. However, it is almost impossible to conduct a model training experiment with federated learning while actually running real buildings. There are several studies of building simulators, but none of them are designed for simultaneous operation of large numbers of buildings, as is the case with federated learning. Therefore, we propose a distributed building simulator for federated reinforcement learning. This simulator is both scalable and flexible, allowing users to conduct experiments scaled to multiple machines without writing complex code for distributed processing. In this paper, we describe the design and implementation of this simulator and demonstrate that it can perform experiments with a large number of buildings, at least 1024 buildings, in a scalable manner.