I-Corps: Determining occupant load and location through machine vision with on-device image processing

I-Corps项目更广泛的影响/商业潜力是开发带有机载处理的智能相机。该技术将被用作建筑和智慧城市管理系统的一部分。建筑供暖、通风和空调（HVAC）占美国总能源使用量的13%，占建筑能源的近40%。准确的占用检测可以将暖通空调系统的能耗降低30%。然而，在评估准确的检测和隐私时，存在占用检测方面的问题。所提出的技术可以解决这些问题，依靠相机的视觉提供精确的信息和设备上的分析，以确保没有图像隐私数据被传输。提出的技术也可用于改善交通灯的规划。行人聚集在十字路口可能会对车辆和人员造成安全问题。拟议的技术允许对人、汽车和自行车进行计数，并将这些信息集成在一起。可以通过提供计数数据进行额外的分析，如果行人计数没有变化，则可能需要修改交通模式或提醒急救人员。这个I-Corps项目是基于嵌入式设备的开发来运行目标检测算法。使用深度神经网络（dnn）进行目标检测涉及大量的计算，这阻碍了其在资源/能源有限的用户端设备上的实现。dnn成功的原因是由于对观察环境的不同领域有了解。然而，在推理时只需要有限的观察环境知识，这可以使用浅神经网络（SHNN）来学习。TKD （Temporal Knowledge Distillation）是一种系统级设计，旨在改善用户端设备上目标检测的能耗。在用户端设备上部署SHNN，用于检测观测环境中的物体。此外，还实现了一种知识转移机制，当对象域发生变化时，利用深度神经网络知识更新SHNN模型。实验表明，与在用户端设备上运行深度模型相比，用户端设备的能耗和推理时间分别提高了78%和71%。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of smart cameras with on-board processing. The proposed technology will be used as a part of building and smart city management systems. Building heating, ventilation and air conditioning (HVAC) accounts for 13% of all energy usage in the United States and nearly 40% of buildings' energy. Accurate occupancy detection may reduce energy use in HVAC systems by as much as 30%. However, there are concerns regarding occupancy detection when assessing accurate detection and privacy. The proposed technology may solve these concerns by relying on the camera's vision to provide precise information and on-device analysis to ensure no image privacy data is transmitted. The proposed technology also may be used to improve traffic light planning. Pedestrians congregating at an intersection may cause safety issues for vehicles and people. The proposed technology allows for the counting of people, cars, and bikes and the integration of this information. Additional analysis may be performed by providing count data and if pedestrian counts are not changing, there may be a need to modify traffic patterns or alert first responders.This I-Corps project is based on the development of embedded devices to run object-detection algorithms. Object detection using deep neural networks (DNNs) involves a large amount of computation, which impedes its implementation on resource/energy-limited, user-end devices. The reason for the success of DNNs is due to having knowledge over different domains of observed environments. However, only a limited knowledge of the observed environment at inference time is required, which may be learned using a shallow neural network (SHNN). The TKD (Temporal Knowledge Distillation) is a system-level design that is proposed to improve the energy consumption of object detection on the user-end device. An SHNN is deployed on the user-end device to detect objects in the observing environment. Also, a knowledge transfer mechanism is implemented to update the SHNN model using the DNN knowledge when there is a change in the object domain. Experiments demonstrate that the user-end device's energy consumption and the inference time can be improved by 78% and 71% compared with running the deep model on the user-end device.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.