I-Corps: Determining occupant load and location through machine vision with on-device image processing

这个I-Corps项目更广泛的影响/商业潜力是开发具有机载处理功能的智能相机。该技术将被用作建筑和智慧城市管理系统的一部分。建筑供暖、通风和空调（HVAC）占美国所有能源使用量的13%，占建筑物能源的近40%。准确的占用检测可以将HVAC系统中的能源使用减少多达30%。然而，在评估准确检测和隐私时存在关于占用检测的担忧。所提出的技术可以通过依赖相机的视觉来提供精确的信息和设备上的分析来解决这些问题，以确保没有图像隐私数据被传输。 所提出的技术还可以用于改进交通灯规划。行人聚集在十字路口可能会导致车辆和人员的安全问题。所提出的技术允许对人、汽车和自行车进行计数并整合这些信息。可以通过提供计数数据进行额外的分析，如果行人计数没有变化，可能需要修改交通模式或提醒第一响应者。这个I-Corps项目是基于嵌入式设备的开发来运行物体检测算法。 使用深度神经网络（DNN）进行对象检测涉及大量计算，这阻碍了其在资源/能源有限的用户端设备上的实现。DNN成功的原因是由于对所观察环境的不同领域的了解。然而，仅需要在推理时间对观察到的环境的有限知识，这可以使用浅神经网络（SHNN）来学习。TKD（Temporal Knowledge Distillation）是一种系统级设计，旨在改善用户端设备上目标检测的能耗。SHNN部署在用户端设备上以检测观察环境中的对象。此外，知识转移机制被实现为当对象域中存在变化时使用DNN知识来更新SHNN模型。实验表明，与在用户端设备上运行深度模型相比，用户端设备的能耗和推理时间分别提高了78%和71%。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of smart cameras with on-board processing. The proposed technology will be used as a part of building and smart city management systems. Building heating, ventilation and air conditioning (HVAC) accounts for 13% of all energy usage in the United States and nearly 40% of buildings' energy. Accurate occupancy detection may reduce energy use in HVAC systems by as much as 30%. However, there are concerns regarding occupancy detection when assessing accurate detection and privacy. The proposed technology may solve these concerns by relying on the camera's vision to provide precise information and on-device analysis to ensure no image privacy data is transmitted. The proposed technology also may be used to improve traffic light planning. Pedestrians congregating at an intersection may cause safety issues for vehicles and people. The proposed technology allows for the counting of people, cars, and bikes and the integration of this information. Additional analysis may be performed by providing count data and if pedestrian counts are not changing, there may be a need to modify traffic patterns or alert first responders.This I-Corps project is based on the development of embedded devices to run object-detection algorithms. Object detection using deep neural networks (DNNs) involves a large amount of computation, which impedes its implementation on resource/energy-limited, user-end devices. The reason for the success of DNNs is due to having knowledge over different domains of observed environments. However, only a limited knowledge of the observed environment at inference time is required, which may be learned using a shallow neural network (SHNN). The TKD (Temporal Knowledge Distillation) is a system-level design that is proposed to improve the energy consumption of object detection on the user-end device. An SHNN is deployed on the user-end device to detect objects in the observing environment. Also, a knowledge transfer mechanism is implemented to update the SHNN model using the DNN knowledge when there is a change in the object domain. Experiments demonstrate that the user-end device's energy consumption and the inference time can be improved by 78% and 71% compared with running the deep model on the user-end device.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.