Improving visual surveillance using infrared and visible imagery

自动检测用于监控和视频监控目的的人是一个非常难以解决的问题。实际上，当与背景的颜色对比度低时以及当场景的照明差时，可见（维斯）图像中的人可能难以检测。一种可能的解决方案是添加红外（IR）相机来监视场景。因为红外摄像机可以探测到热量，所以它可以在与可见光摄像机互补的情况下成功地探测到人。因此，联合使用这两种相机类型以利用其各自的优势是有益的。在这项研究中，长波长红外和可见光相机被用于立体配置，这也可以提供粗略的深度信息。平面和非平面的场景被认为是。除了有助于人员检测外，结合IR/维斯的另一个好处是能够检测和分割人类携带的物体，因为它们通常在温度上形成强烈对比。最后，热信息可以用来检测发烧等症状。结合可见光和红外来提高对人的检测需要解决许多问题。尤其难以找到在两种模态中匹配的特征来执行配准。即使在配准之后，两个对齐的轮廓（IR/维斯）也将被不完美地分割。哪些部分符合人类，哪些部分不符合？我的研究目标与这些研究问题相匹配，旨在设计结合IR/维斯人体轮廓的解决方案，以有效地检测和跟踪人。我的长期目标将通过实现我的短期目标来实现：*1）开发新的检测和跟踪方法，以促进IR/维斯配准;*2）开发新的方法，以提高IR/维斯配准人体轮廓的速度和准确性;*3）开发新的方法，以有效地联合收割机对齐的IR/维斯轮廓;** 为了实现这些目标，将在目标检测、跟踪、配准和分割方面开发新方法。目标检测对于人的初始分割至关重要。然后，配准用于对准可见光和红外人体轮廓。跟踪允许从帧到帧传播分割和配准结果。最后，需要执行最终的联合分割以适当地联合收割机组合可见光和红外数据。** 该计划的研究成果将为计算机视觉社区带来新的想法，以解决实际的现实问题。我的研究团队预计将公开其开发的源代码和数据集，以促进向最终用户和其他研究人员更广泛的传播。这项研究计划的各个方面将涉及学生（HQP），他们将接受培训，学习加拿大各地高需求的技能。

Automatically detecting people for monitoring and video surveillance purposes is a very difficult problem to solve. Indeed, people in visible (Vis) imagery can be hard to detect when the color contrast with the background is low and when the illumination of the scene is poor. A possible solution is to add an infrared (IR) camera to monitor a scene. Because an infrared camera detects heat, it can detect successfully people in situations complementary to those of a visible camera.******As a result, it is beneficial to use both camera types jointly to capitalize on their respective strengths. In this research, long wavelength IR and visible cameras are used in a stereo configuration, which can also provide coarse depth information. Both planar and non-planar scenes are considered. As well as helping in people detection, an additional benefit of combining IR/Vis is the ability to detect and segment objects carried by humans, as they often contrast strongly in temperature. Finally, thermal information can be used to detect symptoms like fever.***Combining visible and IR to improve the detection of people requires solving many problems. It is especially difficult to find features that match in both modalities to perform registration. Even after registration, both aligned silhouettes (IR/Vis) will be imperfectly segmented. Which parts correspond to the human, which parts do not? My research objectives match these research problems and aim at designing solutions for combining IR/Vis human silhouettes for efficient detection and tracking of people. My long-term objective will be achieved through realization of my short-term objectives:***1) Develop novel detection and tracking methods to facilitate IR/Vis registration;***2) Develop novel methods to improve the speed and accuracy of IR/Vis registration of people silhouettes;***3) Develop novel methods to combine efficiently aligned IR/Vis silhouettes of people;******To reach these objectives, new methods will be developed in object detection, tracking, registration and segmentation. Object detection is essential to have initial segmentation of people. Then registration is used to align visible and infrared human silhouettes. Tracking allows propagating segmentation and registration results from frame to frame. Finally, a final joint segmentation needs to be performed to combine properly the visible and infrared data. ******Research resulting from this program will bring novel ideas to the computer vision community to solve practical real-world problems. My research team is expected to make its developed source codes and datasets publicly available to stimulate wider dissemination to end-users and other researchers. All aspects of this research program will involve students (HQPs) that will be trained to learn skills that are in high demand across Canada.********