Improving visual surveillance using infrared and visible imagery

为了监控和视频监控的目的，自动检测人员是一个非常难解决的问题。事实上，在可见光(VIS)图像中，当与背景的颜色对比度较低以及场景的照明较差时，可能很难检测到人。一种可能的解决方案是增加一个红外(IR)摄像机来监控场景。因为红外摄像机可以探测热量，所以它可以成功地探测处于与可见光摄像机互补的情况下的人。因此，联合使用这两种类型的摄像机是有益的，以发挥各自的优势。在这项研究中，长波红外和可见光摄像机用于立体配置，也可以提供粗略的深度信息。同时考虑了平面场景和非平面场景。除了有助于检测人员，红外/可见光组合的另一个好处是能够检测和分割人类携带的物体，因为它们经常在温度上形成强烈对比。最后，热信息可以用来检测发烧等症状。*结合可见光和红外来提高对人的检测需要解决许多问题。特别难找到在两种模式中都匹配的特征来执行配准。即使在配准之后，两个对齐的轮廓(IR/VIS)也将被不完美地分割。哪些部位对应于人类，哪些部位不对应？我的研究目标与这些研究问题相匹配，旨在设计结合红外/可见光人体轮廓的解决方案，以有效地检测和跟踪人。我的长期目标将通过实现我的短期目标来实现：*1)开发新的检测和跟踪方法来促进IR/VIS配准；*2)开发新的方法来提高人的轮廓的IR/VIS配准的速度和准确性；*3)开发新的方法来有效地组合对齐的IR/VIS人的轮廓；*为了实现这些目标，将在对象检测、跟踪、配准和分割方面开发新的方法。目标检测是对人进行初始分割的关键。然后利用配准方法对可见光和红外人体轮廓进行配准。跟踪允许将分割和配准结果从一帧传播到另一帧。最后，需要进行最终的关节分割，以适当地结合可见光和红外数据。*该项目的研究将为计算机视觉领域解决实际问题带来新的想法。预计我的研究团队将公开其开发的源代码和数据集，以刺激对最终用户和其他研究人员的更广泛传播。该研究计划的所有方面都将涉及学生(HQP)，他们将接受培训，学习加拿大各地非常需要的技能。

Automatically detecting people for monitoring and video surveillance purposes is a very difficult problem to solve. Indeed, people in visible (Vis) imagery can be hard to detect when the color contrast with the background is low and when the illumination of the scene is poor. A possible solution is to add an infrared (IR) camera to monitor a scene. Because an infrared camera detects heat, it can detect successfully people in situations complementary to those of a visible camera.******As a result, it is beneficial to use both camera types jointly to capitalize on their respective strengths. In this research, long wavelength IR and visible cameras are used in a stereo configuration, which can also provide coarse depth information. Both planar and non-planar scenes are considered. As well as helping in people detection, an additional benefit of combining IR/Vis is the ability to detect and segment objects carried by humans, as they often contrast strongly in temperature. Finally, thermal information can be used to detect symptoms like fever.***Combining visible and IR to improve the detection of people requires solving many problems. It is especially difficult to find features that match in both modalities to perform registration. Even after registration, both aligned silhouettes (IR/Vis) will be imperfectly segmented. Which parts correspond to the human, which parts do not? My research objectives match these research problems and aim at designing solutions for combining IR/Vis human silhouettes for efficient detection and tracking of people. My long-term objective will be achieved through realization of my short-term objectives:***1) Develop novel detection and tracking methods to facilitate IR/Vis registration;***2) Develop novel methods to improve the speed and accuracy of IR/Vis registration of people silhouettes;***3) Develop novel methods to combine efficiently aligned IR/Vis silhouettes of people;******To reach these objectives, new methods will be developed in object detection, tracking, registration and segmentation. Object detection is essential to have initial segmentation of people. Then registration is used to align visible and infrared human silhouettes. Tracking allows propagating segmentation and registration results from frame to frame. Finally, a final joint segmentation needs to be performed to combine properly the visible and infrared data. ******Research resulting from this program will bring novel ideas to the computer vision community to solve practical real-world problems. My research team is expected to make its developed source codes and datasets publicly available to stimulate wider dissemination to end-users and other researchers. All aspects of this research program will involve students (HQPs) that will be trained to learn skills that are in high demand across Canada.********