Improving visual surveillance using infrared and visible imagery

用于监控和视频监控目的的人员自动检测是一个非常难以解决的问题。事实上，当与背景的颜色对比较低，场景的照明较差时，在可见（Vis）图像中的人很难被发现。一个可能的解决方案是增加一个红外（IR）摄像机来监控场景。由于红外摄像机可以探测到热量，因此它可以在与可见光摄像机互补的情况下成功地探测到人。******因此，这是有益的使用两种相机类型共同利用各自的优势。在本研究中，长波红外和可见光相机在立体配置中使用，也可以提供粗略的深度信息。平面和非平面场景都被考虑。除了有助于人员检测外，结合IR/Vis的另一个好处是能够检测和分割人类携带的物体，因为它们通常在温度上有强烈的对比。最后，热信息可用于检测发烧等症状。***结合可见光和红外来提高对人的检测需要解决很多问题。在这两种模式中找到匹配的特征来执行注册是特别困难的。即使在配准之后，两个对齐的轮廓（IR/Vis）也不会被完美分割。哪些部分与人类相对应，哪些部分不对应？我的研究目标与这些研究问题相匹配，旨在设计结合红外/可见人体轮廓的解决方案，以有效地检测和跟踪人。我的长期目标将通过实现我的短期目标来实现：***1)开发新的检测和跟踪方法，以促进IR/Vis注册；***2)开发新的方法，提高人物轮廓的IR/Vis配准速度和准确性；***3)开发新的方法，有效地结合对齐的IR/Vis人物轮廓；******为了达到这些目标，将在目标检测、跟踪、配准和分割方面开发新的方法。目标检测是对人进行初始分割的关键。然后使用注册来对齐可见和红外人体轮廓。跟踪允许从一帧到另一帧传播分割和注册结果。最后，需要进行最后的联合分割，将可见光和红外数据正确地结合起来。******该计划的研究成果将为计算机视觉社区带来新颖的想法，以解决实际的现实问题。我的研究小组预计将公开其开发的源代码和数据集，以刺激更广泛地传播给最终用户和其他研究人员。这项研究计划的所有方面都将涉及学生（hqp），他们将接受培训，学习加拿大各地高需求的技能。********

Automatically detecting people for monitoring and video surveillance purposes is a very difficult problem to solve. Indeed, people in visible (Vis) imagery can be hard to detect when the color contrast with the background is low and when the illumination of the scene is poor. A possible solution is to add an infrared (IR) camera to monitor a scene. Because an infrared camera detects heat, it can detect successfully people in situations complementary to those of a visible camera.******As a result, it is beneficial to use both camera types jointly to capitalize on their respective strengths. In this research, long wavelength IR and visible cameras are used in a stereo configuration, which can also provide coarse depth information. Both planar and non-planar scenes are considered. As well as helping in people detection, an additional benefit of combining IR/Vis is the ability to detect and segment objects carried by humans, as they often contrast strongly in temperature. Finally, thermal information can be used to detect symptoms like fever.***Combining visible and IR to improve the detection of people requires solving many problems. It is especially difficult to find features that match in both modalities to perform registration. Even after registration, both aligned silhouettes (IR/Vis) will be imperfectly segmented. Which parts correspond to the human, which parts do not? My research objectives match these research problems and aim at designing solutions for combining IR/Vis human silhouettes for efficient detection and tracking of people. My long-term objective will be achieved through realization of my short-term objectives:***1) Develop novel detection and tracking methods to facilitate IR/Vis registration;***2) Develop novel methods to improve the speed and accuracy of IR/Vis registration of people silhouettes;***3) Develop novel methods to combine efficiently aligned IR/Vis silhouettes of people;******To reach these objectives, new methods will be developed in object detection, tracking, registration and segmentation. Object detection is essential to have initial segmentation of people. Then registration is used to align visible and infrared human silhouettes. Tracking allows propagating segmentation and registration results from frame to frame. Finally, a final joint segmentation needs to be performed to combine properly the visible and infrared data. ******Research resulting from this program will bring novel ideas to the computer vision community to solve practical real-world problems. My research team is expected to make its developed source codes and datasets publicly available to stimulate wider dissemination to end-users and other researchers. All aspects of this research program will involve students (HQPs) that will be trained to learn skills that are in high demand across Canada.********