Inferring 3D information from a monocular camera

飞行无人机正变得无处不在。从自动化邮件递送到军事应用，这些飞行机器人在不久的将来肯定会出现在我们的天空中。虽然它们可以提供有趣的好处，但它们的存在也带来了潜在的风险，从扰乱航空旅行到威胁国家安全。因此，至关重要的是，我们必须开发系统，在无人机以潜在的高速飞行时对其进行检测和跟踪。在这项拨款申请中，我们建议开发用于跟踪高速飞行物体的计算机视觉算法。算法必须对遮挡具有健壮性，与真实世界的传感器一起工作，并适应距离相机不同距离的对象。要做到这一点，我们的关键观察是，对象在图像平面中的外观速度和大小取决于它到相机的距离：对象距离越远，它在图像中出现的速度就越小、越慢(反之亦然)。这使我们能够详细阐述基于三个子目标的研究方法。第一个子目标是跟踪在相机附近飞行的非常高速的对象。为此，我们将依赖基于事件的相机：一种新型的相机，它在发生变化时报告变化，因此速度极快。第二个子目标是以较低的速度但较低的分辨率跟踪远处的对象。在这种情况下，我们将依靠高分辨率摄像头。最后，最后一个分目标将在一个连贯的框架中结合前两个目标。拟议中的项目将在几个方面使加拿大工业受益。特别是，工业合作伙伴泰利斯加拿大确定了至少两个潜在的应用。首先，无人机探测对空中交通管理至关重要。及早发现飞行物体是防止与飞机相撞的关键。其次，有证据表明，恐怖组织对无人机军事化表现出越来越大的兴趣，以便在人口稠密的城市地区运送爆炸性有效载荷。这项研究将能够及早发现城市环境中的此类威胁，从而加强加拿大领土上的安全，并保护其关键基础设施。

Flying drones are becoming ubiquitous. From automated mail delivery to military applications, these flying robots are bound to be present in our skies in the near future. While they can provide interesting benefits, their presence also pose potential risks ranging from the disruption of air travel to national security threats. It is therefore critical that we develop systems to detect and track drones as they are flying at potentially high speed. In this grant application, we propose to develop computer vision algorithms for tracking high speed flying objects. The algorithms must be robust to occlusions, operate with real-world sensors, and adapt to objects at varying distances from the camera. To do so, our key observation is that the apparent speed and size of the object in the image plane depend on its distance to the camera: the farther away the object, the smaller and the slower it will appear in the image (and vice-versa). This allows us to elaborate a research methodology based on three sub-objectives. The first sub-objective is to track objects are very high speed that fly close to the camera. For this, we will rely on an event-based camera: a novel type of camera that reports changes as they occur and are thus extremely fast. The second sub-objective is to track far away objects at lower speed but with lower resolution. In this case, we will rely on high resolution cameras. Finally, the last sub-objective will combine the first two in a coherent framework. The proposed project will benefit the Canadian industry in several ways. In particular, the industrial partner Thales Canada identifies at least two potential applications. First, drone detection is critical for air traffic management. Early detection of flying objects is crucial to prevent collisions with aircrafts. Second, evidence exists that terrorist groups are showing a growing interest in militarizing drones to deliver explosive payloads in densely populated urban areas. This research will enable early detection of such threats in urban context, thus enhancing the Canadian security on its territory and protect its critical infrastructures.