TASCC: Pervasive low-TeraHz and Video Sensing for Car Autonomy and Driver Assistance (PATH CAD)

该项目将新型低太赫兹（LTHz）传感器开发与先进的视频分析、融合和交叉学习相结合。利用集成在现代汽车的传感、信息和控制系统中的两种流，我们的目标是绘制地形地图，识别各种天气下的坑洼和表面纹理变化等危险，并检测和分类其他道路使用者（行人、汽车、骑自行车的人等）。即将到来的自动驾驶和辅助驾驶时代需要新的全天候技术。感知和处理的数据、控制系统和驾驶员之间交互的先进概念可以实现自主决策和控制，确保驾驶员在紧急情况下进行干预所需的所有信息。其目的是通过提高态势感知来改善道路安全，并通过减少因控制不善和资源利用而导致的污染物排放来提高能源效率，无论是在公路上还是在非公路上。视频摄像机仍然是我们系统的核心：这有很多原因：低成本、可用性、高分辨率、大量遗留的处理算法来解释数据以及驾驶员/乘客对输出的熟悉程度。然而，人们普遍认为，视频和/或其他光学传感器，如激光雷达（c.f谷歌car）是不够的。大雨、雾、喷雾、雪和灰尘等挑战人类驾驶员的条件限制了光电传感器的能力。我们需要一种新的方法。关键的第二种传感器模式是在0.3-1太赫兹频谱内工作的低太赫兹雷达系统。就其本质而言，雷达在限制视频的条件下是稳健的。然而，相对于现有的汽车雷达系统，这种LTHz雷达的波长相对较短，带宽较宽，可以带来关键的额外功能。这种雷达有潜力提供：(i)比传统雷达提供的图像更接近熟悉的视频，因此可以开始利用图像处理算法的大量遗产；（ii）大大提高了马路对面的图像分辨率，从而相应地显著改善了车辆、行人和其他“行动者”（骑自行车的人、动物等）的检测和分类；（iii）可以突出显示物体并作为制导和控制系统输入的3D图像；（iv）分析有助于分类和控制的雷达图像特征，例如阴影和图像纹理。该项目是三所学术机构之间的合作——伯明翰大学在汽车雷达研究和雷达技术方面长期保持卓越，爱丁堡大学在信号处理和雷达成像方面拥有世界级的专业知识，赫瑞瓦特大学在视频分析、激光雷达和加速算法方面拥有同等的技能。这种新方法将基于交叉学习认知过程中视频和雷达图像的融合，以提高在全天候、全地形道路条件下运行的外部传感系统获得的信息的可靠性和质量，而不依赖于导航辅助系统。

This project combines novel low-THz (LTHz) sensor development with advanced video analysis, fusion and cross learning. Using the two streams integrated within the sensing, information and control systems of a modern automobile, we aim to map terrain and identify hazards such as potholes and surface texture changes in all weathers, and to detect and classify other road users (pedestrians, car, cyclists etc.). The coming era of autonomous and assisted driving necessitates new all-weather technology. Advanced concepts of interaction between the sensed and processed data, the control systems and the driver can lead to autonomy in decision and control, securing all the needed information for the driver to intervene in critical situations. The aims are to improve road safety through increased situational awareness, and increase energy efficiency by reducing the emission of pollutants caused by poor control and resource use in both on and off-road vehicles. Video cameras remain at the heart of our system: there are many reasons for this: low cost, availability, high resolution, a large legacy of processing algorithms to interpret the data and driver/passenger familiarity with the output. However it is widely recognized that video and/or other optical sensors such as LIDAR (c.f. Google car) are not sufficient. The same conditions that challenge human drivers such as heavy rain, fog, spray, snow and dust limit the capability of electro-optical sensors. We require a new approach.The key second sensor modality is a low-THz radar system operating within the 0.3-1 THz frequency spectrum. By its very nature radar is robust to the conditions that limit video. However it is the relatively short wavelength and wide bandwidth of this LTHz radar with respect to existing automotive radar systems that can bring key additional capabilities. This radar has the potential to provide: (i) imagery that is closer to familiar video than those provided by a conventional radar, and hence can begin to exploit the vast legacy of image processing algorithms; (ii) significantly improved across-road image resolution leading to correspondingly significant improvements in vehicle, pedestrian and other 'actor' (cyclists, animals etc.) detection and classification; (iii) 3D images that can highlight objects and act as an input to the guidance and control system; (iv) analysis of the radar image features, such as shadows and image texture that will contribute to both classification and control. The project is a collaboration between three academic institutions - the University of Birmingham with its long standing excellence in automotive radar research and radar technologies, the University of Edinburgh with world class expertise in signal processing and radar imaging and Heriot-Watt University with equivalent skill in video analytics, LiDAR and accelerated algorithms. The novel approach will be based on a fusion of video and radar images in a cross-learning cognitive process to improve the reliability and quality of information acquired by an external sensing system operating in all-weather, all-terrain road conditions without dependency on navigation assisting systems.