RI: Medium: To Sense or Not to Sense: Energy Efficient Adaptive Sensing for Autonomous Systems

传感和计算一直是半自动和全自动车辆和机器人取得重大进展的关键。多种类型传感器(激光雷达、照相机、雷达等)的激增而计算量大、数据需求量大的深度学习方法的出现，极大地提高了自主系统的性能。但各式各样的传感器在性能、成本和操作难度方面存在差异。因此，为特定机器人上的特定任务选择特定的传感器集。这种以马换课的方法经常导致一次性系统无法适应许多任务或机器人。因此，为了确保安全和可靠性，自动驾驶汽车等多任务系统采取了过度工程的方式，任何汽车都有15个以上的传感器和多个GPU/CPU。而且，更糟糕的是，许多检测到的数据最终作为不需要的背景被丢弃。因此，尽管传感和计算的能量消耗正在以惊人的速度增长，但这些系统的灵活性或适应性仍然不足。这种情况在很大程度上可以归因于这样一个事实，即传感器和算法面临着截然不同的硬件和软件挑战，因此在不同的学术单位或行业进行设计、开发和制造。这个项目采取了一种不同的方法：自适应地感知主要(如果不仅仅是)有助于在分配的时间内准确地解决任务的数量。换句话说，该项目倡导在自主系统的学习框架内折叠自适应和灵活的感知。这是通过共同设计和共同执行传感和算法来实现的，以最大限度地提高准确性和灵活性，同时将消耗的能量和成本降至最低。这种方法的动机是人类如何决定什么、哪里、何时以及如何感知，并将其应用于机器人学习框架。研究和教育紧密结合在一个多样化和包容性的环境中。该项目由三个基本研究推动力组成。推力1：开发高度新颖和完全自适应的3D光学传感器的设计和物理实现。这一推力包括一个基本的数学框架，该框架确定了实现手头特定任务的最佳发射和测量射线集。这是开发一种新型传感器的数学基础，这种传感器检测和表征障碍物-任何自主系统的一项时间关键任务-具有最大的能量效率、最小的延迟(即，几乎即时)并且几乎不需要单独计算。推力2：新的决策框架，有效地控制手头任务的自适应传感器。这包括确定何时何地感知并相应调整行为策略。推力3：通过与人类学习和互动来支持机器人学习框架。该项目将使用三个具有广泛社会影响的不同自主系统来展示自适应传感的普遍性：a)自动车辆，b)辅助机器人，c)制造中的机器人。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Sensing and computation have been crucial to the significant progress in semi- and fully-autonomous vehicles and robots. Proliferation of many types of sensors (LIDARs, cameras, RADARs, etc.) and the advent of compute-heavy and data-hungry deep-learning approaches have increased the performance of autonomous systems by leaps and bounds. But the wide variety of sensors differ in terms of their performance, cost, and operational difficulty. Thus, specific sets of sensors are chosen for a particular task on a particular robot. This horses-for-courses approach often results in one-off systems that are incapable of adapting to many tasks or robots. Thus, to ensure safety and reliability, multi-tasking systems like autonomous vehicles have resorted to over-engineering, with upwards of 15 sensors and multiple GPUs/CPUs in any car. And, to make matters worse, many of the sensed data is eventually discarded as unwanted background. Thus, while the energy footprint of sensing and computations is increasing at an alarming rate, the flexibility or adaptability of these systems is still lacking. Much of this state of affairs can be attributed to the fact that sensors and algorithms face vastly different hardware and software challenges and are hence designed, developed, and manufactured in separate academic units or industries. This project takes a different approach: adaptively sense mostly (if not only) quantities which help solve the task accurately and within the allotted time. In other words, this project advocates folding adaptive and flexible sensing within a learning framework for autonomous systems. This is achieved by co-design and co-execution of sensing and algorithms to maximize accuracy and flexibility while minimizing expended energy and cost. The approach is motivated by how humans decide what, where, when, and how to sense and apply that to a robot learning framework. Research and education are closely integrated in a diverse and inclusive environment.The project consists of three fundamental research thrusts. Thrust 1: Development of highly novel and fully adaptive design and physical realization of 3D optical sensors. This thrust includes a fundamental mathematical framework that determines the optimal set of emitted and measured rays to achieve a particular task at hand. This is the mathematical foundation for developing a new class of sensors that detect and characterize obstacles---a time critical task of any autonomous system---with maximum energy efficiency, minimal latency (i.e., near-instantly) and with virtually no separate computation. Thrust 2: Novel decision-making framework that efficiently controls the adaptive sensors for the task at hand. This includes determining where and when to sense and adapting behavior policies accordingly. Thrust 3: Support the robot learning framework by learning and interacting with humans. The project will demonstrate the generality of adaptive sensing using three disparate autonomous systems that have broad societal impact: a) autonomous vehicles, b) assistive robots, and c) robots in manufacturing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.