S&AS: FND: Context-Aware Active Data Gathering for Complex Outdoor Environments

传统的智能体被编程为通过识别和关注给定环境中的预定区域和目标来获取信息。深度学习模型和微型硬件平台的最新进展为人工智能提供了前所未有的处理和解释视觉数据的能力。这些进步为构建具有更大自主性和适应性的智能机器创造了令人兴奋的机会。为了实现这一目标，该项目研究了新的方法，使多个无人机系统能够通过主动寻找，获取，整合和处理跨空间和时间的视觉信息来理解和探索复杂的户外环境。所开发的具有增强的适应性、自我意识和通用性的框架将适用于广泛应用的自主系统，如环境监测、搜索和救援、自动驾驶汽车、智能健康和制造领域。在整个项目中，主要研究人员将公开项目结果，包括创建的数据集、训练的模型、代码和论文。结合视觉，规划和驱动的新的综合研究将被纳入教材，代表性不足和本科生研究项目，以及K-12推广活动。该项目旨在开发上下文感知主动传感算法，其中还包括能源约束。这将通过以下方式实现：首先，提出新的深度学习模型，用于多个鸟瞰图的整体注意力预测。这些模型将利用外部知识，在看不见的背景下进行推理和概括。第二，通过开发新的视图和路径规划方法，这些方法是有效的，并且知道系统的能量、移动性和传感约束。第三，通过提供新的在线学习方法，适应基于不确定性，以实现适应性和意识到不断变化的环境。验证这些发现的实验将在明尼苏达大学新装修的牧羊人无人机实验室的室内和雪松溪生态系统保护区的野外进行。该项目的成果有可能激发对智能和综合感知、规划和驱动系统的进一步研究。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Traditional agents are programmed to acquire information by recognizing and attending to predetermined areas and targets in a given environment. Recent advances in deep learning models and miniature hardware platforms are providing artificial agents unprecedented capability in processing and interpreting visual data. These advancements create an exciting opportunity to build intelligent machines running with greater autonomy and adaptability. Toward this goal, this project investigates new methods that enable multiple unmanned aerial systems to understand and explore complex outdoor environment by actively seeking, acquiring, integrating, and processing visual information across space and time. The developed framework with enhanced adaptability, self-awareness, and generalizability will be applicable to autonomous systems in broad applications such as environmental monitoring, search and rescue, self-driving cars, smart health, and manufacturing domains. Throughout the project, the principal investigators will make project results including created datasets, trained models, code, and papers publicly available. The new integrative research combining vision, planning and actuation will be incorporated into teaching materials, underrepresented and undergraduate research projects, as well as K-12 outreach activities.The project seeks to develop algorithms for context-aware active sensing which also incorporate energy constraints. This will be achieved by: First, proposing new deep learning models for holistic attention prediction with multiple aerial views. The models will leverage external knowledge to enable inference and generalization in unseen contexts. Second, by developing new view and path planning methods that are efficient and aware of systems' energy, mobility and sensing constraints. Third, by contributing novel online learning methods that adapt based on uncertainty to implement adaptiveness and awareness to changing environment. Experiments to validate the findings will take place both indoors in the newly-renovated Shepherd UAV Lab at the University of Minnesota and in the field at the Cedar Creek Ecosystem Reserve. The results of this project have the potential to inspire further research into intelligent and integrative perceptual, planning and actuation systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.