S&AS: FND: Context-Aware Active Data Gathering for Complex Outdoor Environments

传统的代理被编程为通过识别和关注给定环境中的预定区域和目标来获取信息。深度学习模型和微型硬件平台的最新进展为人工代理提供了前所未有的处理和解释视觉数据的能力。这些进步创造了一个令人兴奋的机会，可以建造具有更大自主性和适应性的智能机器。为此，该项目研究了新的方法，通过主动寻找、获取、集成和处理跨越空间和时间的视觉信息，使多个无人机系统能够理解和探索复杂的室外环境。开发的框架具有增强的适应性、自我意识和普适性，将适用于环境监测、搜救、自动驾驶汽车、智能健康和制造领域等广泛应用的自主系统。在整个项目中，主要调查人员将公开项目成果，包括创建的数据集、训练的模型、代码和论文。这项结合了视觉、规划和驱动的新的综合研究将被纳入教材、代表性不足的本科生研究项目以及K-12推广活动。该项目寻求开发背景感知主动感知的算法，其中也包括能源限制。这将通过以下几个方面来实现：第一，提出新的深度学习模型，用于多个鸟瞰的整体注意力预测。这些模型将利用外部知识在看不见的环境中进行推理和概括。第二，通过开发新的视图和路径规划方法，这些方法是高效的，并意识到系统的能量、移动性和感知限制。第三，通过提供基于不确定性适应的创新在线学习方法，实现对不断变化的环境的适应性和感知。验证这一发现的实验将在明尼苏达大学新装修的牧羊人无人机实验室和雪松溪生态系统保护区的野外进行。该项目的成果有可能激发对智能和一体化感知、规划和驱动系统的进一步研究。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Traditional agents are programmed to acquire information by recognizing and attending to predetermined areas and targets in a given environment. Recent advances in deep learning models and miniature hardware platforms are providing artificial agents unprecedented capability in processing and interpreting visual data. These advancements create an exciting opportunity to build intelligent machines running with greater autonomy and adaptability. Toward this goal, this project investigates new methods that enable multiple unmanned aerial systems to understand and explore complex outdoor environment by actively seeking, acquiring, integrating, and processing visual information across space and time. The developed framework with enhanced adaptability, self-awareness, and generalizability will be applicable to autonomous systems in broad applications such as environmental monitoring, search and rescue, self-driving cars, smart health, and manufacturing domains. Throughout the project, the principal investigators will make project results including created datasets, trained models, code, and papers publicly available. The new integrative research combining vision, planning and actuation will be incorporated into teaching materials, underrepresented and undergraduate research projects, as well as K-12 outreach activities.The project seeks to develop algorithms for context-aware active sensing which also incorporate energy constraints. This will be achieved by: First, proposing new deep learning models for holistic attention prediction with multiple aerial views. The models will leverage external knowledge to enable inference and generalization in unseen contexts. Second, by developing new view and path planning methods that are efficient and aware of systems' energy, mobility and sensing constraints. Third, by contributing novel online learning methods that adapt based on uncertainty to implement adaptiveness and awareness to changing environment. Experiments to validate the findings will take place both indoors in the newly-renovated Shepherd UAV Lab at the University of Minnesota and in the field at the Cedar Creek Ecosystem Reserve. The results of this project have the potential to inspire further research into intelligent and integrative perceptual, planning and actuation systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.