FRR: Towards Robust and Perceptual Inclusive Mobile Robots

随着智能移动的原型系统，从自动驾驶汽车到送货机器人，从受控的开发实验室进入现实世界，它们对残疾人的影响变得显而易见。一个智能系统如果不能考虑到个体之间的不同反应和移动特性，可能会产生可怕的后果。例如，送货机器人可能会无意中导致安全关键场景，因为它会以阻碍轮椅使用者的方式停止。在复杂的城市场景中与盲人一起进行无碰撞导航取决于系统考虑与非视觉推理和手杖移动策略相关的因素的能力，以便对未来进行精确的预测。 然而，目前的自主系统无法有效区分移动辅助设备和需求，也无法对各种情况下的无障碍影响进行推理，从楼梯到道路布局和环境条件。为了教系统导航，同时安全地与残疾人互动，这个项目的目标是开发全面的感知，理解和决策能力的可访问性和需求感知的交互式移动的系统。此外，通过在包容性移动的系统方面取得的根本性进展，该系统能够有力地了解人类在其周围环境中的各种需求，该项目对协助和增强数百万人的权能以提高生活质量具有深远的影响。通过使移动的平台能够以互动方式进行调整，以满足固有的多样化需求，项目成果将促进广泛的可用性、信任，并减少阻碍残疾人融入社会的社会和物理障碍。该项目将解决实现通用无障碍驱动的智能系统，可以理解残疾人的不同需求的基本挑战。这项工作解决了基准，模型和技术的基础性进展，以在包容性导航和移动系统的背景下关闭感知到行动的循环。项目成果包括新颖的多任务学习框架，用于强大的，细粒度的和富有表现力的基于视觉的决策导航策略，可以有效地适应安全和可访问性约束的优化。考虑到数据稀缺、注释和共享方面的潜在问题，主要目标在于实现标准化的交互式开发框架，其中包含详细的识别任务和与可访问性相关的定制场景。框架开发和模型设计将通过真实世界的用户研究以及与定向和移动指导员的合作得到广泛的信息。此外，引入的框架将广泛吸引残疾人，学生，开发人员和教育工作者，以产生共享工具，用于培训下一代工程师的概念，以解决机器学习，感知和可访问性交叉点的多方面问题。因此，这项研究有助于未来部署通用自主和辅助导航系统，这些系统可以与环境中的所有人进行无缝交互并赋予他们权力。由工程局（ENG）和计算机与信息科学与工程局（CISE）共同管理和资助该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As prototypical intelligent mobile systems, from autonomous vehicles to delivery robots, move from their controlled development labs into the real-world, their impact on individuals with disabilities becomes discernible. An intelligent system that fails to account for diverse reactions and mobility characteristics among individuals can have dire consequences. For example, a delivery robot may inadvertently cause a safety-critical scenario by stopping in a manner which blocks a wheelchair user in traffic. Collision-free navigation in complex urban scenarios alongside blind individuals depends on the ability of the system to consider factors related to non-visual reasoning and cane mobility strategies in order to make precise future predictions. Yet, current autonomous systems cannot effectively differentiate among mobility aids and needs, nor can they reason over accessibility implications of various situations, from stairs to road layouts and ambient conditions. To teach systems to navigate while safely interacting with individuals with disabilities, the goal of this project is to develop comprehensive perception, understanding, and decision-making capabilities for accessibility and needs-aware interactive mobile systems. Moreover, through fundamental advancements in inclusive mobile systems that can robustly understand diverse needs of humans in their surroundings, this project has far-reaching implications for assisting and empowering millions of people to achieve greater quality-of-life. By enabling mobile platforms to interactively adapt to meet inherently diverse needs, project outcomes will facilitate broad usability, trust, and reduction of social and physical barriers that prevent individuals with disabilities from integrating into society. This project will address fundamental challenges in realizing generalized accessibility-driven intelligent systems that can understand the diverse needs of individuals with disabilities. The work tackles foundational advancements in benchmarks, models, and techniques for closing the perception-to-action loop in the context of inclusive navigational and mobility systems. Project outcomes include novel multi-task learning frameworks for robust, fine-grained, and expressive vision-based decision-making navigation policies that can be efficiently adapted to optimize for safety and accessibility constraints. Given underlying issues in data scarcity, annotation, and sharing, a main goal lies in realizing standardized interactive development framework with detailed recognition tasks and customization scenarios relevant to accessibility. Framework development and model design will be extensively informed through real-world user studies and collaboration with orientation and mobility instructors. In addition, the introduced framework will broadly engage individuals with disabilities, students, developers, and educators to produce shared tools for training the next generation of engineers concepts needed to tackle multifaceted problems at the intersection of machine learning, perception, and accessibility. Thus, this research facilitates future deployment of generalized autonomous and assistive navigation systems that can seamlessly interact with and empower all people in their environment.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.