FRR: Towards Robust and Perceptual Inclusive Mobile Robots

随着从自动驾驶车辆到送货机器人等原型智能移动系统从受控开发实验室进入现实世界，它们对残疾人的影响变得显而易见。如果智能系统未能考虑到个体之间的不同反应和流动性特征，可能会产生可怕的后果。例如，送货机器人可能会以阻碍轮椅使用者在交通中的方式停车，从而无意中导致安全关键场景。在复杂的城市场景中与盲人一起实现无碰撞导航，取决于系统是否能够考虑与非视觉推理和拐杖移动策略相关的因素，以便做出准确的未来预测。然而，目前的自主系统不能有效地区分辅助行动和需求，也不能对从楼梯到道路布局和环境条件等各种情况的无障碍影响进行推理。为了教系统在与残障人士安全互动的同时导航，该项目的目标是为无障碍和有需求的交互式移动系统开发全面的感知、理解和决策能力。此外，通过包容性移动系统的根本进步，该系统可以有力地了解周围人类的不同需求，该项目对于帮助和增强数百万人实现更高的生活质量具有深远的影响。通过使移动平台能够互动地适应内在的多样化需求，项目成果将促进广泛的可用性、信任度，并减少阻碍残疾人融入社会的社会和身体障碍。该项目将解决在实现通用的无障碍驱动的智能系统方面的根本挑战，该系统可以理解残疾人的不同需求。这项工作解决了在包容性导航和移动系统的背景下关闭感知到行动循环的基准、模型和技术方面的基础性进展。项目成果包括用于稳健、细粒度和富有表现力的基于视觉的决策导航策略的新颖多任务学习框架，这些策略可以有效地调整以针对安全和可访问性限制进行优化。考虑到数据稀缺、注释和共享方面的潜在问题，主要目标在于实现标准化的交互开发框架，其中包含详细的识别任务和与可访问性相关的定制场景。框架开发和模型设计将通过真实世界的用户研究以及与定向和机动性指导员的合作而广泛了解。此外，引入的框架将广泛吸引残障人士、学生、开发人员和教育工作者开发共享工具，以培训下一代工程师解决机器学习、感知和可访问性交叉点的多方面问题所需的概念。因此，这项研究促进了未来通用自主和辅助导航系统的部署，这些系统可以与其环境中的所有人无缝交互并赋予其权力。该项目由跨部门机器人基础研究计划支持，该计划由工程指导委员会(ENG)和计算机和信息科学与工程指导委员会(CEISE)联合管理和资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As prototypical intelligent mobile systems, from autonomous vehicles to delivery robots, move from their controlled development labs into the real-world, their impact on individuals with disabilities becomes discernible. An intelligent system that fails to account for diverse reactions and mobility characteristics among individuals can have dire consequences. For example, a delivery robot may inadvertently cause a safety-critical scenario by stopping in a manner which blocks a wheelchair user in traffic. Collision-free navigation in complex urban scenarios alongside blind individuals depends on the ability of the system to consider factors related to non-visual reasoning and cane mobility strategies in order to make precise future predictions. Yet, current autonomous systems cannot effectively differentiate among mobility aids and needs, nor can they reason over accessibility implications of various situations, from stairs to road layouts and ambient conditions. To teach systems to navigate while safely interacting with individuals with disabilities, the goal of this project is to develop comprehensive perception, understanding, and decision-making capabilities for accessibility and needs-aware interactive mobile systems. Moreover, through fundamental advancements in inclusive mobile systems that can robustly understand diverse needs of humans in their surroundings, this project has far-reaching implications for assisting and empowering millions of people to achieve greater quality-of-life. By enabling mobile platforms to interactively adapt to meet inherently diverse needs, project outcomes will facilitate broad usability, trust, and reduction of social and physical barriers that prevent individuals with disabilities from integrating into society. This project will address fundamental challenges in realizing generalized accessibility-driven intelligent systems that can understand the diverse needs of individuals with disabilities. The work tackles foundational advancements in benchmarks, models, and techniques for closing the perception-to-action loop in the context of inclusive navigational and mobility systems. Project outcomes include novel multi-task learning frameworks for robust, fine-grained, and expressive vision-based decision-making navigation policies that can be efficiently adapted to optimize for safety and accessibility constraints. Given underlying issues in data scarcity, annotation, and sharing, a main goal lies in realizing standardized interactive development framework with detailed recognition tasks and customization scenarios relevant to accessibility. Framework development and model design will be extensively informed through real-world user studies and collaboration with orientation and mobility instructors. In addition, the introduced framework will broadly engage individuals with disabilities, students, developers, and educators to produce shared tools for training the next generation of engineers concepts needed to tackle multifaceted problems at the intersection of machine learning, perception, and accessibility. Thus, this research facilitates future deployment of generalized autonomous and assistive navigation systems that can seamlessly interact with and empower all people in their environment.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.