CAREER: Robots, Speech, and Learning in Inclusive Human Spaces

随着机器人变得越来越有能力和无处不在，它们越来越多地进入传统的以人为中心的环境，如医疗保健，教育和老年人护理。随着机器人从事各种各样的任务，如帮助家务劳动，部署药物和辅导学生，它们与周围的人自然互动变得越来越重要。这一进展的关键是机器人的发展，这些机器人通过与各种终端用户的自然通信来理解目标和对象。解决这个问题的一种方法是使用语言来构建系统，从与之交互的人那里学习。在这个项目中开发的算法和系统将允许机器人从语言交互中了解周围的世界。这项研究将侧重于理解来自不同人群的关于物理世界的口头语言，从而使系统能够更好地处理各种现实世界的交互。最终，该项目将提高部署在人类空间中的机器人的可用性和公平性。这个CAREER项目将研究机器人如何通过口语结合感知来学习嘈杂、不可预测的人类环境，使用来自传感器的上下文来约束学习问题。扎根语言是指发生在机器人操作的物理世界中的语言。人类的互动从根本上说是语境性的：当我们学习世界时，我们不仅要考虑直接的交流，还要考虑这种互动的语境。对于许多现有的学习理解物理语言的工作，文本是主要的中介语，上下文被认为是相对狭窄的。此外，对现有大型数据集的依赖已经开始引发关于学习驱动技术的偏见和包容性的问题。为了解决这些限制，这项工作将集中在学习语义直接从感性输入结合语音来自不同的来源。目标是开发学习基础设施、算法和方法，使机器人能够学习理解与最终用户进行口头通信的任务指令和对象描述。该项目将开发新的方法来有效地从多模态数据输入中学习，最终目标是使机器人能够有效地和自然地了解他们的世界和他们应该执行的任务。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

As robots become more capable and ubiquitous, they are increasingly moving into traditionally human-centric environments such as as health care, education, and elder care. As robots engage in tasks as diverse as helping with household work, deploying medication, and tutoring students, it becomes increasingly critical for them to interact naturally with the people around them. Key to this progress is the development of robots that acquire an understanding of goals and objects from natural communications with a diverse set of end users. One way to address this is using language to build systems that learn from people they are interacting with. Algorithms and systems developed in this project will allow robots to learn about the world around them from linguistic interactions. This research will focus on understanding spoken language about the physical world from diverse groups of people, resulting in systems that are more able to robustly handle a wide variety of real-world interactions. Ultimately, the project will increase the usability and fairness of robots deployed in human spaces.This CAREER project will study how robots can learn about noisy, unpredictable human environments from spoken language combined with perception, using context derived from sensors to constrain the learning problem. Grounded language refers to language that occurs in and refers to the physical world in which robots operate. Human interactions are fundamentally contextual: when learning about the world, we focus learning by considering not only direct communication but also the context of that interaction. For much existing work on learning to understand physically situated language, text is the primary interlingua, and context is considered relatively narrowly. Additionally, reliance on pre-existing large datasets has begun to raise questions about bias and inclusivity in learning-driven technologies. To address these limitations, this work will focus on learning semantics directly from perceptual inputs combined with speech from diverse sources. The goal is to develop learning infrastructure, algorithms, and approaches to enable robots to learn to understand task instructions and object descriptions from spoken communication with end users. The project will develop new methods of efficiently learning from multi-modal data inputs, with the ultimate goal of enabling robots to efficiently and naturally learn about their world and the tasks they should perform.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.