CAREER: End-User Robot Programming by Multimodal Instruction

这项研究将使辅助机器人触手可及，使这些用户能够轻松地指导它们帮助完成有用的日常任务，从而最终使许多非专业人士，特别是希望继续独立生活的老年人，能够更好地从这项技术中受益。据预测，到2060年，美国65岁以上人口的比例将达到23.5%。因此，对医疗资源和社会支持的需求正在大幅增长。然而，我们的社会目前正经历着护理人员的短缺，而目前的护理人员面临着越来越大的倦怠风险。项目成果将允许很少或没有编码或工程知识的用户通过各种方式（例如，演示）轻松定制他们的机器人来执行满足他们需求的任务，例如整理鞋子或清洁浴室。这项工作的其他广泛影响将来自教育活动，通过指导不同背景的学生和向广泛年龄的学习者传播研究成果，扩大对计算机的参与。为了实现这些目标，本项目将通过自然多模态指令研究机器人编程的科学和基础技术。人类指令本质上是多模态的，但机器人系统并不总是充分捕获和利用嵌入其中的丰富信息。该研究将对多模态教学线索进行计算建模，以表征改进学习的演示质量，帮助外行人进行任务演示，并在面对教学困难时增强用户体验。这项工作将通过多模态演示创建机器人编程的开源系统，并产生定量和定性的知识，了解老年人如何自然地指导机器人，以及他们如何使用可编程机器人辅助来支持他们在家里的日常任务。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research will bring assistive robots within reach of average people by enabling such users to easily instruct them to help complete useful daily tasks, so that ultimately many lay people, especially older adults who wish to continue living independently, will be better able to benefit from the technology. It is projected that by 2060, the percent of the U.S. population aged 65 years or older will reach 23.5%. Consequently, the demand for healthcare resources and societal support are growing substantially. Our society, however, is presently experiencing a shortage of caregivers, and current caregivers are at increased risk for burnout. Project outcomes will allow users with little or no knowledge of coding or engineering to easily customize their robots through a variety of means (e.g., demonstrations) to perform tasks that meet their needs, such as organizing their shoes or cleaning the bathroom. Additional broad impact of the work will derive from educational activities that broaden participation in computing by mentoring students with diverse backgrounds and dissemination of research findings to a wide age range of learners. To achieve these goals, this project will investigate the science of and fundamental technologies for robot programming by natural multimodal instruction. Human instructions are inherently multimodal, but the rich information embedded in them is not always adequately captured and utilized by robot systems. The research will computationally model multimodal instructional cues to characterize demonstration quality for improved learning, assist lay people in their task demonstrations, and enhance the user experience in the face of instructional difficulties. The work will create open-source systems for robot programming by multimodal demonstration, and produce quantitative and qualitative knowledge of how older adults naturally instruct robots and how they may use programmable robotic assistance to support them in their daily tasks at home.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.