权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

NRI: Collaborative Research: RobotSLANG: Simultaneous Localization, Mapping, and Language Acquisition

NRI：协作研究：RobotSLANG：同时本地化、绘图和语言习得

基本信息

批准号：
1522954
负责人：
Jeffrey Siskind
金额：
$ 65万
依托单位：
Purdue University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2015
资助国家：
美国
起止时间：
2015-09-01 至 2021-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1522954&HistoricalAwards=false
关键词：
NRI Collaborative Research RobotSLANG Simultaneous

项目摘要

Humans and robots alike have a critical need to navigate through new environments to carry out everyday tasks. A parent and child may be touring a college campus; a robot may be searching for survivors after a building has collapsed. In this collaboration by faculty at two institutions, the PIs envision human and robotic partners sharing common perceptual-linguistic experiences and cooperating in mundane tasks like janitorial work and home care as well as in critical tasks like emergency response or search-and-rescue. But while mapping and navigation are now commonplace for mobile robots, when considering human-robot collaboration for even simple tasks one is confronted by a critical barrier: robots and people do not share a common language. Human language is rich in linguistic elements for describing our spatial environment, the objects and places within it, and navigable paths through it (e.g., "go down the hallway and enter the third door on the right."). Robots, on the other hand, inhabit a metric world of occupied and unoccupied discretized grid cells, wherein most objects are devoid of meaning (semantics). The PIs' goal in this project is to overcome this limitation by conjoining the well understood problem of simultaneous localization and mapping (SLAM) with that of language acquisition, in order to enable robots to learn to communicate with people in English about navigation tasks. The PIs will spur interest in this novel research area within the scientific community by means of an Amazing Race challenge problem modeled after the reality television show of the same name, which will place robots and human-robot teams in unknown environments and charge them with completing a specific task as quickly as possible. Other outreach activities will include visits to K-12 schools with demonstrations. This work will focus on simultaneous localization, mapping, and language acquisition, a field of inquiry that remains untouched. The crucial principles are that semantics are formulated as a cost function, which in turn specifies a joint distribution over many variables including those capturing sensory input, language, the environment map, and robot motor control. The cost function and joint distribution support standard inference of many forms, such as command following. More importantly, they support multidirectional inference over multiple variable sets jointly, such as simultaneous mapping and language interpretation. Within this innovative multivariate optimization-based framework, the PIs plan a thorough experimental regimen including both synthetic and real-world datasets of challenging environments, grounding the semantics of natural language in spatial maps of the realistic visual world and robot motor control, while navigating along particular paths or to arrive at particular destinations in (possibly novel) environments that are mapped not only in a geometric sense but also with linguistic underpinning to these particular paths and destinations. The language approach is compositional and uses spatially-grounded representations of nouns (objects/places) and prepositions (relations between them). These spatially-grounded representations will be modeled in the context of mapping. Furthermore, the PIs will consider realistic environments and adapt visual models thereof according to the joint model. The PIs are aware of no other work that jointly models mapping, vision, and language acquisition.

人类和机器人都迫切需要在新的环境中导航，以执行日常任务。一对父母和孩子可能正在参观一所大学校园；一座建筑倒塌后，机器人可能正在寻找幸存者。在两个机构的教职员工的合作中，PI设想人类和机器人合作伙伴分享共同的感知语言经验，并在日常任务中合作，如清洁工作和家庭护理，以及紧急响应或搜救等关键任务。但是，尽管地图和导航现在对移动机器人来说已经司空见惯，但当考虑到即使是简单任务的人-机器人协作时，人们也面临着一个关键障碍：机器人和人类没有共同语言。人类语言包含丰富的语言元素，用来描述我们的空间环境、其中的物体和场所，以及通过它的导航路径(例如，沿着走廊走下去，进入右边的第三扇门。)另一方面，机器人生活在一个由被占用和未被占用的离散网格单元组成的度量世界中，其中大多数对象缺乏意义(语义)。在这个项目中，PI的目标是通过将众所周知的同步定位和地图绘制(SLAM)问题与语言习得问题结合起来，以克服这一限制，以便使机器人能够学习用英语与人类就导航任务进行沟通。PI将在科学界通过仿照同名电视真人秀节目的神奇种族挑战问题来刺激科学界对这一新颖研究领域的兴趣，该问题将把机器人和人类-机器人团队置于未知环境中，并要求他们尽快完成特定任务。其他外展活动将包括参观K-12学校的示威活动。这项工作将专注于同时本地化、地图绘制和语言习得，这是一个仍未触及的研究领域。关键的原则是，语义被制定为一个成本函数，该成本函数反过来指定了在许多变量上的联合分布，这些变量包括那些捕获感觉输入、语言、环境地图和机器人运动控制的变量。费用函数和联合分配支持多种形式的标准推理，如命令跟随。更重要的是，它们支持多个变量集联合的多方向推理，如同步映射和语言翻译。在这个创新的基于多变量优化的框架内，PI计划一个全面的实验方案，包括具有挑战性的环境的合成和真实世界数据集，在现实视觉世界和机器人电机控制的空间地图中建立自然语言的语义基础，同时沿着特定路径导航或在(可能是新颖的)环境中到达特定目的地，这些环境不仅在几何意义上映射，而且在这些特定路径和目的地的语言基础上映射。语言方法是构图的，使用名词(对象/场所)和介词(它们之间的关系)的空间基础表示法。这些基于空间的表示将在映射的环境中建模。此外，PI将考虑真实环境，并根据联合模型调整其视觉模型。PI不知道其他任何联合建模映射、视觉和语言习得的工作。

项目成果

期刊论文数量（6）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Object classification from randomized EEG trials

DOI：
10.1109/cvpr46437.2021.00384
发表时间：
2020-04
期刊：
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
影响因子：
0
作者：
Hamad Ahmed;R. Wilbur;Hari M. Bharadwaj;J. Siskind
通讯作者：
Hamad Ahmed;R. Wilbur;Hari M. Bharadwaj;J. Siskind

Perturbation confusion in forward automatic differentiation of higher-order functions

DOI：
10.1017/s095679681900008x
发表时间：
2012-11
期刊：
Journal of Functional Programming
影响因子：
1.1
作者：
Oleksandr Manzyuk;Barak A. Pearlmutter;Alexey Radul;David R. Rush;J. Siskind
通讯作者：
Oleksandr Manzyuk;Barak A. Pearlmutter;Alexey Radul;David R. Rush;J. Siskind

Divide-and-conquer checkpointing for arbitrary programs with no user annotation