Correcting robot behaviour through embodied natural language interaction

我的工作将是在语义分析领域：即，创建自动系统，将自然语言输入映射到可执行的输出，以捕获用户的交流意图。例如，“把红块放在蓝块上”可以映射到一些正式的表示上，比如“(PUT，RB_1，BB_1)”，这可以由机器人执行。语义解析的一种常见方法是赋予系统关于域中可能的动作和状态的完整知识：任务是学习如何使用自然语言来引用已知域概念和动作。通常，这些系统假定自然语言串是指示性的并且描述当前视觉场景的某些方面，或者它是命令性的并且它描述要执行的动作。但人类语言使用者与他们的具体环境表达了许多其他类型的连贯关系。例如，用户可以通过表达一些规则或启发式规则来“纠正”机器人当前正在做的事情，这些规则或启发式规则在被准确理解后应用于修改机器人未来执行此类操作的方式。例如，当机器人执行将绿色方块放在红色方块上的操作时，用户可能会声明：“不要将绿色方块放在蓝色方块上。将红色方块放在蓝色方块上”。这应该会改变系统将一个块放在蓝色块上的未来操作。第一步将是创建一个改变其行为的系统，如上所述。然而，我们将对感觉输入做出许多假设，例如对话语的视觉和理解，以便这些假设可以被视为确定性的。一旦工作成功，我们将放弃对黄金标准解析和视觉处理的假设，并以概率的方式对流程的噪声方面进行建模。

My work will be in the arena of semantic parsing: i.e., the task of creating automated systems that map natural language input into an executable output that captures the communicative intention of the user. For example, "Put the red block on the blue block" could map onto some formal representation, say "(put, rb_1, bb_1)", that can be executed by a robot. One common approach to semantic parsing is to endow the system with complete knowledge of the possible actions and states in the domain: the task is to learn how natural language is used to refer to known domain concepts and actions. Typically, these systems assume that the natural language string is either indicative and describes some aspect of the current visual scene, or it is imperative and it describes an action to be carried out. But human language users express many other kinds of coherent relations to their embodied environment. For example, a user might *correct* what a robot is currently doing, by expressing some rule or heuristic that, when accurately understood, should be used to revise the way the robot performs such actions in future. For example, in response to the robot performing an action where it is putting a green block on a red block,, the user might state "Don't put green blocks on blue blocks. Put red blocks on blue blocks". This should alter future actions where the system is putting a block on a blue block. The first step will be to create a system that changes its behaviour as described above. However, we will make many assumptions about the sensory input such as vision and understanding of discourse, such that these can be treated as deterministic. Once this works successfully, we will drop assumptions about gold standard parsing and visual processing, and model the noisy aspects of the process probabilistically.