Interactive Natural Language-Based Person Search
Interactive Natural Language-Based Person Search
复制标题
DOI：
10.1109/lra.2020.2969921
复制
发表时间：
2020-01
影响因子：
5.2
通讯作者：
Vikram Shree;Wei-Lun Chao;M. Campbell
中科院分区：
文献类型：
--
作者：
Vikram Shree;Wei-Lun Chao;M. Campbell
在这项工作中，我们考虑使用自然语言描述在不受约束的环境中搜索人员的问题。具体来说，我们研究如何系统地设计一种算法来有效地获取人类的描述。通过调整用于视觉和语言理解的模型，提出了一种算法，以有原则的方式搜索感兴趣的人（POI），无需重新设计另一个复杂的模型即可获得有希望的结果。然后，我们研究了一种迭代问答 (QA) 策略，该策略使机器人能够向用户请求有关 POI 外观的其他信息。为此，我们引入了一种贪婪算法来根据问题的重要性对问题进行排序，并使该算法能够根据模型的不确定性动态调整人机交互的长度。我们的方法不仅在基准数据集上得到验证，而且在移动机器人上得到验证，在动态和拥挤的环境中移动。
In this work, we consider the problem of searching people in an unconstrained environment, with natural language descriptions. Specifically, we study how to systematically design an algorithm to effectively acquire descriptions from humans. An algorithm is proposed by adapting models, used for visual and language understanding, to search a person of interest (POI) in a principled way, achieving promising results without the need to re-design another complicated model. We then investigate an iterative question-answering (QA) strategy that enable robots to request additional information about the POI's appearance from the user. To this end, we introduce a greedy algorithm to rank questions in terms of their significance, and equip the algorithm with the capability to dynamically adjust the length of human-robot interaction according to model's uncertainty. Our approach is validated not only on benchmark datasets but on a mobile robot, moving in a dynamic and crowded environment.