Automatic Spoken Language Acquisition Based on Observation and Dialogue
Automatic Spoken Language Acquisition Based on Observation and Dialogue
复制标题
DOI：
10.1109/jstsp.2022.3189279
复制
发表时间：
2022-10
影响因子：
7.5
通讯作者：
Ryota Komatsu;Shengzhou Gao;Wenxin Hou;Mingxin Zhang;Tomohiro Tanaka;Keisuke Toyoda;Yusuke Kimura;Kent Hino;Yumiko Iwamoto;Kosuke Mori;T. Okamoto;T. Shinozaki
中科院分区：
文献类型：
--
作者：
Ryota Komatsu;Shengzhou Gao;Wenxin Hou;Mingxin Zhang;Tomohiro Tanaka;Keisuke Toyoda;Yusuke Kimura;Kent Hino;Yumiko Iwamoto;Kosuke Mori;T. Okamoto;T. Shinozaki
人类婴儿出生时不知道任何特定的语言。他们直接从观察和对话中获得语言，而不受标记数据的限制。我们提出了模拟这个过程的口语习得代理。这种能力需要多种类型的学习，包括1）单词发现，2）符号基础，3）消息生成，以及4）发音生成。一些研究已经针对一种或组合的学习类型来阐明人类智能，并旨在为口语对话系统提供类似人类的灵活语言学习能力。然而，他们的语言能力部分缺乏某些组成部分。我们的代理商是第一个整合它们的。我们的关键概念是设计一个架构，将无监督、自监督和强化学习集成在一起，利用原始感觉信号中自然存在的线索，并基于智能体的内在动机驱动学习。实验结果表明，智能体通过与环境交互并通过说话采取行动，成功地从头开始习得口语。我们提出的聚焦机制显著提高了学习效率。我们还证明了我们的代理可以学习神经声码器和逻辑否定的概念作为语言习得的一部分。
Human babies are born without knowledge of any specific language. They acquire language directly from observation and dialogue without being limited by the availability of labeled data. We propose spoken language acquisition agents that simulate the process. Such an ability requires multiple types of learning, including 1) word discovery, 2) symbol grounding, 3) message generation, and 4) pronunciation generation. Several studies have targeted one or combined learning types to elucidate human intelligence and aimed to equip spoken dialogue systems with human-like flexible language learning ability. However, their language ability was partially lacking some of the components. Our agents are the first to integrate them all. Our key concept is to design an architecture to integrate unsupervised, self-supervised, and reinforcement learning to utilize clues naturally existing in raw sensory signals and drive the learning based on the agent’s intrinsic motivation. Experimental results show agents successfully acquire spoken language from scratch by interacting with an environment to act by speaking. Our proposed focusing mechanism significantly improves learning efficiency. We also demonstrate that our agents can learn neural vocoder and the concept of logical negation as a part of language acquisition.