CAREER: Integrating Interaction, Embodiment, and Emotion to Transform Language Models
职业:整合交互、体现和情感来转变语言模型
基本信息
- 批准号:2140642
- 负责人:
- 金额:$ 49.75万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2022
- 资助国家:美国
- 起止时间:2022-06-01 至 2027-05-31
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
When children learn language, they incorporate information from many sources including what they see, touch, smell, hear, and how they feel. Before children can speak, they feel emotions such as anger, joy, curiosity, and frustration and these emotions act as internal and external signals as they learn what words mean. For example, a child might show frustration when a caregiver does not understand what the child is saying, which in turn helps the child change what they understand a word or phrase to mean or how it is pronounced. The setting in which child language learning takes place is co-located where children are in the same physical location as the people they are learning language from, and the learning is through the communicative medium of spoken interaction. The information sources, emotions, and spoken interaction setting for children are in direct contrast to how machines learn language, which usually involves some kind of computational model that is given large amounts of text to learn from. This project aims to take inspiration from how children learn language in order to understand how to improve the models and methods of machines that learn language. Improved language learning in machines will enable machines to communicate with people more quickly, clearly, safely, and in ways that lower barriers for people to use complex technology with a natural spoken language interface. This CAREER project examines a novel approach for computer systems to learn spoken language that will advance a theoretical model and improve how people and systems communicate. Research will improve language modeling in natural language processing by taking inspiration from how children learn language: they interact with others to learn words that denote physical entities and events, and, like all humans, often respond emotionally and embody how they feel in their behavior, whereas researchers currently largely train language models only on static text. The research team will use two robotic platforms for the research and significantly enrich model efficacy and will add knowledge of emotion by modeling it based on human perceptions of robot behaviors. The team will add embodied knowledge by grounding into vision and robot states, and finally, the team will train and evaluate a robot that uses the language model as it interacts with humans to learn language from them. The study will also result in two important datasets: robot behaviors with accompanying descriptions of those behaviors and emotion labels, and longitudinal data of robots interacting and learning language from humans. The objectives are to (1) Model emotion; test, and refine through interaction, (2) Develop a unified language model, and (3) Engage people and robots that learn language with emotional content.This project is jointly funded by the CISE/IIS/Robust Intelligence Program and the NSF Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
当儿童学习语言时,他们会从许多来源吸收信息,包括他们所看到的,触摸的,闻到的,听到的以及他们的感受。在孩子会说话之前,他们会感受到愤怒、喜悦、好奇和沮丧等情绪,这些情绪在他们学习单词的含义时充当了内部和外部信号。例如,当照顾者不理解孩子在说什么时,孩子可能会表现出沮丧,这反过来又有助于孩子改变他们理解的单词或短语的含义或发音。儿童语言学习发生的环境是共同定位的,儿童与他们学习语言的人处于相同的物理位置,学习是通过口语互动的交流媒介进行的。儿童的信息源、情感和口语交互设置与机器学习语言的方式形成了直接对比,机器学习语言通常涉及某种计算模型,该模型被赋予了大量的文本来学习。该项目旨在从儿童学习语言的方式中获得灵感,以了解如何改进学习语言的机器的模型和方法。机器中改进的语言学习将使机器能够更快、更清晰、更安全地与人沟通,并降低人们使用复杂技术的障碍。这个CAREER项目研究了一种新的方法,用于计算机系统学习口语,这将推进一个理论模型,并改善人与系统的沟通方式。研究将通过从儿童学习语言的方式中获得灵感来改进自然语言处理中的语言建模:他们与他人互动以学习表示物理实体和事件的单词,并且像所有人一样,经常做出情感反应并在行为中体现他们的感受,而研究人员目前主要只在静态文本上训练语言模型。研究团队将使用两个机器人平台进行研究,并显着丰富模型功效,并将通过基于人类对机器人行为的感知进行建模来增加情感知识。该团队将通过了解视觉和机器人状态来添加具体知识,最后,该团队将训练和评估一个在与人类交互时使用语言模型来学习语言的机器人。该研究还将产生两个重要的数据集:机器人行为以及对这些行为和情感标签的描述,以及机器人与人类互动和学习语言的纵向数据。目标是:(1)情感模型;测试,并通过交互进行完善,(2)开发统一的语言模型,以及(3)让学习语言的人和机器人参与情感内容。该项目由CISE/IIS/Robust Intelligence Program和NSF Established Program to Stimulate Competitive Research(EPSCoR)联合资助。该奖项反映了NSF的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
数据更新时间:{{ journalArticles.updateTime }}
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Casey Kennington其他文献
Better Driving and Recall When In-car Information Presentation Uses Situationally-Aware Incremental Speech Output Generation
当车内信息呈现使用情境感知增量语音输出生成时,可以更好地驾驶和回忆
- DOI:
10.1145/2667317.2667332 - 发表时间:
2014 - 期刊:
- 影响因子:0
- 作者:
Casey Kennington;Spyridon Kousidis;Timo Baumann - 通讯作者:
Timo Baumann
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
与机器人对话:扩大 SLIVAR 社区参与和研究的提案
- DOI:
- 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Casey Kennington;Malihe Alikhani;Heather Pon;Katherine Atwell;Yonatan Bisk;Daniel Fried;Felix Gervits;Zhao Han;Mert Inan;Michael Johnston;Raj Korpan;Diane Litman;M. Marge;Cynthia Matuszek;Ross Mead;Shiwali Mohan;Raymond Mooney;Natalie Parde;Jivko Sinapov;Angela Stewart;Matthew Stone;Stefanie Tellex;Tom Williams - 通讯作者:
Tom Williams
Tiny Language Models Enriched with Multimodal Knowledge from Multiplex Networks
富含来自多重网络的多模态知识的微型语言模型
- DOI:
10.18653/v1/2023.conll-babylm.3 - 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
Clayton Fields;Osama Natouf;Andrew McMains;Catherine Henry;Casey Kennington - 通讯作者:
Casey Kennington
Casey Kennington的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Casey Kennington', 18)}}的其他基金
Collaborative Research: Conference: Dialogue and Robots
合作研究:会议:对话与机器人
- 批准号:
2306113 - 财政年份:2023
- 资助金额:
$ 49.75万 - 项目类别:
Standard Grant
相似海外基金
Deep transformers for integrating protein sequence, structure and interaction data to predict function
用于整合蛋白质序列、结构和相互作用数据以预测功能的深度转换器
- 批准号:
2308699 - 财政年份:2023
- 资助金额:
$ 49.75万 - 项目类别:
Continuing Grant
Integrating soil-structure interaction within seismic design and assesement of buildings in Canada
将土壤-结构相互作用纳入加拿大建筑物的抗震设计和评估
- 批准号:
RGPIN-2018-06584 - 财政年份:2022
- 资助金额:
$ 49.75万 - 项目类别:
Discovery Grants Program - Individual
Integrating soil-structure interaction within seismic design and assesement of buildings in Canada
将土壤-结构相互作用纳入加拿大建筑物的抗震设计和评估
- 批准号:
RGPIN-2018-06584 - 财政年份:2021
- 资助金额:
$ 49.75万 - 项目类别:
Discovery Grants Program - Individual
Integrating soil-structure interaction within seismic design and assesement of buildings in Canada
将土壤-结构相互作用纳入加拿大建筑物的抗震设计和评估
- 批准号:
RGPIN-2018-06584 - 财政年份:2020
- 资助金额:
$ 49.75万 - 项目类别:
Discovery Grants Program - Individual
Integrating systems biology, population-based, and traditional computational biology approaches to describe the modulation of mutation effects by specific combinations of gene variants forming an interaction network.
整合系统生物学、基于群体的生物学和传统计算生物学方法,通过形成相互作用网络的基因变体的特定组合来描述突变效应的调节。
- 批准号:
DDG-2018-00015 - 财政年份:2019
- 资助金额:
$ 49.75万 - 项目类别:
Discovery Development Grant
Integrating soil-structure interaction within seismic design and assesement of buildings in Canada
将土壤-结构相互作用纳入加拿大建筑物的抗震设计和评估
- 批准号:
RGPIN-2018-06584 - 财政年份:2019
- 资助金额:
$ 49.75万 - 项目类别:
Discovery Grants Program - Individual
Integrating Human Computer Interaction, Machine Learning, Game Design, and Educational Assessment in a STEM+C Curriculum
将人机交互、机器学习、游戏设计和教育评估融入 STEM C 课程
- 批准号:
1933848 - 财政年份:2019
- 资助金额:
$ 49.75万 - 项目类别:
Standard Grant
The Interaction of Pyroclastic Density Currents with the Atmosphere & Landscapes: Integrating Experiments and Computational Approaches for Validation & Examination of Entra
火山碎屑密度流与大气的相互作用
- 批准号:
1841376 - 财政年份:2018
- 资助金额:
$ 49.75万 - 项目类别:
Continuing Grant
CAREER: Enhancing Critical Reflection on Data by Integrating Users' Expectations in Visualization Interaction
职业:通过在可视化交互中整合用户的期望来增强对数据的批判性反思
- 批准号:
1930642 - 财政年份:2018
- 资助金额:
$ 49.75万 - 项目类别:
Continuing Grant
Integrating systems biology, population-based, and traditional computational biology approaches to describe the modulation of mutation effects by specific combinations of gene variants forming an interaction network.
整合系统生物学、基于群体的生物学和传统计算生物学方法,通过形成相互作用网络的基因变体的特定组合来描述突变效应的调节。
- 批准号:
DDG-2018-00015 - 财政年份:2018
- 资助金额:
$ 49.75万 - 项目类别:
Discovery Development Grant