III: Small: Collaborative Research: Modeling Pre- and Post- Conditions for Understanding Events

III：小：协作研究：为理解事件建模前置条件和后置条件

• 批准号: 2007290
• 负责人: Niranjan Balasubramanian
• 金额: $ 40.77万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007290&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; Modeling

This project develops new algorithms for learning the typical pre-conditions and post-conditions of real-world events. These logical conditions are crucial for developing better language understanding applications that can reason precisely about situations described in written text. There have been significant technological advances in the automatic understanding of text, but event reasoning requires knowledge that is often unstated and implicit. For example, if a meeting is canceled, it would be unusual for the text to say that the meeting was scheduled to happen (a pre-condition), and that it will now no longer happen (a post-condition). While these are obvious to a human, these conditions are unknown and crucial to building assistive technology. This kind of reasoning can enable complete document understanding, support precise and explainable question answering, and improve the output of language generation. This project has broad applications to a variety of assistive technology for information access and understanding for the general public. Learning pre-conditions can further educational text exploration applications by explaining how a certain situation came about. Better language understanding can also help explain automated decisions, making technology more trustworthy in mission critical domains. And finally, this project will help address the shortage of talent in the critical areas of computer science and machine learning by training graduate and undergraduate students. This project focuses on developing both the models to learn pre- and post-condition relations, but also the large datasets required to enable this learning. The first thrust in the project plan is to develop new datasets of conditional knowledge, and then to develop initial supervised learning algorithms to detect them in text. The project will initially focus on today’s large-scale language models to establish competitive baselines that the rest of the project will improve upon. After creating these annotated datasets and baselines, the focus will then turn to developing generative neural architectures like variational autoencoders that are augmented with rich structured latent spaces. These spaces will be augmented with entity networks that allow it to track generic event knowledge, but also specific knowledge about the entities. The motivation for generative models is to aggregate condition knowledge from large collections of unlabeled text as well. Finally, in addition to developing large scale datasets to learn this knowledge, the project will develop new reasoning tasks that could spur the community to develop more precise language understanding models, and to use these tasks to further research into richer models of event knowledge. All scientific findings, datasets, and other artifacts of the research will be made available for the scientific community and the broader public.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目开发了用于学习现实世界事件的典型前置条件和后设条件的新算法。这些逻辑条件对于开发更好的语言理解应用程序至关重要，这些应用程序可以对书面文本中描述的情况进行精确推理。在文本的自动理解方面已经取得了重大的技术进步，但事件推理需要的知识往往是未陈述和隐含的。例如，如果一个会议被取消了，那么文本就不可能说会议已经安排好了（前置条件），而现在它将不再发生（后置条件）。虽然这些对人类来说是显而易见的，但这些条件是未知的，对构建辅助技术至关重要。这种推理可以实现完整的文档理解，支持精确和可解释的问答，提高语言生成的输出。该项目广泛应用于各种辅助技术，为公众提供信息获取和理解。学习前提条件通过解释某一情境是如何产生的，可以进一步推动教育性文本探究的应用。更好的语言理解也有助于解释自动化决策，使技术在关键任务领域更值得信赖。最后，该项目将通过培训研究生和本科生，帮助解决计算机科学和机器学习关键领域人才短缺的问题。该项目侧重于开发学习前置和后置条件关系的模型，以及实现这种学习所需的大型数据集。项目计划的第一个重点是开发新的条件知识数据集，然后开发初步的监督学习算法，以在文本中检测它们。该项目将首先关注当今的大规模语言模型，以建立具有竞争力的基线，项目的其余部分将在此基础上进行改进。在创建了这些带注释的数据集和基线之后，重点将转向开发生成神经架构，如变分自编码器，它被丰富的结构化潜在空间所增强。这些空间将通过实体网络进行扩充，使其能够跟踪一般事件知识，以及关于实体的特定知识。生成模型的动机是从大量未标记的文本集合中聚合条件知识。最后，除了开发大规模数据集来学习这些知识外，该项目还将开发新的推理任务，以刺激社区开发更精确的语言理解模型，并使用这些任务进一步研究更丰富的事件知识模型。所有科学发现、数据集和其他研究成果将向科学界和广大公众开放。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation

文本衍生知识有助于视觉：基于视频的动作预期的简单跨模式蒸馏

• 发表时间: 2023
• 期刊: Findings of the Association for Computational Linguistics: EACL 2023
• 作者: Ghosh, Sayontan;Aggarwal, Tanvi;Hoai, Minh;Balasubramanian, Niranjan
• 通讯作者: Balasubramanian, Niranjan

Analyzing the Contribution of Commonsense Knowledge Sources for Why-Question Answering

• 发表时间: 2022
• 作者: Yash Kumar Lal

Towards Diverse Precondition Generation

实现多样化的前提条件生成

• 发表时间: 2021
• 期刊: Joint Conference on Lexical and Computational Semantics
• 作者: Kown, Heeyoung;Chambers, Nathanael;Balasubramanian, Niranjan
• 通讯作者: Balasubramanian, Niranjan

NEUROSTRUCTURAL DECODING: Neural Text Generation with Structural Constraints

• DOI: 10.18653/v1/2023.acl-long.528
• 发表时间: 2023
• 作者: Mohaddeseh Bastan;M. Surdeanu;Niranjan Balasubramanian

Using Commonsense Knowledge to Answer Why-Questions

• DOI: 10.18653/v1/2022.emnlp-main.79
• 发表时间: 2022
• 作者: Yash Kumar Lal