CAREER: Long Document Summarization with Question-Summary Hierarchy and User Preference Control

职业：具有问题摘要层次结构和用户偏好控制的长文档摘要

• 批准号: 2046016
• 负责人: Lu Wang
• 金额: $ 54.76万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2046016&HistoricalAwards=false
• 关键词: CAREER; Long; Document; Summarization; Question

In an era when long documents are produced at an overwhelming speed, a reader may not have time even to skim over a document to decide which topics deserve a detailed look. The goal of this CAREER project is to build text summarization systems that can understand and aggregate information from long documents, so as to allow users to explore their content with summaries that are generated in styles they prefer. The summarization tools will make long documents more accessible and comprehensible, easing the knowledge learning experience of the general public. Researchers and practitioners can also use the tools to summarize long documents relevant to their work, and educators can incorporate them in their classes to bolster students' reading and writing skills. The project also broadens the investigator’s efforts of engaging young students in immersive research opportunities, allowing them to participate in the design and implementation of advanced summarization systems. This project develops a new summarization framework for long documents in which article-level abstractive summaries provide an overview, and a question-summary hierarchy presents different levels of details. The technical contributions of this project are three-fold. First, the quadratic time complexity of state-of-the-art summarization (e.g., Transformer) is reduced by using adaptively predicted sparse attentions and augmented with a knowledge encoder. Second, an open-ended question generation model fills automatically learned question templates to produce concrete questions that are coherent within the question-summary hierarchy. Third, summaries are tailored to user-specified styles via iterative adjustments during generation, reflecting important advice in plain-language guidelines. This project experiments with new datasets collected from government reports, since their length, topic diversity, and formulaic verbiage embody many common challenges for long document summarization. New evaluation methods are also designed, with cloze questions to target common erroneous generations, and with model confidence metrics to pinpoint errors without using references.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在一个长文档以压倒性的速度产生的时代，读者可能甚至没有时间浏览文档以决定哪些主题值得详细查看。这个CAREER项目的目标是构建文本摘要系统，可以理解和聚合长文档中的信息，以便允许用户使用以他们喜欢的风格生成的摘要来探索他们的内容。摘要工具将使长文档更容易访问和理解，减轻公众的知识学习体验。研究人员和从业人员还可以使用这些工具来总结与他们的工作相关的长文档，教育工作者可以将它们纳入课堂，以提高学生的阅读和写作技能。该项目还扩大了研究人员的努力，让年轻学生参与沉浸式研究的机会，让他们参与先进的摘要系统的设计和实施。这个项目为长文档开发了一个新的摘要框架，其中文章级摘要提供了一个概述，问题摘要层次结构提供了不同层次的细节。该项目的技术贡献有三个方面。首先，现有技术的摘要的二次时间复杂度（例如，Transformer）通过使用自适应预测稀疏注意力来减少，并使用知识编码器来增强。第二，开放式问题生成模型填充自动学习的问题模板，以生成在问题摘要层次结构内连贯的具体问题。第三，摘要通过生成过程中的迭代调整来适应用户指定的样式，反映了简明语言指南中的重要建议。该项目实验了从政府报告中收集的新数据集，因为它们的长度，主题多样性和公式化的措辞体现了长文档摘要的许多常见挑战。新的评估方法也被设计，完形填空问题针对常见的错误世代，并与模型的信心指标，以查明错误，而不使用reference.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases

• DOI: 10.48550/arxiv.2305.12018
• 发表时间: 2023-05
• 作者: Xin Liu;Muhammad Khalifa;Lu Wang

Time-aware Prompting for Text Generation

• DOI: 10.48550/arxiv.2211.02162
• 发表时间: 2022-11
• 作者: Shuyang Cao;Lu Wang