RAPID: Advanced Topic Modeling Methods to Analyze Text Responses in COVID-19 Survey Data

RAPID：用于分析 COVID-19 调查数据中文本响应的高级主题建模方法

• 批准号: 2031736
• 负责人: Philip Resnik
• 金额: $ 17.68万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-15 至 2023-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2031736&HistoricalAwards=false
• 关键词: RAPID; Advanced; Topic; Modeling; Methods

As the COVID-19 pandemic continues, public and private organizations are deploying surveys to inform responses and policy choices. Survey designs using multiple choice responses are by far the most common -- "open ended" questions, where survey participants provide a longer-form written response, are used far less. This is true despite the fact that when you allow people to provide unconstrained spoken or text responses, it is possible to obtain richer, fine-grained information clarifying the other responses, as well as useful “bottom up” information that the survey designers did not know to ask for. A key problem is that analyzing the unstructured language in open-ended responses is a labor-intensive process, creating obstacles to using them especially when speedy analysis is needed and resources are limited. Computational methods can help, but they often fail to provide coherent, interpretable categories, or they can fail to do a good job connecting the text in the survey with the closed-end responses. This project will develop new computational methods for fast and effective analysis of survey data that includes text responses, and it will apply these methods to support organizations doing high-impact survey work related to COVID-19 response. This will improve these organizations’ ability to understand and mitigate the impact of the COVID-19 pandemic.This project’s technical approach builds on recent techniques bringing together deep learning and Bayesian topic models. Several key technical innovations will be introduced that are specifically geared toward improving the quality of information available in surveys that include both closed- and open-ended responses. A common element in these approaches is the extension of methods commonly used in supervised learning settings, such as task-based fine-tuning of embeddings and knowledge distillation, to unsupervised topic modeling, with a specific focus on producing diverse, human-interpretable topic categories that are well aligned with discrete attributes such as demographic characteristics, closed-end responses, and experimental condition. Project activities include assisting in the analysis of organizations' survey data, conducting independent surveys aligned with their needs to obtain additional relevant data, and the public release of a clean, easy to use computational toolkit facilitating more widespread adoption of these new methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着COVID-19大流行的持续，公共和私人组织正在部署调查，以了解应对措施和政策选择。到目前为止，使用多项选择回答的调查设计是最常见的--“开放式”问题，即调查参与者提供较长形式的书面回答，使用得少得多。尽管当你允许人们提供不受约束的口头或文本回答时，有可能获得更丰富，更细粒度的信息来澄清其他回答，以及调查设计者不知道要问的有用的“自下而上”的信息。 一个关键问题是，分析开放式响应中的非结构化语言是一个劳动密集型过程，这给使用它们造成了障碍，特别是在需要快速分析和资源有限的情况下。计算方法可以提供帮助，但它们通常无法提供连贯的、可解释的类别，或者它们无法很好地将调查中的文本与封闭式回答联系起来。该项目将开发新的计算方法，以快速有效地分析包括文本回复在内的调查数据，并将这些方法应用于支持组织开展与COVID-19应对相关的高影响力调查工作。 这将提高这些组织理解和减轻COVID-19大流行影响的能力。该项目的技术方法建立在将深度学习和贝叶斯主题模型结合在一起的最新技术之上。将采用几项关键的技术创新，专门用于提高包括封闭式和开放式答复的调查中所提供信息的质量。这些方法中的一个共同点是将监督学习环境中常用的方法（例如基于任务的嵌入和知识蒸馏微调）扩展到无监督主题建模，特别关注产生多样化的，人类可解释的主题类别，这些类别与离散属性（例如人口统计特征，封闭式响应和实验条件）保持一致。项目活动包括协助分析各组织的调查数据，根据其需要进行独立调查，以获得更多的相关数据，易于使用的计算工具包，促进了这些新方法的更广泛采用。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响进行评估，被认为值得支持审查标准。

项目成果

Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence

• 发表时间: 2021-07
• 作者: Alexander Miserlis Hoyle;Pranav Goel;Denis Peskov;Andrew Hian-Cheong;Jordan L. Boyd-Graber;P. Resnik

Improving Neural Topic Models Using Knowledge Distillation

• DOI: 10.18653/v1/2020.emnlp-main.137
• 发表时间: 2020-10
• 作者: Alexander Miserlis Hoyle;Pranav Goel;P. Resnik