RAPID: Advanced Topic Modeling Methods to Analyze Text Responses in COVID-19 Survey Data

随着COVID-19大流行的持续，公共和私营组织正在部署调查，为应对措施和政策选择提供信息。到目前为止，使用多项选择回答的调查设计是最常见的——“开放式”问题，即调查参与者提供较长形式的书面回答，使用得很少。这是事实，尽管当你允许人们提供不受约束的口头或文本回答时，就有可能获得更丰富、更细粒度的信息来澄清其他回答，以及有用的“自下而上”的信息，这些信息是调查设计者不知道要问的。一个关键的问题是，在开放式响应中分析非结构化语言是一个劳动密集型的过程，特别是在需要快速分析且资源有限的情况下，这给使用它们制造了障碍。计算方法可以提供帮助，但它们往往不能提供连贯的、可解释的类别，或者它们不能很好地将调查中的文本与封闭式回答联系起来。该项目将开发新的计算方法，用于快速有效地分析包括文本回复在内的调查数据，并将应用这些方法支持组织开展与COVID-19应对相关的高影响力调查工作。这将提高这些组织了解和减轻COVID-19大流行影响的能力。这个项目的技术方法建立在将深度学习和贝叶斯主题模型结合在一起的最新技术的基础上。将采用几项关键的技术革新，专门用于改进包括封闭式和开放式答复的调查中所提供信息的质量。这些方法中的一个共同元素是将监督学习设置中常用的方法（如基于任务的嵌入微调和知识蒸馏）扩展到无监督主题建模，特别关注于生成多样化的、人类可解释的主题类别，这些类别与离散属性（如人口特征、封闭响应和实验条件）很好地结合在一起。项目活动包括协助分析组织的调查数据，根据他们的需要进行独立的调查，以获得更多的相关数据，并公开发布一个干净、易于使用的计算工具包，以促进这些新方法的更广泛采用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As the COVID-19 pandemic continues, public and private organizations are deploying surveys to inform responses and policy choices. Survey designs using multiple choice responses are by far the most common -- "open ended" questions, where survey participants provide a longer-form written response, are used far less. This is true despite the fact that when you allow people to provide unconstrained spoken or text responses, it is possible to obtain richer, fine-grained information clarifying the other responses, as well as useful “bottom up” information that the survey designers did not know to ask for. A key problem is that analyzing the unstructured language in open-ended responses is a labor-intensive process, creating obstacles to using them especially when speedy analysis is needed and resources are limited. Computational methods can help, but they often fail to provide coherent, interpretable categories, or they can fail to do a good job connecting the text in the survey with the closed-end responses. This project will develop new computational methods for fast and effective analysis of survey data that includes text responses, and it will apply these methods to support organizations doing high-impact survey work related to COVID-19 response. This will improve these organizations’ ability to understand and mitigate the impact of the COVID-19 pandemic.This project’s technical approach builds on recent techniques bringing together deep learning and Bayesian topic models. Several key technical innovations will be introduced that are specifically geared toward improving the quality of information available in surveys that include both closed- and open-ended responses. A common element in these approaches is the extension of methods commonly used in supervised learning settings, such as task-based fine-tuning of embeddings and knowledge distillation, to unsupervised topic modeling, with a specific focus on producing diverse, human-interpretable topic categories that are well aligned with discrete attributes such as demographic characteristics, closed-end responses, and experimental condition. Project activities include assisting in the analysis of organizations' survey data, conducting independent surveys aligned with their needs to obtain additional relevant data, and the public release of a clean, easy to use computational toolkit facilitating more widespread adoption of these new methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.