RAPID: Advanced Topic Modeling Methods to Analyze Text Responses in COVID-19 Survey Data

随着新冠肺炎大流行的持续，公共和私人组织正在部署调查，为应对和政策选择提供信息。到目前为止，使用多项选择回答的调查设计是最常见的--调查参与者提供更长形式的书面回答的“开放式”问题使用得少得多。这是正确的，尽管当你允许人们提供不受限制的口头或文字回答时，就有可能获得更丰富的、细粒度的信息来澄清其他回答，以及调查设计者不知道要求的有用的“自下而上”信息。一个关键的问题是，分析开放式回复中的非结构化语言是一个劳动密集型的过程，特别是在需要快速分析且资源有限的情况下，会给使用这些语言造成障碍。计算方法可以提供帮助，但它们往往无法提供连贯的、可解释的类别，或者它们无法很好地将调查中的文本与封闭式回答联系起来。该项目将开发新的计算方法，用于快速有效地分析包括文本回复在内的调查数据，并将应用这些方法来支持组织开展与新冠肺炎回复相关的高影响力调查工作。这将提高这些组织理解和缓解新冠肺炎灾难影响的能力。该项目的技术方法建立在结合深度学习和贝叶斯主题模型的最新技术之上。将推出几项关键技术创新，专门用于提高包括封闭式和开放式答复在内的调查中可用信息的质量。这些方法中的一个共同元素是将监督学习环境中常用的方法(如基于任务的嵌入微调和知识蒸馏)扩展到非监督主题建模，重点是产生与人口统计特征、封闭式响应和实验条件等离散属性良好匹配的多样化、人类可解释的主题类别。项目活动包括协助分析组织的调查数据，进行与他们的需求相一致的独立调查以获得更多相关数据，以及公开发布干净、易于使用的计算工具包，以促进这些新方法的更广泛采用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As the COVID-19 pandemic continues, public and private organizations are deploying surveys to inform responses and policy choices. Survey designs using multiple choice responses are by far the most common -- "open ended" questions, where survey participants provide a longer-form written response, are used far less. This is true despite the fact that when you allow people to provide unconstrained spoken or text responses, it is possible to obtain richer, fine-grained information clarifying the other responses, as well as useful “bottom up” information that the survey designers did not know to ask for. A key problem is that analyzing the unstructured language in open-ended responses is a labor-intensive process, creating obstacles to using them especially when speedy analysis is needed and resources are limited. Computational methods can help, but they often fail to provide coherent, interpretable categories, or they can fail to do a good job connecting the text in the survey with the closed-end responses. This project will develop new computational methods for fast and effective analysis of survey data that includes text responses, and it will apply these methods to support organizations doing high-impact survey work related to COVID-19 response. This will improve these organizations’ ability to understand and mitigate the impact of the COVID-19 pandemic.This project’s technical approach builds on recent techniques bringing together deep learning and Bayesian topic models. Several key technical innovations will be introduced that are specifically geared toward improving the quality of information available in surveys that include both closed- and open-ended responses. A common element in these approaches is the extension of methods commonly used in supervised learning settings, such as task-based fine-tuning of embeddings and knowledge distillation, to unsupervised topic modeling, with a specific focus on producing diverse, human-interpretable topic categories that are well aligned with discrete attributes such as demographic characteristics, closed-end responses, and experimental condition. Project activities include assisting in the analysis of organizations' survey data, conducting independent surveys aligned with their needs to obtain additional relevant data, and the public release of a clean, easy to use computational toolkit facilitating more widespread adoption of these new methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.