CRII: III: Real-World Machine Learning: Adaptation Methods for Addressing Temporal, Geographic, and Demographic Confounds in User-Generated Content

有越来越多的研究使用来自网络的用户生成的内容，例如社交媒体消息，来得出关于世界的结论。使用机器学习和自然语言处理方法，可以根据人们在网上公开分享他们的想法和行动来估计公众舆论、消费者情绪和人口健康。例如，如果有人写道他们发烧了，我们可能会推断他们感染了流感；如果我们像这样汇总所有消息，我们可以跟踪人口层面上流感的流行和传播情况。然而，将机器学习应用于用户生成的内容的一个挑战是，内容的特征高度依赖于用户的人、时间和地点。在线讨论发展迅速；一年建立的系统可能在下一年不能很好地工作，为一个用户社区建立的系统可能对另一个用户社区不起作用。拟议的项目寻求创建机器学习方法，该方法对内容和内容创建者的时间、地理和人口统计的变化具有健壮性。与机器学习中的领域适应技术相关，PI提出了学习跨这些不同内容属性进行泛化的方法。总体目标是创建健壮的、开放源码的工具，以便其他研究人员可以轻松采用。该项目的一个特别成果将是改进以前基于社交媒体的疾病监测工作中使用的机器学习分类器。PI的健康分析系统的输出将整合到HealthTweets.org中，HealthTweets.org是一个可公开访问的网站，为其他研究人员和卫生官员分享疾病流行的每日估计。该项目将创建用于训练分类器的分层贝叶斯模型，这些模型可以跨不同的内容属性进行调整。感兴趣的特定属性包括作者的时间、地理和人口统计组，但所提出的模型不依赖于特定属性，可以广泛应用于其他机器学习环境。作为起点，将构建一种预测性模型(分类或回归)，该模型一次可以适应于一个属性。然后，PI将为模型创建新的扩展，可以跨时间和地点等多个属性的连接词进行调整。这些扩展与PI之前在构建结构化主题模型方面的工作有关，这些主题模型学习内容的不同特征之间的关系。最后，除了创建预测模型之外，PI还将构建可用于推断缺失属性(例如，如果未知则为用户的位置)的内容模型，其可与预测模型组合以联合执行推理和分类。将测试在新设置下对各种数据集的分类性能，以及对不同参数的影响和敏感性的探索。具体成果包括改进Twitter上用于检测流感感染的分类器，并将该分类器集成到网站HealthTweets.org中。

There is a rapidly growing body of research that uses user-generated content from the web, e.g., social media messages, to draw conclusions about the world. Using machine learning and natural language processing methods, it is possible to estimate public opinion, consumer sentiment, and population health based on what people are publicly sharing about their thoughts and actions online. For example, if someone writes that they have a fever, we might infer that they have the flu; if we aggregate all messages like this, we can track the prevalence and spread of the flu at a population level. However, a challenge with applying machine learning to user-generated content is that the characteristics of the content are highly dependent on the Who, When, and Where of the users. Online discussions evolve rapidly; a system built in one year might not work well in the next, and a system built for one community of users might not work for another. The proposed project seeks to create machine learning methods that are robust to variations in time, geography, and demographics of content and content creators. Related to domain adaptation techniques in machine learning, the PI proposes methods that learn to generalize across these various content attributes. The general goal is to create robust, open source tools that can be easily adopted by other researchers. One particular outcome of the project will be to improve the machine learning classifiers used in prior work on social media-based disease surveillance. The output of the PI's health analysis systems will be integrated into HealthTweets.org, a publicly accessible website that shares daily estimates of disease prevalence for other researchers and health officials. The project will create hierarchical Bayesian models for training classifiers that can be adapted across different content attributes. The specific attributes of interest include time, geography, and demographic group of the author, but the proposed models do not depend on the specific attributes, and can be broadly applied to other machine learning settings. As a starting point, a predictive model (classification or regression) will be constructed that can be adapted across one attribute at a time. The PI will then create novel extensions to the model that can adapt across conjunctions of multiple attributes, such as time AND location. These extensions are related to the PI's prior work on building structured topic models that learn relationships between different features of content. Finally, in addition to creating predictive models, the PI will also build models of content that can be used to infer missing attributes (e.g., the location of a user if it is unknown), which can be combined with the predictive models to jointly perform inference and classification. Classification performance in new settings on a variety of datasets and exploration of the effects of, and sensitivity to, different parameters will be tested. Specific deliverables include the improvement a classifier for detecting influenza infection on Twitter, and integrating the classifier into the website, HealthTweets.org.