Reliable and efficient real-time tools for collecting and analyzing large health datasets

研究背景*研究表明，越来越多的人依赖在线资源获取健康信息，包括症状、治疗和一般健康相关建议。此外，目前活跃在社交媒体上的数百万用户的用户行为表明，他们愿意分享与他们目前健康状况有关的事实。这类数据可用于实时跟踪和预测疾病传播和其他健康问题，或提供关于加拿大卫生部、疾病控制中心或世界卫生组织等卫生机构公共卫生宣传战略有效性的重要信息。然而，我们目前对通过社交网络产生的在线健康数据的了解在重要方面是有限的：(A)现有的数据库是特定于项目的，数据收集机制受时间限制；(B)现有的健康跟踪工具依赖于Facebook、Twitter或Instagram等单一界面，以及(C)缺乏实时绘制医疗保健问题地图的能力。*研究计划的具体目标*数据收集：我们将设计一个基础设施(软件和硬件)来持续从卫生机构和医疗协会的社交媒体句柄收集数据，并将数据存储在网络存储系统中。*社交媒体战略有效性：卫生组织为告知公众而创建的数据点可以与普通人群的实际影响相关。与不关注一般用户社交媒体账户的公共卫生事件的展开相比，可以通过观察这些组织的媒体内容的渗透程度和传播战略的整体努力来研究这些组织的影响。这就需要通过不同类型的用户(医疗/卫生组织、国家实验室等)之间共享的数据量来衡量社交媒体内容的吸引力，以此来理解这类运动的有效性。*验证预测模型：社交媒体数据已被用于预测各种医疗行为问题和传染病。这些预测模型的主要挑战是定义基本事实。一种方法是使用众包，但这将评估限制在一个特定的问题或模型上。为了克服这一缺点，拟议的研究计划将使用多个社交媒体数据集开发地面真相社区算法。*社交媒体实时分析：从各种社交媒体平台捕获的数据量可能令人望而生畏，需要巨大的可扩展计算能力来纳入数据流的变化。这将需要使用分布式计算，因此拟议的研究计划旨在使用和构建可以移植到Hadoop集群上的新算法，该集群可以通过高性能计算实验室获得。

Background***Research suggests that more and more people rely on online sources for health information including symptoms, treatments and general health-related advice. Moreover, the user behaviour of millions of users currently active on social media demonstrates an openness to share facts related to their current health status. Such data could be used to provide real-time tracking and prediction of the spread of disease and other health concerns, or provide vital information about the effectiveness of the public health awareness strategies of health agencies such as Health Canada, the Centre for Disease Control or the World Health Organization. However, our current understanding of online health data produced through social networks is limited in important ways: (a) existing databases are project specific and data gathering mechanisms are time-constrained; (b) existing health-tracking tools depend on single interfaces such as Facebook, Twitter or Instagram, and (c) there is a lack of capacity for real-time mapping of health care issues.***Specific Aims of Research Program***Data collection: We will design an infrastructure (software and hardware) to continuously collect data from the social media handles of health agencies and medical associations, storing data on network storage systems. ***Social media strategy effectiveness: Data points created by health organizations to inform the public can be correlated to real effects in the general population. The influence of these organizations can be studied by observing the level of penetration of their media content and overall effort of their communication strategy as compared to the unfolding of public health events without following general users' social media accounts. This requires a new approach of understanding the effectiveness of such campaigns by measuring the attractiveness of the social media content by the volume of data being shared among different types of users (medical/health organizations, national laboratories, etc.). ***Validating the predictive models: Social media data has been used to predict various healthcare behavioural issues and infectious diseases. The major challenge in these predictive models is to define the ground-truth. One way is to use crowd-sourcing but this limits the evaluation to one particular problem or model. To overcome this shortcoming, the proposed research program will develop ground-truth communities algorithms using multiple social media datasets.***Real-time analysis of social media: The amount of data captured from various social media platforms could be daunting and requires large and scalable computational powers to incorporate variations in the volume of data streams. It will be necessary to use distributed computation, so the proposed research program aims to use and build new algorithms that can be ported onto a Hadoop cluster which is available through the High Performance Computing Lab.