Reliable and efficient real-time tools for collecting and analyzing large health datasets

研究表明，越来越多的人依赖在线资源获取健康信息，包括症状、治疗和一般健康相关建议。此外，目前在社交媒体上活跃的数百万用户的用户行为表明，他们愿意分享与他们当前健康状况有关的事实。这些数据可用于实时跟踪和预测疾病和其他健康问题的传播，或提供关于诸如加拿大卫生部、疾病控制中心或世界卫生组织等卫生机构的公共卫生宣传战略有效性的重要信息。然而，我们目前对通过社交网络产生的在线健康数据的理解在许多重要方面是有限的：(a)现有数据库是针对项目的，数据收集机制有时间限制；(b)现有的健康跟踪工具依赖于单一界面，如Facebook、Twitter或Instagram； (c)缺乏实时绘制卫生保健问题地图的能力。***研究项目具体目的***数据收集：我们将设计一个基础设施（软件和硬件），从卫生机构和医学协会的社交媒体处理中持续收集数据，并将数据存储在网络存储系统中。***社会媒体战略有效性：卫生组织为告知公众而创建的数据点可以与普通人群的实际效果相关联。这些组织的影响力可以通过观察其媒体内容的渗透程度和传播策略的总体努力来研究，而不是在没有关注一般用户的社交媒体账户的情况下展开公共卫生事件。这就需要采用一种新的方法，通过在不同类型的用户（医疗/卫生组织、国家实验室等）之间共享的数据量来衡量社交媒体内容的吸引力，从而了解此类活动的有效性。***验证预测模型：社交媒体数据已被用于预测各种医疗行为问题和传染病。这些预测模型的主要挑战是定义基本事实。一种方法是使用众包，但这将评估限制在一个特定的问题或模型上。为了克服这一缺点，拟议的研究计划将使用多个社交媒体数据集开发真实社区算法。***社交媒体的实时分析：从各种社交媒体平台捕获的数据量可能令人望而生畏，需要大量可扩展的计算能力来整合数据流量的变化。使用分布式计算将是必要的，因此拟议的研究计划旨在使用和构建新的算法，这些算法可以通过高性能计算实验室移植到Hadoop集群上。

Background***Research suggests that more and more people rely on online sources for health information including symptoms, treatments and general health-related advice. Moreover, the user behaviour of millions of users currently active on social media demonstrates an openness to share facts related to their current health status. Such data could be used to provide real-time tracking and prediction of the spread of disease and other health concerns, or provide vital information about the effectiveness of the public health awareness strategies of health agencies such as Health Canada, the Centre for Disease Control or the World Health Organization. However, our current understanding of online health data produced through social networks is limited in important ways: (a) existing databases are project specific and data gathering mechanisms are time-constrained; (b) existing health-tracking tools depend on single interfaces such as Facebook, Twitter or Instagram, and (c) there is a lack of capacity for real-time mapping of health care issues.***Specific Aims of Research Program***Data collection: We will design an infrastructure (software and hardware) to continuously collect data from the social media handles of health agencies and medical associations, storing data on network storage systems. ***Social media strategy effectiveness: Data points created by health organizations to inform the public can be correlated to real effects in the general population. The influence of these organizations can be studied by observing the level of penetration of their media content and overall effort of their communication strategy as compared to the unfolding of public health events without following general users' social media accounts. This requires a new approach of understanding the effectiveness of such campaigns by measuring the attractiveness of the social media content by the volume of data being shared among different types of users (medical/health organizations, national laboratories, etc.). ***Validating the predictive models: Social media data has been used to predict various healthcare behavioural issues and infectious diseases. The major challenge in these predictive models is to define the ground-truth. One way is to use crowd-sourcing but this limits the evaluation to one particular problem or model. To overcome this shortcoming, the proposed research program will develop ground-truth communities algorithms using multiple social media datasets.***Real-time analysis of social media: The amount of data captured from various social media platforms could be daunting and requires large and scalable computational powers to incorporate variations in the volume of data streams. It will be necessary to use distributed computation, so the proposed research program aims to use and build new algorithms that can be ported onto a Hadoop cluster which is available through the High Performance Computing Lab.