Reliable and efficient real-time tools for collecting and analyzing large health datasets

研究表明，越来越多的人依赖在线来源获取健康信息，包括症状，治疗和一般健康相关建议。此外，目前活跃在社交媒体上的数百万用户的用户行为表明，他们愿意分享与其当前健康状况有关的事实。这些数据可用于实时跟踪和预测疾病和其他健康问题的传播，或提供关于加拿大卫生部、疾病控制中心或世界卫生组织等卫生机构公共卫生意识战略有效性的重要信息。然而，我们目前对通过社交网络产生的在线健康数据的理解在一些重要方面受到限制：（a）现有数据库针对具体项目，数据收集机制受到时间限制;（B）现有健康跟踪工具依赖于单一界面，如Facebook、Twitter或Instagram;（c）缺乏对保健问题进行实时绘图的能力。研究计划的具体目标 * 数据收集：我们将设计一个基础设施（软件和硬件），以持续收集来自卫生机构和医疗协会的社交媒体处理的数据，并将数据存储在网络存储系统中。* 社交媒体战略的有效性：卫生组织为告知公众而创建的数据点可以与普通人群中的真实的影响相关联。这些组织的影响力可以通过观察其媒体内容的渗透程度和其传播战略的整体努力来研究，与不关注一般用户社交媒体账户的公共卫生事件的展开进行比较。这就需要采取一种新的方法，通过不同类型的用户（医疗/卫生组织、国家实验室等）之间共享的数据量来衡量社交媒体内容的吸引力，从而了解此类运动的有效性。* 验证预测模型：社交媒体数据已被用于预测各种医疗保健行为问题和传染病。这些预测模型的主要挑战是定义地面实况。一种方法是使用众包，但这将评估限制在一个特定的问题或模型上。为了克服这一缺点，拟议的研究计划将使用多个社交媒体数据集开发地面实况社区算法。社交媒体的实时分析：从各种社交媒体平台捕获的数据量可能令人望而生畏，需要庞大且可扩展的计算能力来整合数据流量的变化。有必要使用分布式计算，因此拟议的研究计划旨在使用和构建可以移植到Hadoop集群的新算法，该集群可通过高性能计算实验室获得。

Background***Research suggests that more and more people rely on online sources for health information including symptoms, treatments and general health-related advice. Moreover, the user behaviour of millions of users currently active on social media demonstrates an openness to share facts related to their current health status. Such data could be used to provide real-time tracking and prediction of the spread of disease and other health concerns, or provide vital information about the effectiveness of the public health awareness strategies of health agencies such as Health Canada, the Centre for Disease Control or the World Health Organization. However, our current understanding of online health data produced through social networks is limited in important ways: (a) existing databases are project specific and data gathering mechanisms are time-constrained; (b) existing health-tracking tools depend on single interfaces such as Facebook, Twitter or Instagram, and (c) there is a lack of capacity for real-time mapping of health care issues.***Specific Aims of Research Program***Data collection: We will design an infrastructure (software and hardware) to continuously collect data from the social media handles of health agencies and medical associations, storing data on network storage systems. ***Social media strategy effectiveness: Data points created by health organizations to inform the public can be correlated to real effects in the general population. The influence of these organizations can be studied by observing the level of penetration of their media content and overall effort of their communication strategy as compared to the unfolding of public health events without following general users' social media accounts. This requires a new approach of understanding the effectiveness of such campaigns by measuring the attractiveness of the social media content by the volume of data being shared among different types of users (medical/health organizations, national laboratories, etc.). ***Validating the predictive models: Social media data has been used to predict various healthcare behavioural issues and infectious diseases. The major challenge in these predictive models is to define the ground-truth. One way is to use crowd-sourcing but this limits the evaluation to one particular problem or model. To overcome this shortcoming, the proposed research program will develop ground-truth communities algorithms using multiple social media datasets.***Real-time analysis of social media: The amount of data captured from various social media platforms could be daunting and requires large and scalable computational powers to incorporate variations in the volume of data streams. It will be necessary to use distributed computation, so the proposed research program aims to use and build new algorithms that can be ported onto a Hadoop cluster which is available through the High Performance Computing Lab.