III: Small: Data Management for Real-Time Data Driven Epidemic Spread Simulations

最近的流行病迅速产生了巨大的全球影响，这突出表明了在地方和全球各级作出实时反应和公共卫生决策的重要性。例如，SARS（严重急性呼吸系统综合症）疫情估计于2002年11月在中国开始，到2003年8月已蔓延到29个国家，共造成916人确诊死亡。 一场类似于2009年猪流感的大流行病，估计在爆发的第一年内，在温和的情况下将给全球经济造成3600亿美元的损失，在极端情况下将高达4万亿美元。 今天，决策者手中的关键武器库试图计划和/或应对这些疫情是软件，使模型驱动的流行病，以及药物和计算机模拟疾病传播的影响。这些软件有助于预测非药物控制措施和干预措施的地理时间演变，依赖于数据和模型，包括社会联系网络，个人的地方和全球流动模式，传播和恢复率以及爆发条件。 不幸的是，由于数据和模型的数量和复杂性，关键传播过程的运行和相关观察的空间和时间尺度各不相同，今天运行和解释模拟以生成可操作的计划是极其困难的。如果有效利用，反映过去爆发的模型，从模拟运行中获得的现有模拟痕迹，在疫情爆发期间获得的实时观测结果可共同用于更好地了解流行病的特征和潜在的扩散过程，形成和修订模型，并对流行病情景进行探索性的if-then类型的假设分析。 更具体地说，所提出的流行病模拟数据管理系统（epiDMS）将解决由于需要以可扩展的方式获取、建模、分析、索引、可视化、搜索和重组大量数据而产生的计算挑战，这些数据来自疾病爆发期间的观察和模拟。因此，epiDMS填补了医疗紧急情况下数据驱动决策的一个重要漏洞，因此，将使应用程序和服务具有重大的经济和健康影响。关键的观察结果是，使用数据驱动的方法，支持在新的设置和环境中重用数据和模拟，可以大大减少建模和执行。基于这一观察，为了支持数据驱动的建模和执行流行病传播模拟，该团队将开发+流行病数据和模型存储（epiStore），以支持相关数据和模型的获取和集成。一种新的网络痕迹（NT）数据模型，以适应多分辨率，相互关联和相互依赖，不完整/不精确，多层（网络）和时间（时间序列或痕迹）流行病数据。算法和数据结构，以支持网络的痕迹（NT）数据集的索引，包括提取显着的多变量的时间特征，从相互依赖的参数，跨越多个模拟层和地理空间框架，由复杂的动态过程在不同的分辨率操作驱动。算法来支持网络的痕迹（NT）数据集的分析，包括识别未知的依赖关系，在输入参数和输出变量跨越不同层次的观察和模拟数据。提出的NT数据模型和算法将被带到一起，在一个流行病模拟数据管理系统（epiDMS）。为了产生最广泛的影响，拟议的流行病模拟数据管理系统（epiDMS）的设计方式将与流行的全球流行病和流动性（GLEaM）模拟引擎接口，这是一个公开的软件套装，用于探索全球范围内的流行病传播情景。 为了实现必要的可扩展性，epiDMS将采用新的多分辨率数据划分和资源分配策略，并将利用大规模并行性。

The speed with which recent pandemics had immense global impact highlights the importance of realtime response and public health decision making, both at local and global levels. For instance, the SARS (Severe Acute Respiratory Syndrome) epidemic is estimated to have started in China in November 2002, had spread to 29 countries by August 2003, and generated a total of 916 confirmed deaths. A pandemic similar to the swine flu in 2009 is estimated to cost $360 billion in a mild scenario to the global economy and up to $4 trillion in an ultra scenario, within just the first year of the outbreak. Today, the key arsenal in the hands of decision makers who try to plan for and/or react to these outbreaks is software that enable model-driven epidemics and as well as the impacts of pharmaceutical and computer simulations for disease spreading. These software help predict geo-temporal evolution of non-pharmaceutical control measures and interventions, relying on data and models including social contact networks, local and global mobility patterns of individuals, transmission and recovery rates, and outbreak conditions. Unfortunately, because of the volume and complexity of the data and the models, the varying spatial and temporal scales at which the key transmission processes operate and relevant observations are made, today running and interpreting simulations to generate actionable plans are extremely difficult.If effectively leveraged, models reflecting past outbreaks, existing simulation traces obtained from simulation runs, and real-time observations incoming during an outbreak can be collectively used for obtaining a better understanding of the epidemic's characteristics and the underlying diffusion processes, forming and revising models, and performing exploratory, if-then type of hypothetical analyses of epidemic scenarios. More specifically, the proposed epidemic simulation data management system (epiDMS) will address computational challenges that arise from the need to acquire, model, analyze, index, visualize, search, and recompose, in a scalable manner, large volumes of data that arise from observations and simulations during a disease outbreak. Consequently, epiDMS fill an important hole in data-driven decision making during health-care emergencies and, thus, will enable applications and services with significant economic and health impact.The key observation is that the modeling and execution can be significantly reduced using a data-driven approach that supports data and simulation reuse in new settings and contexts. Relying on this observation, in order to support data-driven modeling and execution of epidemic spread simulations, this team will develop+ an epidemic data and model store (epiStore) to support acquisition and integration of relevant data and models.+ a novel networks-of-traces (NT) data model to accommodate multi-resolution, interconnected and inter-dependent, incomplete/imprecise, multi-layer (networks), and temporal (time series or traces) epidemic data.+ algorithms and data structures to support indexing of networks-of-traces (NT) data sets, including extraction of salient multi-variate temporal features from inter-dependent parameters, spanning multiple simulation layers and geo-spatial frames, driven by complex dynamic processes operating at different resolutions.+ algorithms to support the analysis of networks-of-traces (NT) datasets, including identification of unknown dependencies across theinput parameters and output variables spanning the different layers of the observation and simulation data.The proposed NT data model and algorithms will be brought together in an epidemic simulation data management system (epiDMS). For broadest impact, the proposed epidemic simulation data management system (epiDMS) will be designed in a way that interfaces with the popular Global Epidemic and Mobility (GLEaM) simulation engine, a publicly available software suit to explore epidemic spreading scenarios at the global scale. To achieve necessary scalabilities, epiDMS will employ novel multiresolution data partitioning and resource allocation strategies and will leverage massive parallelism.