III: Small: Data Management for Real-Time Data Driven Epidemic Spread Simulations

最近的大流行对全球产生巨大影响的速度突显了在地方和全球两级作出实时反应和公共卫生决策的重要性。例如，据估计，SARS(严重急性呼吸系统综合症)疫情于2002年11月在中国开始，到2003年8月已蔓延到29个国家，共造成916例确诊死亡。一场与2009年猪流感类似的大流行对全球经济造成的损失估计为3600亿美元，在极端情况下，仅在疫情爆发的第一年，就会造成高达4万亿美元的损失。今天，试图计划和/或应对这些疫情的决策者手中的关键武器是软件，这些软件能够实现模型驱动的流行病，以及药物和计算机模拟对疾病传播的影响。这些软件有助于根据数据和模型预测非药物控制措施和干预措施的地理时间演变，这些数据和模型包括社会接触网络、个人的本地和全球流动模式、传播和恢复率以及疫情情况。不幸的是，由于数据和模型的数量和复杂性，关键传播过程的操作和相关观测的空间和时间尺度的变化，今天运行和解释模拟以生成可行的计划是极其困难的。如果有效地利用，反映过去暴发的模型、从模拟运行中获得的现有模拟踪迹和暴发期间传入的实时观测可以共同用于更好地了解疫情的特征和潜在的扩散过程，形成和修订模型，并对流行病情景进行探索性的、如果-然后类型的假设分析。更具体地说，拟议的流行病模拟数据管理系统(EEPDMS)将解决因需要以可扩展的方式获取、建模、分析、索引、可视化、搜索和重组来自疾病暴发期间的观察和模拟的大量数据而产生的计算挑战。因此，EPDMS填补了卫生保健突发事件期间数据驱动决策的一个重要漏洞，从而使应用程序和服务具有显著的经济和健康影响。关键观察是，使用支持新环境和环境中的数据和模拟重用的数据驱动方法，可以显著减少建模和执行。基于这一观察，为了支持疫情传播模拟的数据驱动建模和执行，该团队将开发一个疫情数据和模型库(EPERSTORE)，以支持相关数据和模型的获取和集成。一种新的痕迹网络(NT)数据模型，以适应多分辨率、相互关联和相互依赖、不完整/不精确、多层(网络)和时间(时间序列或痕迹)的流行病数据。支持踪迹网络(NT)数据集索引的算法和数据结构，包括从相互依赖的参数中提取显著的多变量时间特征，这些参数跨越多个模拟层和地理空间框架，由以不同分辨率运行的复杂动态过程驱动。一种支持踪迹网络(NT)数据集分析的算法，包括识别跨越观测和模拟数据不同层的输入参数和输出变量之间的未知依赖关系。所提出的NT数据模型和算法将被集成到疫情模拟数据管理系统(EEPDMS)中。为了产生最广泛的影响，拟议的流行病模拟数据管理系统(EPEPDMS)的设计将与流行的全球流行病和流动性(GLEAM)模拟引擎接口，GLEAM模拟引擎是一种适合在全球范围内探索流行病传播情景的公开可用的软件。为了实现必要的可伸缩性，EPEPDMS将采用新颖的多分辨率数据划分和资源分配策略，并将利用大规模并行。

The speed with which recent pandemics had immense global impact highlights the importance of realtime response and public health decision making, both at local and global levels. For instance, the SARS (Severe Acute Respiratory Syndrome) epidemic is estimated to have started in China in November 2002, had spread to 29 countries by August 2003, and generated a total of 916 confirmed deaths. A pandemic similar to the swine flu in 2009 is estimated to cost $360 billion in a mild scenario to the global economy and up to $4 trillion in an ultra scenario, within just the first year of the outbreak. Today, the key arsenal in the hands of decision makers who try to plan for and/or react to these outbreaks is software that enable model-driven epidemics and as well as the impacts of pharmaceutical and computer simulations for disease spreading. These software help predict geo-temporal evolution of non-pharmaceutical control measures and interventions, relying on data and models including social contact networks, local and global mobility patterns of individuals, transmission and recovery rates, and outbreak conditions. Unfortunately, because of the volume and complexity of the data and the models, the varying spatial and temporal scales at which the key transmission processes operate and relevant observations are made, today running and interpreting simulations to generate actionable plans are extremely difficult.If effectively leveraged, models reflecting past outbreaks, existing simulation traces obtained from simulation runs, and real-time observations incoming during an outbreak can be collectively used for obtaining a better understanding of the epidemic's characteristics and the underlying diffusion processes, forming and revising models, and performing exploratory, if-then type of hypothetical analyses of epidemic scenarios. More specifically, the proposed epidemic simulation data management system (epiDMS) will address computational challenges that arise from the need to acquire, model, analyze, index, visualize, search, and recompose, in a scalable manner, large volumes of data that arise from observations and simulations during a disease outbreak. Consequently, epiDMS fill an important hole in data-driven decision making during health-care emergencies and, thus, will enable applications and services with significant economic and health impact.The key observation is that the modeling and execution can be significantly reduced using a data-driven approach that supports data and simulation reuse in new settings and contexts. Relying on this observation, in order to support data-driven modeling and execution of epidemic spread simulations, this team will develop+ an epidemic data and model store (epiStore) to support acquisition and integration of relevant data and models.+ a novel networks-of-traces (NT) data model to accommodate multi-resolution, interconnected and inter-dependent, incomplete/imprecise, multi-layer (networks), and temporal (time series or traces) epidemic data.+ algorithms and data structures to support indexing of networks-of-traces (NT) data sets, including extraction of salient multi-variate temporal features from inter-dependent parameters, spanning multiple simulation layers and geo-spatial frames, driven by complex dynamic processes operating at different resolutions.+ algorithms to support the analysis of networks-of-traces (NT) datasets, including identification of unknown dependencies across theinput parameters and output variables spanning the different layers of the observation and simulation data.The proposed NT data model and algorithms will be brought together in an epidemic simulation data management system (epiDMS). For broadest impact, the proposed epidemic simulation data management system (epiDMS) will be designed in a way that interfaces with the popular Global Epidemic and Mobility (GLEaM) simulation engine, a publicly available software suit to explore epidemic spreading scenarios at the global scale. To achieve necessary scalabilities, epiDMS will employ novel multiresolution data partitioning and resource allocation strategies and will leverage massive parallelism.