III: Small: Data Management for Real-Time Data Driven Epidemic Spread Simulations

最近的流行病迅速对全球产生了巨大影响，这凸显了在地方和全球各级实时应对和公共卫生决策的重要性。例如，SARS（严重急性呼吸系统综合症）流行病估计于2002年11月在中国开始，到2003年8月已蔓延到29个国家，共造成916人确认死亡。据估计，一场类似2009年猪流感的大流行，在温和的情况下会给全球经济造成3600亿美元的损失，在极端的情况下，仅在疫情爆发的第一年，就会给全球经济造成高达4万亿美元的损失。今天，试图规划和/或应对这些疫情的决策者手中的关键武器是软件，它可以实现模型驱动的流行病，以及药物和计算机模拟疾病传播的影响。这些软件依靠数据和模型，包括社会联系网络、个人的地方和全球流动模式、传播和恢复率以及疫情条件，帮助预测非药物控制措施和干预措施的地理-时间演变。不幸的是，由于数据和模型的数量和复杂性，以及关键传输过程运行和相关观测的空间和时间尺度的变化，今天运行和解释模拟以生成可操作的计划是极其困难的。如果得到有效利用，反映过去疫情的模型、从模拟运行中获得的现有模拟痕迹以及疫情期间的实时观测结果可以共同用于更好地了解疫情的特征和潜在的扩散过程，形成和修改模型，并对疫情情景进行探索性的假设分析。更具体地说，拟议的流行病模拟数据管理系统（epiDMS）将解决因需要以可扩展的方式获取、建模、分析、索引、可视化、搜索和重组疾病爆发期间观察和模拟产生的大量数据而产生的计算挑战。因此，流行病综合管理系统填补了紧急卫生保健期间数据驱动决策的一个重要漏洞，从而使应用和服务能够产生重大的经济和健康影响。关键的观察结果是，使用支持在新设置和上下文中重用数据和模拟的数据驱动方法，可以显著减少建模和执行。基于这一观察结果，为了支持数据驱动的流行病传播模拟建模和执行，该团队将开发一个流行病数据和模型存储（epiStore），以支持相关数据和模型的获取和集成。+一种新的轨迹网络（NT）数据模型，以适应多分辨率、相互关联和相互依赖、不完整/不精确、多层（网络）和时间（时间序列或轨迹）流行病数据。+算法和数据结构，以支持网络轨迹（NT）数据集的索引，包括从相互依赖的参数中提取显著的多变量时间特征，跨越多个模拟层和地理空间框架，由不同分辨率下运行的复杂动态过程驱动。+算法，以支持网络轨迹（NT）数据集的分析，包括识别跨输入参数和输出变量的未知依赖关系，跨越观察和模拟数据的不同层。建议的NT数据模型和算法将被整合到流行病模拟数据管理系统（epiDMS）中。为了产生最广泛的影响，拟议的流行病模拟数据管理系统（epiDMS）将以与流行的全球流行病和流动性（GLEaM）模拟引擎接口的方式设计，GLEaM是一个公开可用的软件套件，用于探索全球范围内的流行病传播场景。为了实现必要的可扩展性，epiDMS将采用新颖的多分辨率数据分区和资源分配策略，并将利用大规模并行性。

The speed with which recent pandemics had immense global impact highlights the importance of realtime response and public health decision making, both at local and global levels. For instance, the SARS (Severe Acute Respiratory Syndrome) epidemic is estimated to have started in China in November 2002, had spread to 29 countries by August 2003, and generated a total of 916 confirmed deaths. A pandemic similar to the swine flu in 2009 is estimated to cost $360 billion in a mild scenario to the global economy and up to $4 trillion in an ultra scenario, within just the first year of the outbreak. Today, the key arsenal in the hands of decision makers who try to plan for and/or react to these outbreaks is software that enable model-driven epidemics and as well as the impacts of pharmaceutical and computer simulations for disease spreading. These software help predict geo-temporal evolution of non-pharmaceutical control measures and interventions, relying on data and models including social contact networks, local and global mobility patterns of individuals, transmission and recovery rates, and outbreak conditions. Unfortunately, because of the volume and complexity of the data and the models, the varying spatial and temporal scales at which the key transmission processes operate and relevant observations are made, today running and interpreting simulations to generate actionable plans are extremely difficult.If effectively leveraged, models reflecting past outbreaks, existing simulation traces obtained from simulation runs, and real-time observations incoming during an outbreak can be collectively used for obtaining a better understanding of the epidemic's characteristics and the underlying diffusion processes, forming and revising models, and performing exploratory, if-then type of hypothetical analyses of epidemic scenarios. More specifically, the proposed epidemic simulation data management system (epiDMS) will address computational challenges that arise from the need to acquire, model, analyze, index, visualize, search, and recompose, in a scalable manner, large volumes of data that arise from observations and simulations during a disease outbreak. Consequently, epiDMS fill an important hole in data-driven decision making during health-care emergencies and, thus, will enable applications and services with significant economic and health impact.The key observation is that the modeling and execution can be significantly reduced using a data-driven approach that supports data and simulation reuse in new settings and contexts. Relying on this observation, in order to support data-driven modeling and execution of epidemic spread simulations, this team will develop+ an epidemic data and model store (epiStore) to support acquisition and integration of relevant data and models.+ a novel networks-of-traces (NT) data model to accommodate multi-resolution, interconnected and inter-dependent, incomplete/imprecise, multi-layer (networks), and temporal (time series or traces) epidemic data.+ algorithms and data structures to support indexing of networks-of-traces (NT) data sets, including extraction of salient multi-variate temporal features from inter-dependent parameters, spanning multiple simulation layers and geo-spatial frames, driven by complex dynamic processes operating at different resolutions.+ algorithms to support the analysis of networks-of-traces (NT) datasets, including identification of unknown dependencies across theinput parameters and output variables spanning the different layers of the observation and simulation data.The proposed NT data model and algorithms will be brought together in an epidemic simulation data management system (epiDMS). For broadest impact, the proposed epidemic simulation data management system (epiDMS) will be designed in a way that interfaces with the popular Global Epidemic and Mobility (GLEaM) simulation engine, a publicly available software suit to explore epidemic spreading scenarios at the global scale. To achieve necessary scalabilities, epiDMS will employ novel multiresolution data partitioning and resource allocation strategies and will leverage massive parallelism.