Collaborative Research: CDS&E: Investigating a Self-Assembling Data Paradigm for Detector Arrays

科学研究、安全和商业中的许多问题涉及由多个位置的许多设备注册的事件。其结果是，必须收集支离破碎的信息，并将其纳入一个连贯的整体。此外，这些事件可能会接连发生。当事件发生率很高，碎片的数量很大时，问题就像是将数十、数百甚至数千块拼图碎片不断地倒入一个共同的容器中。此外，拼图块可能会损坏或丢失，从而在拼图组装过程中引入错误。这些挑战在计算（纳米级）自组装领域得到了很好的研究，该领域对溶液中有机分子的晶体生长等过程进行了建模。该项目采用计算自组装模型来创建一种新的范式，该范式将来自多个传感器的信息片段视为分子随机相遇并在溶液中组装。其结果是一个动态的、流动的信息块数据库，这些信息块随着时间的推移而演变，形成完整、准确的关联。这种方法适用于收集来自甚高能伽马射线观测站望远镜阵列的数据。在这一领域成功的概念证明不仅仅是高能天体物理学家感兴趣的。这里开发的方法是相关的高数据量实验在物理学的其他领域，并可能有进一步的应用程序的数据传输和挖掘问题的经济和安全部门。这种完全不同的分布式传感器信息容错关联方法需要进行概念验证研究，该研究将在两年时间内进行。选择的测试案例是科学的。极高能量的伽马射线和宇宙射线在地球大气层中引发带电粒子的阵雨，由于切伦科夫辐射效应，这些带电粒子又产生了光。大气切伦科夫望远镜阵列从多个方向对来自阵雨的光进行采样，以便更准确地推断给定伽马射线的起源和能量。将这些望远镜的数据组合成单个伽马射线或宇宙射线簇射（事件构建）的描述通常只需要一次。由于重新访问事件构建过程对于大量数据（每年高达100 PB）是不切实际的，因此错误被冻结在数据归档中。这个问题是解决的算法自组装范式。真实的和模拟数据从操作伽马射线天文台VERITAS和模拟数据从计划的下一代天文台，切伦科夫望远镜阵列（CTA），被用来开发的概念和迭代设计，原型，并测试这些仪器的简单实现。新的信号处理技术将被利用，以快速提取在关联过程中使用的信息。一系列依赖于用例的基准被用来评估性能。CTA的大小，大约100个望远镜分布在一平方公里，高数据速率（每秒30千兆字节）使其成为一个特别适合的测试案例，成功的概念验证可能会导致CTA采用这种模型。

A host of problems in scientific research, security, and commerce involve events registered by many devices in multiple locations. The result is fragmented information that must be gathered and built into a coherent whole. In addition, these events may come in rapid succession. When the event rate is high and the number of fragments large, the problem comes to resemble that of assembling tens, hundreds or even thousands of puzzle pieces that are continually being dumped into a common container. Further, puzzle pieces can become damaged or lost, introducing errors into the puzzle assembly process. These challenges are well-studied in the field of computational (nanoscale) self-assembly, which models processes such as the growth of crystals from organic molecules in solution. This project adapts computational self-assembly models to create a new paradigm that treats pieces of information from multiple sensors like molecules randomly meeting and assembling in solution. The result is a dynamic, fluid database of information chunks that evolve over time to form complete, accurate associations. This approach is applied to assemble data from the telescope arrays of very-high-energy gamma-ray observatories. A successful proof of concept in this domain is of interest to more than high-energy astrophysicists. The methods developed here are relevant to high data-volume experiments in other areas of physics and may have further applications to data transport and mining problems in the economic and security sectors. This radically different method of fault-tolerant association of information from distributed sensors requires a proof-of-concept study, which will take place over a two-year period. The chosen test case is scientific. Very-high-energy gamma rays and cosmic rays initiate showers of charged particles in Earth's atmosphere, which in turn produce light due to an effect known as Cherenkov radiation. Arrays of atmospheric Cherenkov telescopes sample the light from a shower from multiple directions in order to more accurately infer the origin and energy of a given gamma ray. Assembling data from these telescopes into a description of a single gamma- or cosmic-ray shower (event-building) is typically done only once. Since revisiting the event-building process is impractical for a large (up to 100 petabytes per year) volume of data, errors become frozen into the data archive. This problem is addressed by the algorithmic self-assembly paradigm. Real and simulated data from the operating gamma-ray observatory VERITAS and simulated data from a planned next-generation observatory, the Cherenkov Telescope Array (CTA), are used to develop the concept and iteratively design, prototype, and test simple implementations for these instruments. Novel signal processing techniques will be exploited to rapidly extract information used in the association process. A series of use-case-dependent benchmarks are used to assess the performance. CTA's size, roughly 100 telescopes distributed over a square kilometer, and high (30 gigabytes per second) data rates make it a particularly apt test case and a successful proof of concept could lead to adoption of this model by CTA.