Collaborative Research: CDS&E: Investigating a Self-Assembling Data Paradigm for Detector Arrays

科学研究、安全和商业中的许多问题都涉及到由多个地点的许多设备记录的事件。其结果是碎片化的信息必须被收集并构建成一个连贯的整体。此外，这些事件可能会接二连三地发生。当事件率高且碎片数量大时，问题就变得类似于组装数十、数百甚至数千个不断被扔进一个普通容器的拼图。此外，拼图碎片可能会损坏或丢失，从而在拼图组装过程中引入错误。这些挑战在计算（纳米尺度）自组装领域得到了很好的研究，该领域模拟了有机分子在溶液中生长晶体等过程。这个项目采用计算自组装模型来创建一个新的范例，它处理来自多个传感器的信息片段，就像分子在溶液中随机相遇和组装一样。其结果是一个动态的、流动的信息块数据库，随着时间的推移而演变，形成完整、准确的关联。这种方法被应用于收集来自高能伽玛射线天文台的望远镜阵列的数据。在这个领域，一个成功的概念证明不仅仅是高能天体物理学家感兴趣的。这里制定的方法与物理其他领域的高数据量实验有关，并可能进一步应用于经济和安全部门的数据传输和挖掘问题。这种对来自分布式传感器的信息进行容错关联的完全不同的方法需要进行概念验证研究，这将在两年的时间内进行。所选择的测试用例是科学的。高能伽马射线和宇宙射线在地球大气中引发带电粒子阵雨，这些粒子又由于切伦科夫辐射效应而产生光。大气切伦科夫望远镜阵列从多个方向对流星雨的光线进行采样，以便更准确地推断出给定伽马射线的来源和能量。将来自这些望远镜的数据汇总成对单个伽马射线或宇宙射线阵雨的描述（事件构建）通常只需要完成一次。由于对于大量（每年高达100 pb）的数据来说，重新访问事件构建过程是不切实际的，因此错误被冻结在数据存档中。算法自组装范式解决了这个问题。来自运行中的伽马射线天文台VERITAS的真实和模拟数据，以及来自计划中的下一代天文台切伦科夫望远镜阵列（CTA）的模拟数据，用于开发这些仪器的概念和迭代设计、原型和测试简单实现。新的信号处理技术将被用于快速提取关联过程中使用的信息。使用一系列与用例相关的基准来评估性能。CTA的规模，大约100个望远镜分布在一平方公里上，高数据速率（每秒30千兆字节）使它成为一个特别合适的测试案例，一个成功的概念证明可能会导致CTA采用这种模式。

A host of problems in scientific research, security, and commerce involve events registered by many devices in multiple locations. The result is fragmented information that must be gathered and built into a coherent whole. In addition, these events may come in rapid succession. When the event rate is high and the number of fragments large, the problem comes to resemble that of assembling tens, hundreds or even thousands of puzzle pieces that are continually being dumped into a common container. Further, puzzle pieces can become damaged or lost, introducing errors into the puzzle assembly process. These challenges are well-studied in the field of computational (nanoscale) self-assembly, which models processes such as the growth of crystals from organic molecules in solution. This project adapts computational self-assembly models to create a new paradigm that treats pieces of information from multiple sensors like molecules randomly meeting and assembling in solution. The result is a dynamic, fluid database of information chunks that evolve over time to form complete, accurate associations. This approach is applied to assemble data from the telescope arrays of very-high-energy gamma-ray observatories. A successful proof of concept in this domain is of interest to more than high-energy astrophysicists. The methods developed here are relevant to high data-volume experiments in other areas of physics and may have further applications to data transport and mining problems in the economic and security sectors. This radically different method of fault-tolerant association of information from distributed sensors requires a proof-of-concept study, which will take place over a two-year period. The chosen test case is scientific. Very-high-energy gamma rays and cosmic rays initiate showers of charged particles in Earth's atmosphere, which in turn produce light due to an effect known as Cherenkov radiation. Arrays of atmospheric Cherenkov telescopes sample the light from a shower from multiple directions in order to more accurately infer the origin and energy of a given gamma ray. Assembling data from these telescopes into a description of a single gamma- or cosmic-ray shower (event-building) is typically done only once. Since revisiting the event-building process is impractical for a large (up to 100 petabytes per year) volume of data, errors become frozen into the data archive. This problem is addressed by the algorithmic self-assembly paradigm. Real and simulated data from the operating gamma-ray observatory VERITAS and simulated data from a planned next-generation observatory, the Cherenkov Telescope Array (CTA), are used to develop the concept and iteratively design, prototype, and test simple implementations for these instruments. Novel signal processing techniques will be exploited to rapidly extract information used in the association process. A series of use-case-dependent benchmarks are used to assess the performance. CTA's size, roughly 100 telescopes distributed over a square kilometer, and high (30 gigabytes per second) data rates make it a particularly apt test case and a successful proof of concept could lead to adoption of this model by CTA.