Collaborative Research: CDS&E: Investigating a Self-Assembling Data Paradigm for Detector Arrays

科学研究、安全和商业中的许多问题涉及多个地点的多个设备注册的事件。其结果是，必须收集零散的信息，并将其构建为一个连贯的整体。此外，这些事件可能会接踵而至。当事故率很高，碎片的数量很大时，问题就像是将数十个、数百个甚至数千个拼图碎片不断地倾倒在一个普通的容器中。此外，拼图碎片可能损坏或丢失，给拼图组装过程带来错误。这些挑战在计算(纳米级)自组装领域得到了很好的研究，该领域模拟了有机分子在溶液中生长晶体的过程。这个项目采用计算自组装模型来创建一种新的范例，它处理来自多个传感器的信息片段，就像分子在溶液中随机相遇和组装一样。结果是一个动态的、流动的信息块数据库，随着时间的推移不断演变，形成完整、准确的关联。这种方法被应用于收集来自超高能伽马射线天文台的望远镜阵列的数据。在这一领域成功的概念证明不仅仅是高能天体物理学家感兴趣的。这里开发的方法与其他物理领域的大数据量实验相关，并可能进一步应用于经济和安全部门的数据传输和挖掘问题。这种完全不同的分布式传感器信息容错关联方法需要一项概念验证研究，这项研究将在两年内进行。选定的测试案例是科学的。超高能伽马射线和宇宙射线在地球大气层中引发带电粒子的阵雨，这些粒子又因切伦科夫辐射效应而产生光。大气切伦科夫望远镜阵列从多个方向对流星雨发出的光进行采样，以便更准确地推断给定伽马射线的来源和能量。将来自这些望远镜的数据汇编成对单一伽马或宇宙射线流星雨的描述(事件构建)通常只需一次。由于重新访问事件构建过程对于大量(每年高达100 PB)的数据来说是不切实际的，因此错误会冻结在数据存档中。这个问题是由算法自组装范例解决的。来自运行中的伽马射线观测站Veritas的真实和模拟数据以及计划中的下一代观测站切伦科夫望远镜阵列(CTA)的模拟数据被用来发展这一概念，并迭代地为这些仪器设计、制作原型和测试简单的实施。将利用新的信号处理技术来快速提取用于关联过程的信息。使用一系列依赖于用例的基准测试来评估性能。CTA的大小，大约100个望远镜分布在一平方公里内，以及高数据速率(30G/秒)使其成为一个特别合适的测试案例，成功的概念验证可能会导致CTA采用这一模型。

A host of problems in scientific research, security, and commerce involve events registered by many devices in multiple locations. The result is fragmented information that must be gathered and built into a coherent whole. In addition, these events may come in rapid succession. When the event rate is high and the number of fragments large, the problem comes to resemble that of assembling tens, hundreds or even thousands of puzzle pieces that are continually being dumped into a common container. Further, puzzle pieces can become damaged or lost, introducing errors into the puzzle assembly process. These challenges are well-studied in the field of computational (nanoscale) self-assembly, which models processes such as the growth of crystals from organic molecules in solution. This project adapts computational self-assembly models to create a new paradigm that treats pieces of information from multiple sensors like molecules randomly meeting and assembling in solution. The result is a dynamic, fluid database of information chunks that evolve over time to form complete, accurate associations. This approach is applied to assemble data from the telescope arrays of very-high-energy gamma-ray observatories. A successful proof of concept in this domain is of interest to more than high-energy astrophysicists. The methods developed here are relevant to high data-volume experiments in other areas of physics and may have further applications to data transport and mining problems in the economic and security sectors. This radically different method of fault-tolerant association of information from distributed sensors requires a proof-of-concept study, which will take place over a two-year period. The chosen test case is scientific. Very-high-energy gamma rays and cosmic rays initiate showers of charged particles in Earth's atmosphere, which in turn produce light due to an effect known as Cherenkov radiation. Arrays of atmospheric Cherenkov telescopes sample the light from a shower from multiple directions in order to more accurately infer the origin and energy of a given gamma ray. Assembling data from these telescopes into a description of a single gamma- or cosmic-ray shower (event-building) is typically done only once. Since revisiting the event-building process is impractical for a large (up to 100 petabytes per year) volume of data, errors become frozen into the data archive. This problem is addressed by the algorithmic self-assembly paradigm. Real and simulated data from the operating gamma-ray observatory VERITAS and simulated data from a planned next-generation observatory, the Cherenkov Telescope Array (CTA), are used to develop the concept and iteratively design, prototype, and test simple implementations for these instruments. Novel signal processing techniques will be exploited to rapidly extract information used in the association process. A series of use-case-dependent benchmarks are used to assess the performance. CTA's size, roughly 100 telescopes distributed over a square kilometer, and high (30 gigabytes per second) data rates make it a particularly apt test case and a successful proof of concept could lead to adoption of this model by CTA.