Frameworks: SCiMMA: Real-time Orchestration of Multi-Messenger Astrophysical Observations

在引力波(GW170817)中探测到的第一个中子星合并(GW170817)，相关的瞬发伽马射线爆发和千诺瓦余辉，为宇宙打开了一扇新的窗口，揭示了天体物理的许多领域，包括恒星演化、中子星状态方程、宇宙学、核合成、重元素的不透明度、伽马射线爆发和喷流物理、星系演化、广义相对论测试等。这只是多信使天体物理学(MMA)力量的一个例子，在MMA中，电磁信息与感知宇宙的方式完全不同，要么是通过粒子探测，要么是通过引力波。MMA活动带来了独特的挑战--将以前孤立的群体聚集在一起，需要不同子领域之间的标准化，快速反应，以及有限资源的全球协调。这就需要对目前使用的过时的通信基础设施进行彻底改革，相对适度的投资可以大大提高LIGO-Virgo-KAGRA(LVK)合作、冰立方中微子实验和Vera C.Rubin天文台等实验的数十亿美元投资的效率。SCIMMA团队正在开发基础设施，将这些孤立的设施及其数据实时整合在一起，首次实现了一个协调的国家天体物理生态系统。这一基础设施将提高识别引力波、高能粒子和中微子实验的电磁对应物的效率，允许在高能物理和天文学的交叉点上有更多新的发现，并提供探索引力本质的新方法。在之前的工作中，SCIMMA团队对社区进行了调查，并开发了满足他们需求的工具，如基于Kafka的强大、快速和可扩展的消息系统Hopskotch。在此基础上，SCIMMA分层了Hopskotch启用的实时消息服务(Hermes)，这是一个基于Web的图形用户界面和相关的API。这解决了标准化问题，既简单易用(不需要安装软件)，又足够复杂，可以通过API实现新类型的自动化。通过Hopskotch框架和Hermes服务，SCIMMA将一个完全不同的设施网络整合在一起，在LVK第四次观测运行期间及以后协调后续工作。Hermes融合了人和机器的可读性，这是机器人和基于队列的望远镜所必需的关键基础设施，而机器人和基于队列的望远镜对于MMA事件的快速跟进至关重要。Hopskotch正在使用中，并且已成为LIGO内部使用的关键，包括在O4观测运行中分发LIGO-Virgo-KAGRA(LVK)警报。SCIMMA正在将Hopskotch从原型带到生产状态，并根据社区的要求添加新功能。新功能包括将LVK连接到IceCube等粒子探测器，IceCube又将连接和互连地面观测设施，如即将到来的Vera C.Rubin天文台。这建立在天体物理多信使天文台网络(AMON)和天文事件观测网络(AEON)等协会的基础上。这些工作包括：(1)一个DEVOPS团队，以确保持续的正常运行时间，安全，并收集反馈；(2)身份和访问管理系统的升级，以允许私人主题，以便分组团队可以使用与他们在公共系统上相同的基础设施进行通信；(3)继续开发Hopskotch和Hermes，使其从原型转变为行业领先的通信和标准化工具；(4)与现有的基础设施、天文设施以及NASA GCNS、临时名称服务器、宝藏地图、经纪人和TOM工具包等服务相接口。总而言之，这些改进减少了观测中的冗余，允许更快、更频繁地发现电磁对应物，并使天文台更加高效。此外，Hermes降低了规模较小或资源较差的社区面临的进入门槛。SCIMMA的目标是让全球每个人都能流畅地实时访问MMA活动的所有相关公共数据，使所有研究人员更容易地促进发现，从而使科学民主化。该项目推进了“宇宙之窗：多信使天体物理时代”的目标，未来NSF投资的十大想法之一。这一奖项由高级网络基础设施办公室获得，由天文科学部和数学和物理科学局内物理部信息前沿项目的物理学共同支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The first neutron star merger detected in gravitational waves (GW170817), the associated prompt gamma ray burst, and kilonova afterglow, opened a new window on the universe revealing insights into many areas of astrophysics, including stellar evolution, the neutron star equation-of-state, cosmology, nucleosynthesis, the opacity of heavy elements, gamma-ray burst and jet physics, galactic evolution, tests of general relativity, and more. This is but one example of the power of multimessenger astrophysics (MMA), where electromagnetic information is combined with sensing the universe in completely different ways, either through the detections of particles or gravitational waves. MMA events present unique challenges - bringing together previously siloed groups and requiring standardization between different subfields, rapid response, and global coordination of limited resources. This necessitates an overhaul in the outdated communications infrastructure currently employed, where a relatively modest investment can produce huge gains in the efficiency of billion-dollar investments for experiments such as the LIGO-VIRGO-Kagra (LVK) collaboration, the IceCube Neutrino Experiment, and the Vera C. Rubin Observatory. The SCIMMA team is developing the infrastructure to bring together these siloed facilities and their data in real-time, enabling for the first time a coordinated national astrophysical ecosystem. This infrastructure will improve the efficiency of identifying the electromagnetic counterparts to gravitational waves, high energy particle and neutrino experiments, permitting more novel discoveries at the intersection of high-energy physics and astronomy, and providing new ways to probe the nature of gravity itself.In previous work, the SCIMMA team surveyed the community and developed tools to serve their needs, such as Hopskotch, a robust, rapid, and scalable Kafka-based messaging system. On top of that SCIMMA has layered the Hopskotch Enabled Real-time Messaging Service (HERMES), a web-based graphical user interface and associated API. This solves the problem of standardization and is both simple to use (no software needs to be installed), and yet complex enough to achieve new kinds of automation through APIs. With the Hopskotch framework and HERMES service, SCIMMA brings together a disparate network of facilities, orchestrating follow-up during the LVK fourth observing run and beyond. HERMES blends human and machine readability, a critical piece of missing infrastructure necessary for the robotic and queue-based telescopes that are essential for the rapid follow-up of MMA events. Hopskotch is in use, and has become critical for internal LIGO use, including the distribution of LIGO-Virgo-KAGRA (LVK) alerts in the O4 observing run. SCIMMA is taking Hopskotch from a prototype to a production state and adding new features as requested by the community. New features include connecting LVK to particle detectors such as IceCube, which in turn will be connected to and interconnect ground-based observing facilities such as the upcoming Vera C. Rubin Observatory. This builds on associations like the Astrophysical Multi-messenger Observatory Network (AMON), and the Astronomical Event Observatory Network (AEON). These efforts include: (1) a devops team to insure continuous uptime, security, and gather feedback; (2) upgrades to the Identity and Access Management system to allow private topics so that sub-teams can use the same infrastructure to communicate as they would on the public system; (3) the continued development of Hopskotch and HERMES to take it from prototype to an industry-leading communications and standardization tool; (4) interfacing with existing infrastructure, both astronomical facilities, as well as services such as NASA GCNs, the Transient Name Server, Treasure Map, brokers, and the TOM Toolkit. Together, these improvements reduce redundancy in observations, allow for more rapid and frequent discovery of electromagnetic counterparts, and make observatories more efficient. Additionally, HERMES lowers the barriers to entry faced by smaller or less well-resourced communities. SCIMMA’s aim is to give everyone across the globe streamlined real-time access to all the relevant public data for MMA events to make it increasingly easier for all researchers to facilitate discovery, thereby democratizing science.This project advances the objectives of "Windows on the Universe: the Era of Multi-Messenger Astrophysics", one of the 10 Big Ideas for Future NSF Investments.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Astronomical Sciences and the Physics at the Information Frontier program in the Division of Physics within the Directorate for Mathematical and Physical Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.