SBIR Phase I: Cloud computing for real-time processing of photon light source data

该SBIR第一阶段项目建议使用商业云计算资源对光子光源设施收集的X射线计算机断层扫描数据进行实时处理。光子光源收集的数据量有望大幅增加，这是由于探测器的改进，以实现更快速的数据收集，以及基础加速器技术的进步，以实现更大的光子生产。因此，在科学家的个人工作站或小型计算集群上存储和处理数据集将变得不切实际，因为数据集大小继续以比摩尔定律快得多的速度增长。拟议的解决方案是在云中开发一个持久的托管计算集群，该集群可以增长以满足科学家的计算和存储需求，而无需他们处理管理物理硬件。X射线计算机断层扫描被用作测试应用程序，可以开发自动化工作流程，在实验过程中以最少的用户干预存储和分析收集的断层扫描数据。云中持久的托管计算资源的更广泛/商业影响为科学家提供了一种经济实惠的工具，可以满足他们的计算和存储需求，而无需管理物理硬件。存储和分析数据是科学研究的基本日常任务。随着技术提高数据收集率和数据保真度，科学家们正在处理越来越大的数据集。这种增长的一些例子包括基因组学，其中基因组测序的成本急剧下降，天文学，强大的望远镜可以快速捕获高分辨率图像，以及高能物理学，其中数据集需要实时减少，以便其大小更易于管理。这种急剧增长导致了科学家面临的“大数据”问题，因为他们的传统数据处理工具不再足以处理这些数据集。通过利用云服务提供商实现的规模经济，Billow可以以较低的成本提供可靠的存储和高可用性计算。这种方法使科学家能够专注于科学，为他们提供必要的工具来分析数据并开发新的数据分析算法。

This SBIR Phase I project proposes to use commercial cloud computing resources for real-time processing of X-ray computed tomography data collected at a photon light source facility. The amount of data being collected at photon light sources is poised to increase dramatically due to improvements in detectors for more rapid data collection and advancements in the underlying accelerator technology for greater production of photons. As a result, storing and processing data sets on a scientist's personal workstation or small computing cluster will become impractical as data set sizes continue to grow much faster than Moore's law. The proposed solution is to develop a persistent, managed computing cluster in the cloud that can grow to meet the computational and storage needs of scientists without them having to deal with managing physical hardware. X-ray computed tomography is used as a test application where an automated workflow can be developed that will store and analyze the collected tomography data during the experiment with minimal user intervention.The broader/commercial impact of a persistent, managed computing resource in the cloud is providing scientists an affordable tool that can grow to meet their computational and storage needs without the hassle of managing physical hardware. Storing and analyzing data are routine tasks that are fundamental to scientific research. As technology improves data collection rates and data fidelity, scientists are dealing with larger and larger data sets. Some examples of this growth include genomics, where the cost to sequence a genome has dropped precipitously, astronomy, where powerful telescopes can quickly capture high resolution images, and high energy physics, where data sets need to be reduced in real-time so that their sizes are more manageable. This dramatic increase is contributing to the "Big Data" problem that scientists are facing as their traditional data processing tools are no longer sufficient to tackle these data sets. By taking advantage of the economies of scale achieved by cloud service providers, Billow can provide reliable storage and high availability computing at a lower cost. This approach allows scientists to focus on the science by giving them the necessary tools to analyze their data and develop new data analysis algorithms.