CDI-Type II: Real-time Classification of Massive Time-series Data Streams

一般来说，在任何学科中收集新的数据并不会导致新知识的产生。 随着当前数据的泛滥，人类在科学发现中的作用，传统上如此重要，现在必须通过强大的算法来部分实现。 然而，当前的工具和技术开始崩溃，因为发现和理解，根据手头的科学的本质，必须快速和近实时地发生。未来几年将上线的新天文观测，许多观测会在同一时间重复观测天空的同一区域，在未来十年收集的数据将比迄今为止人类历史上所有的数据都要多。 在动态宇宙中开辟真正的新前景需要快速的数据处理和关于哪些可用资源的快速决策（例如，世界各地的天文望远镜必须被召集起来研究新发现的现象。 这需要一个智能的“实时”基于机器的决策或“分类”框架，该框架应该能够处理不完整的（在某些情况下是虚假的）信息。该项目将产生一个框架，用于在计算需求远远超过可用设施的环境中从大量数据中提取新的科学，并且需要智能（以及动态）资源分配。 将开发新的理论，使当前的机器学习范式能够扩展到大型并行计算环境。 其核心结果是，对于每晚产生数千千兆字节新数据的项目（如拟议中的大型综合巡天望远镜），产生了关于天文事件物理性质的概率陈述。 这项工作将特别强调发现不容易纳入目前公认的分类法的异常事件--可能导致全新科学发现的事件。现在考虑到具体的科学回报，构建这些计算工具将为具有类似需求和限制的其他领域的更快速变革性应用奠定基础（高频金融数据、机器人技术、医疗信号监测、地球物理学、天气和粒子物理学）。 这一奋进也将多年来作为跨几个部门和学科的学生和研究人员的培训基地，并将扩大他们的范围，走向真正的跨学科教育。 通过让物理科学领域的学生接触到前沿的计算机科学和机器学习概念，该项目将为计算思维提供一个框架，这将导致未来的创新。

The collection of new data in any discipline does not, in general, lead to the creation of new knowledge. With the current data deluge, the human role in scientific discovery, traditionally so important, must now be partially fulfilled by powerful algorithms. However, current tools and technology start to break down when discovery and understanding, by the very nature of the science at hand, must happen quickly and in near real-time.New astronomical surveys coming online in the next few years, many observing the same regions of the sky repeatedly in time, will collect more data in the next decade than in all of human history so far. Opening up truly new vistas on the dynamic universe requires both rapid data processing and quick decisions about what available resources (e.g., telescopes) worldwide must be marshalled to study newly discovered phenomena. This necessitates an intelligent "real-time" machine-based decision or "classification" framework that should be able to deal with incomplete (and in some cases spurious) information.This project will produce a framework for extracting novel science from large amounts of data in an environment where the computational needs vastly outweigh the available facilities, and intelligent (as well as dynamic) resource allocation is required. New theory will be developed that will allow current machine learning paradigms to scale to large parallel computing environments. The core result is the production, for projects generating thousands of gigabytes of new data a night (such as the proposed Large Synoptic Survey Telescope), of probabilistic statements about the physical nature of astronomical events. Uncovering anomalous events that do not fit easily into a currently accepted classification taxonomy - events that may lead to completely new scientific discoveries - will be particularly emphasized in this work.Building these computational tools now with concrete scientific returns in mind will form the foundation for more rapid transformative applications in other fields with similar demands and constraints (high-frequency financial data, robotics, medical signal monitoring, geophysics, weather, and particle physics). This endeavor will also serve for years as a training ground for students and researchers across several departments and disciplines, and will broaden their scope towards a truly interdisciplinary education. By exposing students in the physical sciences to cutting-edge computer science and machine learning concepts, this project will provide a frame-work for computational thinking that will lead to future innovation.