Collaborative Research: Online Monitoring of High-Dimensional Streaming Data Using Adaptive Order Shrinkage

现代制造机器和系统采用传感器来监控工艺条件，但来自它们的海量数据往往很难解释。这个项目研究了一种从这些数据中提取有用信息的新方法。该方法潜在地可以通过采用专门用于高维流数据的在线监控的统计过程控制(SPC)方法来降低计算成本并提高对预测的置信度。虽然这项工作的重点是监测化学机械平坦化(CMP)过程，但这些方法也可以应用于其他制造应用，如轧制、锻造和铸造过程。这些方法的影响将超越制造业，包括但不限于流行病学中的疾病监测、网络流量控制、入侵检测和监控视频。研究方法的成功实施不仅将对国家产生重大的经济影响，还将通过快速检测异常事件来防止随之而来的损害。此外，教育计划将通过课程和实验室开发、教学创新和其他外展活动对劳动力培训产生广泛影响。这项合作研究的目标是开发可扩展和自适应的方法，用于在线监控高维流数据。具体来说，在方法论的发展中计划了三个相互关联的研究任务：(1)基于自适应阶数收缩和全观测的高效可扩展方案，其关键创新思想是首先通过一些经典的、计算简单但高效的局部检测统计来对每个数据流进行局部监测，然后将这些局部过程巧妙地结合在一起。为了产生单一的全球监测方案；(2)在空间域而不是传统的时间域上采用自适应采样策略，以便在考虑资源限制的情况下，主动选择/采样信息量最大的数据流，以最大限度地提高变化检测的灵敏度和有效性；以及(3)工程知识增强型监测方案，其将领域知识与局部检测统计发展和自适应采样策略相结合，以进一步提高性能。这项研究的成功将促进统计过程控制的发展，并为制造系统质量改进的科学基础做出贡献。

Modern manufacturing machines and systems incorporate sensors to monitor process conditions, but the massive amount of data coming from them is often difficult to interpret. This project investigates a new method for extracting useful information from such data. The method potentially can reduce computational cost and improve confidence in the predictions that are made by adapting statistical process control (SPC) methodologies specifically for online monitoring of high-dimensional streaming data. While this work focuses on monitoring the Chemical Mechanical Planarization (CMP) process, the methods can also be applied to other manufacturing applications, such as rolling, forging and casting processes. The impacts of the methodologies will go beyond manufacturing, including but not limited to disease surveillance in epidemiology, network traffic control, intrusion detection and surveillance video. The success of the implementation of the research methods will not only generate significant economic impacts to the nation, but also prevent consequent damages through quick detection of abnormal events. In addition, the education plan will make broad impacts on the workforce training through curriculum and lab developments, teaching innovations, and other outreach activities.The objective of this collaborative research is to develop scalable and adaptive methodologies for online monitoring of high-dimensional streaming data. In particular, three interrelated research tasks are planned in the methodology development: (1) Efficient scalable schemes via adaptive order shrinkage with full observations, and the key novel idea is to first monitor each data stream locally through some classical, computationally simple, but efficient local detection statistics, and then combine these local procedures ?smartly? to produce a single global monitoring scheme; (2) Adaptive sampling strategies over the spatial domain, rather than the conventional time domain, such that the most informative data streams are actively selected/sampled to maximize the sensitivity and effectiveness for change detection with consideration of resources constraints; and, (3) The engineering knowledge enhanced monitoring scheme that integrates domain knowledge with local detection statistics development and adaptive sampling strategy to further improve performance. The success of this research will advance the state of the art in statistical process control and contribute to the science base of quality improvement for manufacturing systems.