Collaborative Research: High-Performance Techniques, Designs and Implementation of software Infrastructure for Change Detection and Mining

协作研究：用于变更检测和挖掘的软件基础设施的高性能技术、设计和实现

• 批准号: 0536994
• 负责人: Alok Choudhary
• 金额: $ 51.45万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2005
• 资助国家: 美国
• 起止时间: 2005-09-15 至 2009-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0536994&HistoricalAwards=false
• 关键词: Collaborative; Research; Performance; Techniques; Designs

ABSTRACTNSF 0536994, ChoudharyNSF 0536947, FoxProblems in managing, automatically discovering, and disseminating information are of critical importance to national defense, homeland security, and emergency preparedness and response. Much of this data originates from on-line sensors that act as streaming data sources, providing a continuous flow of information. As sensor sources proliferate, the flow of data becomes a deluge, and the extraction and delivery of important features in a timely and comprehensible manner becomes an ever increasingly difficult problem. More specifically, developing data mining and assimilation tools for data deluged applications faces three fundamental challenges. The amount of distributed real time streaming data is so large that even current extreme scale computing cannot effectively process it. Second, today's broadly deployable network protocols and web services do not provide the low latency and high bandwidth required by high volume real time data streams and distributed computing resources connectedover networks with high bandwidth delay products. Finally, the vast majority of today's statistical and data mining algorithms assume that all the data is co-located and at rest in files. Here, the real time data streams are distributed and the applications that consume them must be optimized to process multiple high volume real time streams. The goal is to develop novel algorithms and hardware acceleration schemes to allow real-time statistical modeling and change detection on such large-scale streaming data sets. By using Service Oriented Architecture principles, a framework for integrating high -performance change detection software services, including accelerations of commonly used kernels in statistical modeling, into a Grid messaging substrate will be developed and tested. Geographical Information System (GIS) services will be supported usingOpen Geospatial Consortium standards to enable geo-referencing. This project has the potential to have near-term and long-term impact in several important areas. In the near-term, the implementation of kernels and modules of statistical modeling and change detection algorithms will allow the end-user applications (e.g., homeland security, defense) to achieve one to two orders of magnitude improvement in performance for data driven decision support. In the longer term, the availability of toolkits and kernels for the change detection and data mining algorithms will facilitate the development of applications in many areas including defense, security, science and others. Furthermore, this research will bring the use of reconfigurablearchitectural acceleration of functions on streaming data including change detection and data mining, thereby opening new avenues of research and enabling newer data-driven applications on complex datasets. Both graduate and undergraduate students (through undergraduate fellowships) are engaged in the research. In addition, team members actively engage with minority serving institutions using audio/video and distance education tools.

摘要信息的管理、自动发现和传播问题对国防、国土安全、应急准备和响应至关重要。这些数据大多来自在线传感器，这些传感器充当流数据源，提供连续的信息流。随着传感器来源的激增，数据流变得泛滥，及时和可理解地提取和传递重要特征成为一个越来越困难的问题。更具体地说，为数据泛滥的应用程序开发数据挖掘和同化工具面临三个基本挑战。分布式实时流数据的量是如此之大，以至于当前的极端规模计算也无法有效地处理它。其次，今天广泛部署的网络协议和web服务不能提供高容量实时数据流和分布式计算资源所需的低延迟和高带宽，这些资源连接到具有高带宽延迟产品的网络。最后，今天的绝大多数统计和数据挖掘算法都假设所有数据都位于同一位置，并且静止在文件中。在这里，实时数据流是分布式的，使用它们的应用程序必须进行优化，以处理多个大容量的实时数据流。目标是开发新的算法和硬件加速方案，以允许在这种大规模流数据集上进行实时统计建模和变化检测。通过使用面向服务的体系结构原则，将开发和测试一个将高性能变更检测软件服务（包括统计建模中常用内核的加速）集成到网格消息传递基板中的框架。地理信息系统（GIS）服务将使用开放地理空间联盟标准来支持地理参考。该项目有可能在几个重要领域产生短期和长期影响。在短期内，统计建模和变更检测算法的内核和模块的实现将允许最终用户应用程序（例如，国土安全、国防）在数据驱动的决策支持方面实现一到两个数量级的性能改进。从长远来看，变更检测和数据挖掘算法的工具包和内核的可用性将促进包括国防、安全、科学等许多领域的应用程序的发展。此外，该研究将使用可重构架构加速流数据的功能，包括变化检测和数据挖掘，从而开辟新的研究途径，并在复杂数据集上启用更新的数据驱动应用程序。研究生和本科生（通过本科生奖学金）都参与研究。此外，团队成员积极参与少数民族服务机构使用音频/视频和远程教育工具。