STCI: Middleware for Monitoring and Troubleshooting of Large-Scale Applications on National Cyberinfrastructure

STCI：用于国家网络基础设施大规模应用监控和故障排除的中间件

• 批准号: 0943705
• 负责人: Ewa Deelman
• 金额: $ 187.58万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0943705&HistoricalAwards=false
• 关键词: STCI; Middleware; Monitoring; Troubleshooting; Large

This proposal will be awarded using funds made available by the American Recovery and Reinvestment Act of 2009 (Public Law 111-5), and meets the requirements established in Section 2 of the White House Memorandum entitled, Ensuring Responsible Spending of Recovery Act Funds, dated March 20, 2009. The STCI: Middleware for Monitoring and Troubleshooting of Large-Scale Applications on National Cyberinfrastructure project aims to provide robust and scalable workflow monitoring services that can be used to track the progress of workflow-based applications as they are executing on the distributed cyberinfrastructure. New anomaly detection and troubleshooting services will also be developed to alert users to problems with the application and cyberinfrastructure services and allow them to quickly navigate and mine the application's execution records. The foundation of this work is the development of a robust and scalable infrastructure for performance information gathering and distribution. Information flowing through this infrastructure will be stored in high-performance archives and distributed to interested entities through subscription interfaces. Three main services will be developed: 1) an online monitoring service, 2) an anomaly detection service based on dynamic mining of application and cyberinfrastructure logs and 3) a troubleshooting service that will help trace the source of a failure.Intellectual Merit This work will potentially increase scientists' productivity by allowing them to quickly identify problems in an application, thus reducing the time it takes to generate scientifically meaningful results. This work will also make the performance of complex scientific workflows more transparent, which will enable the generation of accurate estimates of overall time to completion, more efficient use of resources, and easier resolution of end-to-end performance problems in collaboration with network and resource providers. Broader Impact Scientific communities in astronomy, biology, earthquake science, physics, and others will immediately benefit from the proposed system. Because the approach relies on simple, well-defined logging formats, this work is applicable to a range of workflow management systems as well as sub-components of those systems such as job managers and data transfer tools.

该提案将使用2009年《美国复苏和再投资法案》（公共法111-5）提供的资金进行授予，并满足2009年3月20日发布的题为“确保负责任地使用复苏法案资金”的白宫备忘录第2节中规定的要求。STCI：国家网络基础设施大规模应用程序监控和故障排除中间件项目旨在提供强大且可扩展的工作流监控服务，可用于跟踪基于工作流的应用程序在分布式网络基础设施上执行的进度。还将开发新的异常检测和故障排除服务，以提醒用户注意应用程序和网络基础设施服务的问题，并使他们能够快速导航和挖掘应用程序的执行记录。这项工作的基础是为性能信息收集和分发开发一个健壮的、可扩展的基础设施。流经这一基础设施的信息将存储在高性能档案中，并通过订阅接口分发给感兴趣的实体。将开发三项主要服务：1）在线监控服务，2）基于应用和网络基础设施日志动态挖掘的异常检测服务，3）帮助追踪故障来源的故障排除服务。智力优势这项工作将通过允许科学家快速识别应用中的问题，从而潜在地提高科学家的生产力，从而减少产生有科学意义的结果所需的时间。 这项工作还将使复杂的科学工作流程的性能更加透明，这将有助于生成对完成总时间的准确估计，更有效地利用资源，并与网络和资源提供商合作更容易解决端到端性能问题。天文学、生物学、地震科学、物理学等领域的科学界将立即从拟议的系统中受益。由于该方法依赖于简单，定义良好的日志格式，这项工作适用于一系列的工作流管理系统以及这些系统的子组件，如作业管理器和数据传输工具。