I-Corps: Interactive and Automated Debugging for Big Data Analytics

I-Corps：大数据分析的交互式和自动调试

• 批准号: 1842657
• 负责人: Miryung Kim
• 金额: $ 5万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-15 至 2020-02-29
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1842657&HistoricalAwards=false
• 关键词: Corps; Interactive; Automated; Debugging; Big

The broader impact of this I-Corps project is to investigate the challenges that data scientists face in debugging big data analytics today and to investigate the commercial potential of research work on interactive and automated debugging of big data analytics. Big data analytics is increasingly important in the 21st century, where our daily lives leave behind a detailed digital record. Decision-makers of all kinds, from companies to government agencies, would like to base their actions on data. If successful, this project will offer a unique opportunity to discover software development tooling needs for big data systems and to identify innovative software tooling products that sit across the software stack from the user-facing API all the way down to the systems infrastructure.This I-Corps project builds on early research work on real-time debugging primitives and tool-assisted fault-localization services for big data processing applications written in modern data intensive scalable computing (DISC) systems like Apache Spark. Designing debugging primitives for DISC requires re-thinking the traditional step-through debugging primitives as provided by tools such as gdb. For example, a breakpoint feature that simply pauses the entire computation would waste large amounts of computational resources and prevent correct tasks from completing, reducing overall throughput. Further, requiring the user to inspect the millions of intermediate records produced during execution is clearly infeasible. When a failure or incorrect result is generated (e.g., outlier), pinpointing the root cause is extremely time-consuming and expensive due to massive scale of data. In short, the intellectual merit of this I-Corps project is to investigate how users can benefit from expressive debugging primitives and automated fault localization services when they must leverage for data science and big data analytics capabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个I-Corps项目的更广泛的影响是调查数据科学家在调试大数据分析时面临的挑战，并调查大数据分析的交互式和自动化调试研究工作的商业潜力。大数据分析在世纪越来越重要，我们的日常生活留下了详细的数字记录。从公司到政府机构，各种决策者都希望将他们的行动建立在数据的基础上。如果成功，该项目将提供一个独特的机会，以发现大数据系统的软件开发工具需求，并确定创新的软件工具产品，这些产品横跨软件堆栈，从面向用户的API一直到系统基础设施。该I-Corps项目建立在实时调试原语和工具辅助故障的早期研究工作基础上，为在现代数据密集型可扩展计算（DISC）系统（如Apache Spark）中编写的大数据处理应用程序提供本地化服务。为DISC设计调试原语需要重新考虑由gdb等工具提供的传统分步调试原语。例如，简单地暂停整个计算的断点功能将浪费大量的计算资源，并阻止正确的任务完成，从而降低整体吞吐量。此外，要求用户检查在执行期间产生的数百万个中间记录显然是不可行的。当产生失败或不正确的结果时（例如，异常值），由于数据规模庞大，查明根本原因非常耗时且昂贵。简而言之，这个I-Corps项目的智力价值是调查用户在必须利用数据科学和大数据分析能力时，如何从表达性调试原语和自动故障定位服务中受益。这个奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。