CAREER: A Runtime for Fast Data Analysis on Modern Hardware

在过去60年里持续改变我们社会的计算机革命之所以发生，是因为计算机处理器每年都在可靠地变得更快。不幸的是，这一趋势已经停止。新的处理器不再能轻易地变得更快。取而代之的是，新的计算机硬件使用并行或专门的组件来实现性能，这使得构建高性能应用程序变得更加困难，因为大多数现有的数据处理系统的运行速度甚至比在当前处理器上的运行速度慢10-100倍，而且在新兴硬件上会遇到更多麻烦。为了推动信息处理的进步，需要能够自动将应用程序映射到新兴硬件的计算机系统。这是一个具有挑战性的智力问题。本项目提出了Weld，一种在现代硬件上进行数据密集型并行计算的运行时。该项目包括两个主要研究项目：*一种用于数据密集型计算的中间语言(IL)，它可以捕获常见的数据密集型应用程序，但很容易针对并行硬件进行优化。该语言支持将工作负载映射到各种硬件，如CPU和GPU。*运行时API，允许Weld在同一程序中使用的不同库之间进行动态优化。此API将允许Weld执行复杂的优化，如跨并行库的循环阻塞，解锁加速。此项目的成功将导致创建软件，将现有的关键数据密集型应用程序(例如，数据分析、机器学习和搜索)自动映射到新兴硬件设备，并实现当前应用程序的10-100倍加速。除了生产新技术外，该项目还将在高性能处理、在线教学资源和为本科生和研究生提供研究指导方面培训下一代工程师。教育和新技术结合在一起，可能会使大数据的工业、科学和政府用户的工作效率提高10-100倍，并使下一代知识驱动系统成为可能。

The computer revolution that continuously transformed our society throughout the past 60 years happened because every year computer processors reliably became faster. Unfortunately, this trend has stopped. New processors can no longer easily be made faster. Instead, new computer hardware uses parallelism or specialized components to achieve performance, which has made it much harder to build high-performance applications since most existing data processing systems run 10-100x slower than they could even on current processors and will have even more trouble on emerging hardware. To drive advances in information processing, computer systems that automatically map applications to emerging hardware are needed. This is a challenging intellectual problem.This project proposes "Weld", a run-time for data-intensive parallel computation on modern hardware. The project includes 2 main research thrusts: *An intermediate language (IL) for data-intensive computation that can capture common data-intensiveapplications but is easy to optimize for parallel hardware. This language enables mapping workloads to diverse hardware like CPUs and GPUs. *A runtime API that lets Weld dynamically optimize across different libraries used in the same program. This API will allow Weld to perform complex optimizations like loop blocking across parallel libraries, unlocking speedups not yet possible.Success of this project will result in the creation of software that automatically maps existing key data intensive applications (e.g., data analytics, machine learning and search) to emerging hardware devices and achieves a 10-100x speedup over current applications. Beyond producing new technology, this project will train the next generation of engineers in high performance processing, online teaching resources, and research mentoring for undergraduate and graduate students. Together, education and new technology may make industrial, scientific, and government users of big data 10-100x more productive and enable the next generation of knowledge-driven systems.