CAREER: A Runtime for Fast Data Analysis on Modern Hardware

在过去的60年里，计算机革命不断改变着我们的社会，因为每年计算机处理器都变得更快。不幸的是，这一趋势已经停止。 新的处理器不再能轻易地变得更快。相反，新的计算机硬件使用并行或专用组件来实现性能，这使得构建高性能应用程序变得更加困难，因为大多数现有的数据处理系统运行速度比当前处理器慢10- 100倍，并且在新兴硬件上会遇到更多麻烦。为了推动信息处理的进步，需要自动将应用程序映射到新兴硬件的计算机系统。这是一个具有挑战性的智力问题。本项目提出了“焊接”，一个运行时的数据密集型并行计算在现代硬件上。 该项目包括两个主要研究方向： * 用于数据密集型计算的中间语言（IL），可以捕获常见的数据密集型应用程序，但易于针对并行硬件进行优化。 这种语言可以将工作负载映射到CPU和GPU等各种硬件。 * 一个运行时API，允许Weld在同一程序中使用的不同库之间进行动态优化。 该API将允许Weld执行复杂的优化，如跨并行库的循环阻塞，解锁尚不可能的加速。该项目的成功将导致创建自动映射现有关键数据密集型应用程序的软件（例如，数据分析、机器学习和搜索）到新兴硬件设备，并实现了比当前应用程序快10- 100倍的速度。除了生产新技术外，该项目还将为本科生和研究生培养高性能处理，在线教学资源和研究指导方面的下一代工程师。教育和新技术共同作用，可以使大数据的工业、科学和政府用户的生产力提高10- 100倍，并实现下一代知识驱动系统。

The computer revolution that continuously transformed our society throughout the past 60 years happened because every year computer processors reliably became faster. Unfortunately, this trend has stopped. New processors can no longer easily be made faster. Instead, new computer hardware uses parallelism or specialized components to achieve performance, which has made it much harder to build high-performance applications since most existing data processing systems run 10-100x slower than they could even on current processors and will have even more trouble on emerging hardware. To drive advances in information processing, computer systems that automatically map applications to emerging hardware are needed. This is a challenging intellectual problem.This project proposes "Weld", a run-time for data-intensive parallel computation on modern hardware. The project includes 2 main research thrusts: *An intermediate language (IL) for data-intensive computation that can capture common data-intensiveapplications but is easy to optimize for parallel hardware. This language enables mapping workloads to diverse hardware like CPUs and GPUs. *A runtime API that lets Weld dynamically optimize across different libraries used in the same program. This API will allow Weld to perform complex optimizations like loop blocking across parallel libraries, unlocking speedups not yet possible.Success of this project will result in the creation of software that automatically maps existing key data intensive applications (e.g., data analytics, machine learning and search) to emerging hardware devices and achieves a 10-100x speedup over current applications. Beyond producing new technology, this project will train the next generation of engineers in high performance processing, online teaching resources, and research mentoring for undergraduate and graduate students. Together, education and new technology may make industrial, scientific, and government users of big data 10-100x more productive and enable the next generation of knowledge-driven systems.