CAREER: A Runtime for Fast Data Analysis on Modern Hardware

在过去的60年里，计算机革命不断地改变着我们的社会，因为计算机处理器每年都在变得更快。不幸的是，这种趋势已经停止。新的处理器不再能轻易地变得更快。相反，新的计算机硬件使用并行性或专用组件来实现性能，这使得构建高性能应用程序变得更加困难，因为大多数现有数据处理系统的运行速度比当前处理器慢10-100倍，并且在新兴硬件上甚至会遇到更多麻烦。为了推动信息处理的进步，需要能够自动将应用程序映射到新兴硬件的计算机系统。这是一个具有挑战性的智力问题。这个项目提出了“Weld”，一个在现代硬件上进行数据密集型并行计算的运行时。该项目包括两个主要研究重点：*用于数据密集型计算的中间语言（IL），可以捕获常见的数据密集型应用程序，但易于针对并行硬件进行优化。这种语言支持将工作负载映射到不同的硬件，如cpu和gpu。*运行时API，允许Weld在同一程序中使用的不同库之间动态优化。这个API将允许Weld执行复杂的优化，比如跨并行库的循环阻塞，解锁目前还不可能实现的加速。该项目的成功将导致创建自动将现有关键数据密集型应用程序（例如，数据分析，机器学习和搜索）映射到新兴硬件设备的软件，并实现比当前应用程序10-100倍的加速。除了生产新技术外，该项目还将为本科生和研究生培训高性能处理、在线教学资源和研究指导方面的下一代工程师。教育和新技术共同作用，可以使工业、科学和政府大数据用户的生产力提高10-100倍，并使下一代知识驱动系统成为可能。

The computer revolution that continuously transformed our society throughout the past 60 years happened because every year computer processors reliably became faster. Unfortunately, this trend has stopped. New processors can no longer easily be made faster. Instead, new computer hardware uses parallelism or specialized components to achieve performance, which has made it much harder to build high-performance applications since most existing data processing systems run 10-100x slower than they could even on current processors and will have even more trouble on emerging hardware. To drive advances in information processing, computer systems that automatically map applications to emerging hardware are needed. This is a challenging intellectual problem.This project proposes "Weld", a run-time for data-intensive parallel computation on modern hardware. The project includes 2 main research thrusts: *An intermediate language (IL) for data-intensive computation that can capture common data-intensiveapplications but is easy to optimize for parallel hardware. This language enables mapping workloads to diverse hardware like CPUs and GPUs. *A runtime API that lets Weld dynamically optimize across different libraries used in the same program. This API will allow Weld to perform complex optimizations like loop blocking across parallel libraries, unlocking speedups not yet possible.Success of this project will result in the creation of software that automatically maps existing key data intensive applications (e.g., data analytics, machine learning and search) to emerging hardware devices and achieves a 10-100x speedup over current applications. Beyond producing new technology, this project will train the next generation of engineers in high performance processing, online teaching resources, and research mentoring for undergraduate and graduate students. Together, education and new technology may make industrial, scientific, and government users of big data 10-100x more productive and enable the next generation of knowledge-driven systems.