CSR: Small: Algorithms and Abstractions for Efficient Virtual-Memory Streaming and Big-Data Computing

就目前的领域而言，操作系统为数据密集型计算提供了一个很差的接口，要求程序员从事乏味、不可重新配置和容易出错的代码开发。这些软件工程实践往往会导致容易被利用的漏洞和毁灭性的安全漏洞。该项目为虚拟流接口构建了各种算法和实现，使我们的社会能够开发更易于管理、更易于理解、本质上更快、错误更少的大数据软件。操作系统使用虚拟内存和分页已经有几十年了；然而，仍然需要用户级应用程序以固定大小的块来处理输入/输出。该项目采用了一种不同的方法，创建了零拷贝流抽象，以前所未有的简单性、灵活性和速度提供对批量数据的顺序访问。这项研究的结果不仅改善了操作系统和硬件的内部功能，而且允许在外部内存操作中重用现有的库，导致就地排序和线程间通信的速度显著加快，并为更具可扩展性的数据库计算铺平了道路。该项目提供了新颖的系统级概念和原型，简化了算法设计，实现了大规模数据流的更快处理，降低了软件成本，并有助于为21世纪创造更好的技术。该项目还让德克萨斯农工大学的学生参与跨学科领域的研究密集型教育，扩大基础研究与课堂教学的整合，指导学生，并通过公开分享我们的工作成果，允许世界各地的行业和机构进行相关研究。项目数据将在可行的情况下在线维护。共享产品包括出版物、数据、软件和各种研究成果。项目网址为http://irl.cs.tamu.edu/projects/streams/

As the field stands today, operating systems provide a poor interface for data-intensive computing, requiring programmers to engage in tedious, non-reconfigurable, and error-prone code development. These software-engineering practices often lead to easily exploitable vulnerabilities and devastating security breaches. This project builds various algorithms and implementations for a virtual-stream interface that enables our society to develop big-data software that is more easily managed, simpler to understand, inherently faster, and less buggy. Operating systems have used virtual memory and paging for decades; however, user-level applications are still required to process input/output in blocks of fixed size. This project takes a different approach by creating a zero-copy streaming abstraction that offers sequential access to bulk data with unprecedented simplicity, flexibility, and speed. The outcomes of this research not only improve the internal functionality of operating systems and hardware, but also permit reuse of existing libraries in external-memory operation, lead to significantly faster in-place sorting and inter-thread communication, and pave the way to more scalable database computing. The project delivers novel system-level concepts and prototypes that simplify algorithm design, enable faster processing of large-scale data streams, reduce software cost, and help produce better technology for the 21st century. The project also engages students at Texas A&M University in research-intensive education in cross-disciplinary fields, broadens integration of fundamental research into classroom teaching, mentors students, and permits related research in the industry and institutions around the world through publicly shared outcomes of our work.Project data will be maintained online for as long as it is feasible. The shared products include publications, data, software, and various research artifacts. The project URL is http://irl.cs.tamu.edu/projects/streams/