CAREER: System Support for Data-Intensive Online Applications

Web信息检索和科学数据驱动的应用程序等数据密集型应用程序消耗大量的计算和I/O资源。 随着计算机网络的普及和服务器集群技术的发展，使大规模数据集和相关服务实时提供给多个并发用户已经成为一个可实现的目标。 在线应用程序的系统支持必须在并发执行和运行时工作负载波动的情况下实现高性能。 扩展对数据密集型应用的这种支持提出了重大挑战，包括关于主机级的高度并发I/O和多维资源分配（例如，该项目的首要目标是开发系统级支持，以实现数据密集型在线应用程序的高吞吐量和高效资源利用。 大型在线应用程序通常包含多个应用程序组件和数据分区，托管在服务器和存储集群上。 主机级和集群级的资源利用效率对于实现高系统吞吐量是重要的。 在这个项目中有两个平行的研究方向。 在主机级别，该项目将研究操作系统对I/O密集型并发应用程序执行的支持（特别是I/O预取和内存管理）。 在集群级别，拟议的研究将探索有效的资源分配到异构分布式平台上的应用程序和数据组件使用配置文件驱动的approach.Techniques开发在这个项目中将通过实验与真实的应用程序，从科学领域和基于Web的商业应用程序进行评估。 开发的软件工件和实验数据集将被发布供公众使用。在进行研究的同时，该项目将加强参与者所在机构的相关系统领域课程。 这项工作包括开发一个新的操作系统教学实验室平台，重点关注面向性能的系统设计和实施。 此外，一个新的研究生课程可扩展的在线系统和应用程序将创建，使用本研究和其他相关研究产生的结果。

Data-intensive applications such as web information retrieval and scientific data-driven applications consume significant computational and I/O resources. With the ubiquity of computer networks and advances in server clustering technologies, it has become a reachable goal to make large-scale datasets and related services available to many concurrent users in real-time. System support for online applications must achieve high performance in the presence of concurrent execution and runtime workload fluctuation. Extending such support for data intensive applications presents significant challenges, including those concerning highly concurrent I/O at the host level and multi-dimensional resource allocation (e.g., CPU, memory, network, and disk I/O) on distributed platforms.The overarching goal of this project is to develop system-level support to achieve high throughput and efficient resource utilization for data intensive online applications. Large-scale online applications often contain multiple application components and data partitions, hosted on server and storage clusters. The resource utilization efficiencies at both the host-level and the cluster-level are important for achieving high system throughput. There are two parallel research thrusts in this project. At the host level, the project will investigate operating system support (particularly I/O prefetching and memory management) for I/O-intensive concurrent application execution. At the cluster level, the proposed research will explore efficient resource allocation to application and data components on heterogeneous distributed platforms using a profile-driven approach.Techniques developed in this project will be evaluated through experimentation with real applications, drawn from both the scientific domain and web-based commercial applications. Developed software artifacts and experimental datasets will be released for public use. In parallel to research, this project will enhance the curriculum of relevant systems-area courses at the PI's institution. This effort includes the development of a new operating system instructional laboratory platform focusing on performance-oriented system design and implementation. Additionally, a new graduate course on scalable online systems and applications will be created, using results produced in this research and by other related research.