Adaptive Data Systems

点击翻译按钮获取中文摘要

Data (management) systems have become a critical part of serving transactional requests from applications in many sectors such as science, business and health. Data systems are typically subjected to highly variable workloads in terms of transactional requests and load spikes. For example, e-commerce sites are subject to heavy and variable loads during holiday shopping sales while the sudden popularity of particular items can overwhelm the data system serving these requests. Provisioning more resources to boost system performance is a time-consuming process and takes much longer than the timespan over which the system load can increase. To deliver good performance in the face of unanticipated workload changes, it is crucial that data systems automatically adapt their behaviour to meet the variable workload demands of their clients.  To deliver good performance through adaptivity, data systems need to understand the characteristics of the workloads that they execute. This research program proposes to address these challenges. A dynamic data replication architecture will be developed and implemented to allow flexible, dynamic, allocation of resources without incurring the cost of data movement. For online transaction processing (OLTP) applications, this design incorporates algorithms that identify servers or sites where transactions can execute without distributed coordination while having enough resources to execute quickly. The challenge here is to identify which sites can offer better performance to particular transactions while ensuring that adequate resources are provisioned at that site. This adaptation will support dynamic mastership of data while maintaining even load distribution throughout the system. This proposal advocates the need to understand the behavioural patterns of systems that execute data system workloads. This task entails the collection and analysis of system resource execution data for workload requests and their behaviour. The objective is to understand the conditions under which a system made particular algorithmic and resource usage decisions and to understand their effect on system performance, which would allow the configuration of the system to be optimized to deliver the best possible performance for workloads. Achieving this goal would enable the system to deliver superior performance over its un-optimized configurations that abound in most real-world data system deployments and require expensive, and extensive, manual tuning effort.