Knowledge Representation in Transfer Optimisation System and Applications for Highly Configurable Software Systems

该项目计划开发迁移优化算法，结合自然启发优化和迁移学习之间的思想，为优化算法提供足够的智能，从而导致自适应行为。为此，研究将集中在转移优化的一个关键问题上，即用于评估和比较从以前的优化过程中学习到的不同“经验”之间的相似性的知识表示和度量。通过这样做，它能够克服负迁移带来的灾难，跨任务转移不相关或无用的知识。我们将从图论开始，因为图是对各种结构的一般但强大的表示。在这种情况下，我们设想它能够成为各种景观知识表示的构建块。表示学习技术，学习数据的内在结构和表示，以方便有用的信息提取，将发展到从适应度景观本身的表示中理解问题特征。对于连续变量，我将研究使用基于重建的方法，学习从观察数据到表示（如自动编码器框架）的参数映射。对于离散变量，我将研究将适应度景观表示为信息网络。然后将开发网络表示学习方法，在保持网络拓扑结构的同时学习网络顶点的潜在低维表示。为了衡量不同知识之间的相似性，我将开发一些指标来服务于定量评估。这本质上与知识的表示方式有关。对于表示为低维编码器的知识，我将基于标准距离度量（如欧几里得距离）来评估相似性。对于表示为信息网络的知识，我将从图匹配的角度[2]进行研究，并开发相似度函数来度量不同网络之间的结构相似度。一旦开发了知识表示和相似性度量。我将研究如何在自然启发的计算中使用它们来提出一个传输优化算法。机器学习文献b[5]中的许多迁移学习技术都能够服务于迁移学习的目的。特别是，我将考虑两个层次的知识转移。一个是遗传水平，其目的是利用在以前的优化练习中发现的优化来加速潜在的优化。另一个是模型级别，它将使用迁移学习技术来调整不同任务之间的模型。本项目开发的传输优化算法将应用于优化高度可配置软件系统的非功能性能。现代工业软件系统非常复杂，有许多配置选项，其设置直接关系到它们的非功能性能。有争议的是，这些系统过于复杂，无法手动配置，以便在各种环境和不同的用户需求下实现其运行时的峰值性能。当底层系统产生大量数据吞吐量时，评估底层系统的非功能性能也非常耗时。构建代理来理解和预测配置选项的效果是在运行时优化自适应软件系统的一种很有希望的替代方法。更具体地说，在这个博士项目中开发的知识表示将服务于代理建模的目的，而转移优化将用于通过优化学习和积累知识。

This project plan to develop transfer optimisation algorithms that combines the idea between nature-inspired optimisation and transfer learning to equip an optimisation algorithm with adequate intelligence thus lead to a self-adaptive behaviour. To this end, the research will focus on one of the key questions to the endeavour of transfer optimisation, i.e., the knowledge representation and metrics used to evaluate and compare the similarity between different "experience" learned from the previous optimisation process. By doing so, it is able to overcome the negative transfer which brings disasters to transfer irrelevant or useless knowledge across tasks.We will start from graph theory given that graph is a general but powerful representation to various structures. In this case, we envisage that it is able to be the building block for knowledge representation for various landscapes. Representation learning techniques, which learn the intrinsic structure and representation of the data to facilitate useful information extraction, will be developed to understand the problem features from the representation of the fitness landscape itself. As for continuous variables, I will study to use reconstruction-based approaches that learns a parametric mapping from observed data to a representation like the autoencoder framework. For discrete variables, I will study to represent the fitness landscape as an information network. Then network representation learning approaches will be developed to learn a latent low-dimensional representations of network vertices while preserving network topology structure. In order to measure the similarity between different knowledge, I will develop some metrics to serve the quantitative evaluation. This is essentially related to the way how the knowledge is represented.For the knowledge represented as a low-dimensional encoder, I will evaluate similarity based on standard distance measures like Euclidean distance. For the knowledge represented as an information network, I will study from the graph matching perspective [2] and to develop similarity functions to measure the structural similarity between different networks.Once the knowledge representation and similarity measure are developed. I will study how to use them within nature-inspired computation to come up with a transfer optimisation algorithm.Many transfer learning techniques in the machine learning literature [5] are able to serve the purpose of transfer learning. In particular, I will consider two levels of knowledge transfer. One is genetic-level which aims to leverage the optima found in the previous optimisation exercises to accelerate the underlying optimisation. The other one is model level which is going to use transfer learning techniques to align the models across various tasks.The transfer optimisation algorithms developed in this project will be applied to optimise the non-functional performance of highly configurable software systems. Modern industrial software systems are super complex with many configuration options, the setting of which is directly related to their non-functional performance. It is arguable that those systems are too complex to be manually configured in order to achieve their peak performance at runtime under various environments and different user requirements. It is also time consuming to evaluate the non-functional performance of the underlying system when it incurs the throughput of huge volume of data. Building a surrogate to understand and predict the effect of a configuration option is promising alternative to enable the optimisation of a self-adaptive software system at runtime. More specifically, the knowledge representation developed in this PhD project will serve the purpose of surrogate modelling whilst the transfer optimisation will be used to learn and accumulate knowledge through optimisation.