Knowledge Representation in Transfer Optimisation System and Applications for Highly Configurable Software Systems

该项目计划开发转移优化算法，该算法结合了自然启发的优化和转移学习之间的思想，以使优化算法具有足够的智能，从而导致自适应行为。为此，研究将集中于迁移优化努力的关键问题之一，即用于评估和比较从先前优化过程中学到的不同“经验”之间的相似性的知识表示和度量。通过这样做，它能够克服带来灾难的负迁移，从而在任务之间传递无关或无用的知识。我们将从图论开始，因为图是对各种结构的一般但强大的表示。在这种情况下，我们设想它能够成为各种景观的知识表示的构建块。表征学习技术将学习数据的内在结构和表征，以促进有用信息的提取，从而从适应度景观本身的表征中理解问题特征。至于连续变量，我将学习使用基于重建的方法，该方法从观察数据学习参数映射到像自动编码器框架这样的表示。对于离散变量，我将研究如何将健康状况表示为一个信息网络。然后开发网络表示学习方法，学习网络顶点的潜在低维表示，同时保持网络的拓扑结构。为了衡量不同知识之间的相似性，我将开发一些度量来服务于定量评估。这本质上与知识的表示方式有关。对于表示为低维编码器的知识，我将基于欧几里德距离等标准距离度量来评估相似性。对于被表示为信息网络的知识，我将从图匹配的角度进行研究[2]，并建立相似性函数来度量不同网络之间的结构相似性，然后给出知识表示和相似性度量。我将研究如何在自然启发的计算中使用它们来提出迁移优化算法。机器学习文献[5]中的许多迁移学习技术都能够服务于迁移学习的目的。特别是，我将考虑两个层面的知识转移。一种是遗传级的，它旨在利用之前的优化练习中发现的最优来加速潜在的优化。另一个是模型级，它将使用迁移学习技术来调整不同任务的模型。本项目中开发的迁移优化算法将用于优化高度可配置的软件系统的非功能性能。现代工业软件系统超级复杂，配置选项众多，配置选项的设置直接关系到系统的非功能性能。有争议的是，这些系统太复杂了，不能手动配置，以便在各种环境和不同用户需求下实现运行时的最佳性能。当底层系统产生海量数据吞吐量时，评估底层系统的非功能性能也是非常耗时的。构建代理来理解和预测配置选项的效果是实现在运行时优化自适应软件系统的有前途的替代方案。更具体地说，在这个博士项目中开发的知识表示将服务于代理建模的目的，而转移优化将用于通过优化来学习和积累知识。

This project plan to develop transfer optimisation algorithms that combines the idea between nature-inspired optimisation and transfer learning to equip an optimisation algorithm with adequate intelligence thus lead to a self-adaptive behaviour. To this end, the research will focus on one of the key questions to the endeavour of transfer optimisation, i.e., the knowledge representation and metrics used to evaluate and compare the similarity between different "experience" learned from the previous optimisation process. By doing so, it is able to overcome the negative transfer which brings disasters to transfer irrelevant or useless knowledge across tasks.We will start from graph theory given that graph is a general but powerful representation to various structures. In this case, we envisage that it is able to be the building block for knowledge representation for various landscapes. Representation learning techniques, which learn the intrinsic structure and representation of the data to facilitate useful information extraction, will be developed to understand the problem features from the representation of the fitness landscape itself. As for continuous variables, I will study to use reconstruction-based approaches that learns a parametric mapping from observed data to a representation like the autoencoder framework. For discrete variables, I will study to represent the fitness landscape as an information network. Then network representation learning approaches will be developed to learn a latent low-dimensional representations of network vertices while preserving network topology structure. In order to measure the similarity between different knowledge, I will develop some metrics to serve the quantitative evaluation. This is essentially related to the way how the knowledge is represented.For the knowledge represented as a low-dimensional encoder, I will evaluate similarity based on standard distance measures like Euclidean distance. For the knowledge represented as an information network, I will study from the graph matching perspective [2] and to develop similarity functions to measure the structural similarity between different networks.Once the knowledge representation and similarity measure are developed. I will study how to use them within nature-inspired computation to come up with a transfer optimisation algorithm.Many transfer learning techniques in the machine learning literature [5] are able to serve the purpose of transfer learning. In particular, I will consider two levels of knowledge transfer. One is genetic-level which aims to leverage the optima found in the previous optimisation exercises to accelerate the underlying optimisation. The other one is model level which is going to use transfer learning techniques to align the models across various tasks.The transfer optimisation algorithms developed in this project will be applied to optimise the non-functional performance of highly configurable software systems. Modern industrial software systems are super complex with many configuration options, the setting of which is directly related to their non-functional performance. It is arguable that those systems are too complex to be manually configured in order to achieve their peak performance at runtime under various environments and different user requirements. It is also time consuming to evaluate the non-functional performance of the underlying system when it incurs the throughput of huge volume of data. Building a surrogate to understand and predict the effect of a configuration option is promising alternative to enable the optimisation of a self-adaptive software system at runtime. More specifically, the knowledge representation developed in this PhD project will serve the purpose of surrogate modelling whilst the transfer optimisation will be used to learn and accumulate knowledge through optimisation.