Scalable Design of Robust Neural Network Controllers

研究描述：对神经网络模型的研究是一个很大的领域，需要各种数学技术来解决任何相关的研究问题。最近，由于大数据和计算能力的普及，人们的兴趣重新抬头。这些领域的例子包括图像识别、天气预测和自然语言处理。一个重要的考虑因素是神经网络在安全关键应用中的使用越来越多，例如自动驾驶汽车技术。这突出了神经网络的最大缺点，即它们对对抗性输入的敏感性：输入集的微小变化可能导致输出的巨大变化。尽管研究界做出了相当大的努力来提高我们的理解并允许对神经网络进行认证，但迄今为止，对这些系统的保证不足以使其在安全关键应用中广泛使用。这个博士项目将建立在现有的研究，探索与神经网络的鲁棒性相关的各种问题。一种流行的方法，已经看到了大量的成功是使用这些网络中的激活函数的界限，以提供这样的保证。然而，由于大量的可能的方式来约束激活函数，有一个保守性和复杂性之间的权衡。利用弦图理论，将大的约束矩阵分解为等价的小的约束矩阵，可以提高优化问题的可扩展性。这些想法也可以与平方和编程相结合-一种使用半定编程的技术。这种技术可以用于获得神经网络输出的更严格的界限，同时保持获得解决方案的计算可扩展方法。这些思想也可以扩展到神经网络控制器，以提供更好的控制性能和反馈系统的鲁棒性。目的和目标：这项工作的最终目标是创建一个框架，以可扩展的方式设计鲁棒的神经网络控制器。鲁棒性可以使用稳定性理论量化，并通过求解平方和程序来确定。由于神经网络结构可能变得非常大，因此确定稳定性可能是计算昂贵的。然而，有方法重新制定的问题，以减少这种计算负担，通过使用弦稀疏的想法。这些技术是本项目正在探索的一个关键领域。在这个博士项目中将解决的主要问题集中在将平方和编程和弦稀疏性与神经网络验证问题相结合。一旦建立了这个目标，下一个目标是看看这些想法如何扩展到神经网络控制器，然后如何将它们用于提高反馈系统的性能。 研究方法的新奇：平方和技术尚未应用于神经网络周围的问题。这导致了研究领域的差距，这将在本博士论文中进行探讨。将平方和方法与稀疏性挖掘方法相结合是一个开放的研究领域。神经网络控制器也在研究界出现，有许多问题需要探索。

Research description: Research into neural network models is a large field and requires a variety of mathematical techniques to address any relevant research questions. There has been a recent resurgence of interest due to the increase in the prevalence of big-data and computational power available. Examples of such areas include image recognition, weather prediction and natural language processing. One important consideration is the increasing use of neural networks in safety-critical applications, such as autonomous vehicle technology. This accentuates the biggest shortcoming of neural networks, which is their sensitivity to adversarial inputs: small changes in the input set can lead to large changes in the output. Despite considerable effort from the research community to improve our understanding and to allow certification of neural networks, to date guarantees on these systems are not sufficient for their widespread use in safety-critical applications. This doctoral project will build upon the existing research to explore various problems related to the robustness of neural networks. One popular method that has seen a large amount of success is to use bounds on the activation functions within these networks to provide such guarantees. However, due to the large number of possible ways to bound the activation functions, there is a trade-off between conservativeness and complexity. It is possible to improve the scalability of optimization problem by using theory from chordal graphs, where large constraints matrices are split into equivalent smaller constraints matrices. These ideas can also be combined with Sum of Squares programming - a technique that uses semi-definite programming. This technique can be used to obtain tighter bounds on the neural network output, whilst maintaining a computational scalable method of obtaining a solution. These ideas can also be extended to neural network controllers, to provide better control performance and robustness of a feedback system. Aims and objectives: The end goal of this work is to create a framework to design robust neural network controllers in a scalable way. The robustness can be quantified using stability theory and determined through solving a Sum of Squares program. Since neural network structures can become very large, determining the stability can be computational expensive. However, there are ways to reformulate the problem to reduce this computational burden by using ideas from chordal sparsity. These techniques are a key area that is being explored in this project. The main questions that will be addressed in this doctoral project focus on combining Sum of Squares programming and chordal sparsity to the neural network verification problem. Once this is established the next objective is to see how these ideas extend to neural network controllers and then how they can be used to improve the performance of feedback systems. Novelty of the research methodology: Sum of Squares techniques have not yet been applied to problems surrounding neural networks. This has led to gaps in the research area, which will be explored in this DPhil. Combining Sum of Squares and sparsity exploiting methods is an open research area. Neural network controllers are also emerging in the research community and there are many questions that need exploring.