Deep Learning and AI Alignment

神经网络由于具有自动学习高级特征表示的能力，已经成为人工智能许多领域的核心机器学习技术。尽管网络在预测的准确性方面取得了令人印象深刻的表现，但仍然存在几个障碍：神经网络往往对自己的预测过于自信，对数据分布的变化很敏感，或者倾向于利用虚假的相关性。此外，神经网络结构在很大程度上是基于模式识别的，而将其扩展到更困难的问题将需要更多的深思熟虑。格罗斯博士的大部分研究都集中在理解神经网络训练动力学--训练过程中发生的事情影响最终结果的方式。他的程序的独特之处在于，虽然大多数这样的工作都集中在模式识别设置上，但他将通过满足以下目标将这种分析扩展到网络在测试时执行复杂推理或优化的情况：1)将先前对神经网络训练动力学的调查扩展到结构执行更复杂的计划、推理或优化，以及训练制度可能涉及到针对不同目标训练的多个神经网络的设置。2)了解这些设置如何以及为什么会导致与更传统的神经网络相比不同的泛化模式3)通过确定如果网络在略有不同的数据上训练，预测将如何改变来理解神经网络预测的原因；以及4)开发训练神经网络的算法，以不仅产生答案，而且为该答案产生独立可检查的理由。最终，这一计划将提高网络预测的可解释性，并减少对数据中虚假关联的依赖。

Neural networks have become the core machine learning technology across many areas of Artificial Intelligence, due to their ability to automatically learn high-level feature representations. While networks have achieved impressive performance in terms of the accuracy of their predictions, there remain several obstacles: neural networks are often overconfident about their predictions, sensitive to shifts in the data distribution, or prone to exploit spurious correlations. Furthermore, neural net architectures are largely based on pattern recognition, whereas extending them to more difficult problems will require more deliberative reasoning.Much of Dr. Grosse's research has focused on understanding neural net training dynamics - theways in which what happens during training affects the final outcome. His program is unique in that while most such work has focused on the pattern recognition setting, he will extend such analyses to cases where the network performs sophisticated reasoning or optimization at test time by meeting the following objectives: 1) extending prior investigations of neural net training dynamics to settings where the architectures perform more sophisticated planning, reasoning, or optimization, and where the training regime may involve multiple neural nets trained to different objectives2) understanding how and why these settings can lead to different patterns of generalization compared with more traditional neural nets3) understanding the reasons for a neural net's predictions by determining how the predictions would have changed if the network were trained on slightly different data; and 4) developing algorithms for training a neural net to produce not only an answer, but also an independently checkable justification for that answer. Ultimately, this program will lead to greater interpretability of the network's predictions, as well as reducing the reliance on spurious correlations in the data.