CRII: SHF: Enabling Neuroevolution in Hardware

在过去的几年里，机器学习算法，特别是神经网络（NN），由于其在解决图像分类和语音识别中各种复杂问题的潜力而受到欢迎。不幸的是，为了有效，NN需要有适当的拓扑结构（神经元之间的连接）来完成手头的任务，并在连接上具有正确的权重。这被称为监督学习，需要通过TB到PB的数据来训练NN。这种形式的机器学习对于新兴的自主系统（机器人/无人机/汽车）领域是不可行的，这些系统通常会在任务的正确拓扑可能未知或不断变化的环境中运行，并且无法获得强大的训练数据。自主系统需要能够反映类似人类的学习，我们不断学习，通常是从经验中学习，而不是经过明确的训练。这被称为强化学习（RL）。该项目的目标是在能量受限的自主设备中实现RL。如果成功，这项研究将使自动化机器人或无人机的大规模扩散，以帮助人类社会。这些学习还将用于开发机器学习跨层支持的新课程。研究的重点将是神经进化（NE），这是一类使用进化算法来进化NN拓扑和权重的RL算法。 这个想法是并行运行多个“父”NN，让环境为所有NN的动作提供奖励（分数），并创建一个新的“子”NN种群，这些子NN保留那些导致产生最大奖励的动作的节点和连接。在多次迭代中运行NE算法已被证明会在NN中演化出复杂的行为。不幸的是，内斯在计算上非常昂贵，并且在收敛之前需要大规模计算集群运行数小时。将执行NE算法的计算和存储器行为的表征，并且将有机会跨基因大规模并行化这些算法（即，NN中的节点和连接）将被探索。交叉和变异的进化学习步骤将在专用硬件引擎上执行，并将演示在边缘运行NE算法的低功耗架构平台。此外，所提出的研究将作为进一步研究快速和节能RL算法的基础，以帮助实现通用人工智能。

Over the past few years, machine learning algorithms, especially neural networks (NN) have seen a surge of popularity owing to their potential in solving a wide variety of complex problems across image classification and speech recognition. Unfortunately, in order to be effective, NNs need to have the appropriate topology (connections between neurons) for the task at hand and have the right weights on the connections. This is known as supervised learning and requires training the NN by running it through terabytes to petabytes of data. This form of machine learning is infeasible for the emerging domain of autonomous systems (robots/drones/cars) which will often operate in environments where the right topology for the task may be unknown or keep changing, and robust training data is not available. Autonomous systems need the ability to mirror human-like learning, where we are continuously learning, and often from experiences rather than being explicitly trained. This is known as reinforcement learning (RL). The goal of this project will be on enabling RL in energy-constrained autonomous devices. If successful, this research will enable mass proliferation of automated robots or drones to assist human society. The learnings will also be used to develop new courses on cross-layer support for machine learning. The focus of the research will be on neuroevolution (NE), a class of RL algorithms that evolve NN topologies and weights using evolutionary algorithms. The idea is to run multiple "parent" NNs in parallel, have the environment provide a reward (score) to the actions of all NNs, and create a population of new "child" NNs that preserve those nodes and connections from the parents that lead to actions producing the maximum reward. Running NE algorithms over multiple iterations has been shown to evolve complex behaviors in NNs. Unfortunately, NEs are computationally very expensive and have required large scale compute clusters running for hours before converging. A characterization of the computation and memory behavior of NE algorithms will be performed, and opportunities to massively parallelize these algorithms across genes (i.e., nodes and connections in the NN) will be explored. The evolutionary learning steps of crossover and mutation will be performed over specialized hardware engines, and a low-power architectural platform running NE algorithms at the edge will be demonstrated. Furthermore, the proposed research will serve as the foundation for further research in fast and energy-efficient RL algorithms to help realize general-purpose artificial intelligence.