CAREER: Programmable Smart Machines

速度更快的计算机推动了科学、商业和日常生活的进步。不幸的是，计算机也变得越来越复杂，越来越难以有效地编程。这一趋势威胁着未来进步的可持续性。然而，也许我们可以利用生物启发的学习技术来揭示一种新的混合计算机模型，一种“可编程智能机器”，它从过去的行为中固有地学习，自动提高性能，而不需要更复杂的编程负担。具体地说，这项工作探索了在计算机中添加智能内存，使其具有学习、存储和利用过去执行中的模式的能力，以提高其性能。这项工作的核心是引入一种新的基于全局长期机器学习的“缓存”，可以被视为一种自动联想内存。“高速缓存”被提供原始的低级别执行痕迹，从中提取并存储可以识别和预测的常见模式。对核心执行过程进行修改，以将跟踪发送到“缓存”，并利用其反馈来实施加速。长期目标是一种系统，其性能随着“高速缓存”的大小和内容而提高，该“高速缓存”可以用本地联想存储设备和许多系统贡献和利用的共享在线储存库来构建。通过这种方式，一种共享的计算历史被自然地创建和利用。这项工作实验性地探索了关于具体化“可编程智能机器”模型的问题。在检测执行中的模式时，有哪些有用且易于处理的跟踪？当前的无监督深度学习技术能否检测、存储和回忆有用的模式？如何利用基于机器学习的记忆中的预测来自动提高性能？基于机器学习的记忆需要多大才能产生有用的预测和加速？这项工作使用模拟和受控工作负载实验来探索这些问题，以创建包括所有指令、寄存器值和I/O事件的完整跟踪。使用这些轨迹，将根据它们识别的模式的数量和大小来评估至少两种深度学习方法。产生的训练模型将被整合到已发表的自动并行化方法中，该方法为这项工作建立了初步结果。模拟基础设施、轨迹数据和实验结果将公开，以便进行更广泛的研究。这项工作产生了独特的计算机操作轨迹数据。PI发现，初步数据的可视和音频演示反映了计算机科学家对计算机工作原理的那种直觉。这方面将被用来开发一个研讨会，“从比特到国际象棋到超级计算机”，以及一个联合的“计算直觉”网站，让K-12的学生参与计算。

Faster computers have enabled advances in science, commerce and daily life. Unfortunately, computers have also become complex and more and more difficult to program efficiently. This trend threatens the sustainability of future advances. Perhaps, however, we can draw upon biologically inspired learning techniques to shed light into a new model of hybrid computers, a ``Programmable Smart Machine'', that inherently learns from its past behavior to automatically improve its performance without the burden of more complex programming. Specifically this work explores the addition of a smart memory to a computer that gives it the abilities to learn, store and exploit patterns in past execution to improve its performance.Central to this work is the introduction of a new kind of global long-term machine learning based 'cache' that can be viewed as an auto-associative memory. The 'cache' is fed raw low-level traces of execution, from which it extracts and stores commonly occurring patterns that can be recognized and predicted. The core execution process is modified to send the trace to the 'cache' and to exploit its feedback to enact acceleration. The long-term goal is a system whose performance improves with the size and contents of the 'cache', which can be constructed with local associative memory devices and a shared online repository that is contributed to and leveraged by many systems. In this way a kind of shared computational history is naturally created and exploited.This work experimentally explores questions with respect to concretizing the ``Programmable Smart Machine'' model. What are useful and tractable traces for detecting patterns in execution? Can current unsupervised deep learning techniques detect, store and recall useful patterns? How can the predictions from the machine learning based memory be utilized to automatically improve performance? How big does the machine learning based memory need to be to yield useful predictions and acceleration? This work explores these questions using simulation and controlled workload experiments to create complete traces including all instructions, register values, and I/O events. Using the traces, at least two deep learning approaches will be evaluated with respect to the number and size of patterns they recognize. The resulting trained models will be integrated into the published auto-parallelization methodology that established preliminary results for this work. The simulation infrastructure, trace data and experimental results will be made publicly available to enable broader study.This work produces unique trace data of computer operation. The PI has found that visual and audio presentations of the preliminary data reflect the kind of intuition that computer scientists develop about how computers work. This aspect will be leveraged to develop both a seminar, ``From Bits to Chess to Supercomputers'' and an associated``Computing Intuition'' website that engages K-12 students with computing.