CNS Core: Small: Ultra-Efficient Neural Network and LSTM Architectures

神经网络（NN）已经开始对各种重要应用产生广泛的影响，例如图像识别，语音识别和机器翻译。 在这十年中，人们对机器学习和人工智能的兴趣激增可以追溯到NN实现的准确性的提高。 然而，如何提出最好的神经网络结构仍然是一个悬而未决的问题。 因此，它吸引了学术界和工业界的大量关注。这项工作将解决这个问题。NN合成在很大程度上仅限于大数据应用，并且NN模型通常期望在云中运行。 然而，最近业界对具有边缘级（例如，在智能手机或智能手表中）NN模型。 目前的边缘级NN为了能量和延迟效率牺牲了准确性（4-5%）。 对于中等数据和小数据应用，NN通常也无法与其他模型竞争。 最后，序列到序列建模（例如，对于语言翻译）也需要变得更加准确、快速和紧凑，足以用于边缘设备。 所有这些问题都将通过新的NN合成技术和工具在这项工作中得到解决。这项研究有可能使变革性的进步，克服目前NN合成方法的不足。由于机器学习应用的爆炸式增长，这项研究有望为美国公司和经济提供重大推动。因此，它将涉及重大的工业参与。一些代表性不足的人（少数民族/女性）将参与研究。研究成果将被纳入机器学习和嵌入式计算两门本科课程。将通过发表论文、海报和研讨会向学术界和工业界广泛传播。此外，各种工具和模型将在网上分发。出版物/学生和工具/数据的列表以及相应的文档将在https://www.princeton.edu/~jha/上提供。将允许出于研究和教育目的免费使用数据和人工制品。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Neural networks (NNs) have begun to have widespread impact on various important applications, such as image recognition, speech recognition, and machine translation. The spurt of interest in machine learning and artificial intelligence in this decade can be traced back to the increase in accuracy that NNs have enabled. Yet, how to come up with the best NN architecture has remained an open problem. Hence, it is attracting a lot of attention from the academia and industry. This work will address this problem.NN synthesis has largely been limited to big-data applications and the NN models are typically expected to run in the cloud. However, there is recent interest from the industry to have edge-level (e.g., in smartphone or smartwatch) NN models. The current edge-level NNs sacrifice accuracy (by 4-5%) for energy and latency efficiency. NNs are also often not competitive with other models for medium-data and small-data applications. Finally, sequence-to-sequence modeling (e.g., for language translation) also needs to be made much more accurate, fast, and compact enough for edge devices. All these problems will be tackled in this work through new NN synthesis techniques and tools.This research has the potential to enable transformative advances in overcoming the deficiencies of current NN synthesis methodologies. Due to the explosion in machine learning applications, this research has the promise to provide a significant boost to U.S. companies and economy. Thus, it will involve significant industrial engagements. Several underrepresented (minority/female) will be involved in the research. The research outcomes will be included in two undergraduate courses on Machine Learning and Embedded Computing. Broad dissemination to the academic and industrial communities will be achieved through published papers, posters, and seminars. Additionally, various tools and models will be distributed online.The list of publications/students and tools/data with appropriate documentation will be made available at https://www.princeton.edu/~jha/. Free use of data and artifacts will be permitted for research and educational purposes. The data will be available online for at least five years following the completion of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.