CNS Core: Small: Ultra-Efficient Neural Network and LSTM Architectures

神经网络已经开始在图像识别、语音识别和机器翻译等各种重要应用中产生广泛的影响。近十年来，人们对机器学习和人工智能的兴趣激增，可以追溯到神经网络所带来的准确性提高。然而，如何提出最好的神经网络架构仍然是一个悬而未决的问题。因此，它引起了学术界和工业界的广泛关注。这项工作将解决这个问题。神经网络合成在很大程度上仅限于大数据应用，而且神经网络模型通常预计在云中运行。然而，最近业界对边缘级（例如智能手机或智能手表）神经网络模型感兴趣。当前的边缘级神经网络为了能量和延迟效率而牺牲了精度（4-5%）。对于中等数据和小数据应用，神经网络通常也无法与其他模型竞争。最后，对于边缘设备，序列到序列建模（例如，用于语言翻译）也需要更加准确、快速和紧凑。所有这些问题将在本工作中通过新的神经网络合成技术和工具来解决。这项研究有可能在克服当前神经网络合成方法的缺陷方面取得变革性进展。由于机器学习应用的爆炸式增长，这项研究有望为美国公司和经济提供重大推动。因此，它将涉及重要的工业约定。一些代表性不足的人（少数民族/女性）将参与这项研究。研究成果将包括在机器学习和嵌入式计算两门本科课程中。将通过发表论文、海报和研讨会向学术界和工业界广泛传播。此外，各种工具和模型将在网上分发。出版物/学生名单和附有适当文件的工具/数据将在https://www.princeton.edu/~jha/上提供。允许为研究和教育目的免费使用数据和工件。这些数据将在项目完成后至少五年内在线提供。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Neural networks (NNs) have begun to have widespread impact on various important applications, such as image recognition, speech recognition, and machine translation. The spurt of interest in machine learning and artificial intelligence in this decade can be traced back to the increase in accuracy that NNs have enabled. Yet, how to come up with the best NN architecture has remained an open problem. Hence, it is attracting a lot of attention from the academia and industry. This work will address this problem.NN synthesis has largely been limited to big-data applications and the NN models are typically expected to run in the cloud. However, there is recent interest from the industry to have edge-level (e.g., in smartphone or smartwatch) NN models. The current edge-level NNs sacrifice accuracy (by 4-5%) for energy and latency efficiency. NNs are also often not competitive with other models for medium-data and small-data applications. Finally, sequence-to-sequence modeling (e.g., for language translation) also needs to be made much more accurate, fast, and compact enough for edge devices. All these problems will be tackled in this work through new NN synthesis techniques and tools.This research has the potential to enable transformative advances in overcoming the deficiencies of current NN synthesis methodologies. Due to the explosion in machine learning applications, this research has the promise to provide a significant boost to U.S. companies and economy. Thus, it will involve significant industrial engagements. Several underrepresented (minority/female) will be involved in the research. The research outcomes will be included in two undergraduate courses on Machine Learning and Embedded Computing. Broad dissemination to the academic and industrial communities will be achieved through published papers, posters, and seminars. Additionally, various tools and models will be distributed online.The list of publications/students and tools/data with appropriate documentation will be made available at https://www.princeton.edu/~jha/. Free use of data and artifacts will be permitted for research and educational purposes. The data will be available online for at least five years following the completion of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.