SHF: Large: Collaborative Research: Exploiting the Naturalness of Software

这个跨学科的项目源于自然语言（NL）处理。像英语这样的语言允许复杂的，可爱的和复杂的结构;然而，每天，“自然？”言语和写作是简单的、可预测的和重复的，并且因此服从于统计建模。一旦大型NL语料库可用，计算能力和算法洞察力导致自然话语统计建模的快速发展，并彻底改变了翻译，语音识别，文本摘要等任务。人们实际编写的程序是有规律的、重复的和可预测的。该项目将使用统计模型来捕获和利用这种规律性，以创建新一代软件工程工具，从而实现软件质量和生产力的变革性改进。 该项目将利用语言建模技术来捕捉自然程序在词汇，句法和语义层面上的规律性。统计建模也将被用来捕捉对齐在``双语？诸如带有注释的代码的语料库，或者解释性文本（例如，Stackoverflow）和在两个平台上开发的系统中，如Java和C#。 这些统计模型将有助于推动新的、数据驱动的应用方法，如代码建议和完成，以及为有运动或视觉挑战的程序员提供辅助设备。这些模型也将被用来纠正程序中的简单错误。双语数据模型将用于构建代码摘要和代码检索工具，以及跨平台移植工具。最后，该项目将创建一个大型的软件和代码分析产品的精选语料库，以及软件双语语料库中的对齐语料库，以帮助创建和培育这一领域的研究社区。

This inter-disciplinary project has its roots in Natural Language (NL) processing. Languages such as English allow intricate, lovely and complex constructions; yet, everyday, ``natural? speech and writing is simple, prosaic, and repetitive, and thus amenable to statistical modeling. Once large NL corpora became available, computational muscle and algorithmic insight led to rapid advances in the statistical modeling of natural utterances, and revolutionized tasks such as translation, speech recognition, text summarization, etc. While programming languages, like NL, are flexible and powerful, in theory allowing a great variety of complex programs to be written, we find that ``natural? programs that people actually write are regular, repetitive and predictable. This project will use statistical models to capture and exploit this regularity to create a new generation of software engineering tools to achieve transformative improvements in software quality and productivity. The project will exploit language modeling techniques to capture the regularity in natural programs at the lexical, syntactic, and semantic levels. Statistical modeling will also be used to capture alignment regularities in ``bilingual? corpora such as code with comments, or explanatory text (e.g., Stackoverflow) and in systems developed on two platforms such as Java and C#. These statistical models will help drive novel, data-driven approaches for applications such as code suggestion and completion, and assistive devices for programmers with movement or visual challenges. These models will also be exploited to correct simple errors in programs. Models of bilingual data will used to build code summarization and code retrieval tools, as well as tools for porting across platforms. Finally, this project will create a large, curated corpus of software, and code analysis products, as well as a corpus of alignments within software bilingual corpora, to help create and nurture a research community in this area.