CAREER: Regularizing Large Language Models for Safe and Reliable Program Generation

随着计算几乎融入商业和社会的各个方面，对各种编程技能的需求越来越大，以推动经济增长并保持国家在技术创新方面的竞争力。然而，编程对认知要求很高，很难学习。ChatGPT等大型语言模型（LLM）的最新进展已经证明了通过从自然语言生成复杂程序来解决这一挑战的巨大潜力，消除了记忆和处理程序语法和语义的需要。尽管有这一突破，作为一种统计方法，LLM在应用于强调功能正确性、安全性和鲁棒性的程序生成等领域时会遇到许多问题。例如，最近的研究表明，LLM可能会生成与用户意图不一致的代码，具有安全漏洞或违反编码标准。这些问题因此降低了程序员的生产力和对LLM的信任。该项目将促进对LLM在程序生成中的局限性的理解，并开发原则性的监管方法，以提高LLM生成代码的正确性，安全性和鲁棒性。该项目的研究成果和教育活动将为新的教学材料和教学方法的设计提供信息，以帮助计算机科学专业的学生为大型语言模型时代的未来职业做好准备。该项目遵循混合方法研究设计，将经验数据驱动研究，算法开发和工具构建相结合。虽然最近在评估和改进基于LLM的程序生成方面做出了许多努力，但仍然不清楚LLM会产生什么类型的程序生成错误，不同类型的LLM是否会产生不同类型的错误，以及为什么会产生这样的错误。本项目将通过使用扎根理论深入分析各种LLM程序生成错误的症状和特征，并开发根本原因分析方法来调查程序错误与LLM内部状态之间的相关性，从而弥合知识差距。此外，该项目还将开发新的错误定位和缓解方法，利用LLM的内部状态，以低成本精确定位不同类型错误的根本原因。该项目还将首次尝试通过开发新的轻量级模型自适应方法来减轻LLM生成代码中的非功能性问题。通过这些进步，该项目旨在从根本上改变LLM的培训和更新方式，以实现安全可靠的代码生成。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With computing woven into almost every aspect of business and society, there is an increasing demand for various programming skills to drive economic growth and keep the nation competitive in technology innovation. However, programming is cognitively demanding and hard to learn. Recent advances in Large Language Models (LLMs) such as ChatGPT have demonstrated a great potential to address this challenge by generating complex programs from natural language, eliminating the need to memorize and grapple with program syntax and semantics. Despite this breakthrough, as a statistical method, LLMs suffer from a number of issues when applied to domains like program generation that emphasize functional correctness, safety, and robustness. For instance, recent studies have shown that LLMs may generate code that does not align with user intent, has security vulnerabilities, or violates coding standards. These issues consequently diminish programmer productivity and trust in LLMs. This project will advance the understanding of the limitations of LLMs in program generation and develop principled regulation approaches to enhance the correctness, safety, and robustness of LLM-generated code. The research findings and education activities of this project will inform the design of new instructional materials and pedagogical approaches to prepare computer science students for future careers in the age of large language models.This project follows a mixed-methods research design that combines empirical data-driven studies, algorithm development, and tool building. While there have been many recent efforts in evaluating and improving LLM-based program generation, it remains unclear what types of program generation errors LLMs make, whether different kinds of LLMs make different types of errors, and why they make such errors. This project will bridge the knowledge gap by conducting an in-depth analysis of the symptoms and characteristics of program generation errors made by various LLMs using grounded theory and developing root cause analysis methods to investigate the correlation between program errors and the internal states of LLMs. In addition, the project will develop new error localization and mitigation methods that leverage the internal states of LLMs to precisely target the root causes of different kinds of errors at a low cost. The project will also make the first attempt to mitigate non-functional concerns in LLM-generated code by developing new lightweight model adaptation methods. With these advancements, this project aims to fundamentally chang the way LLMs are trained and updated for safe and reliable code generation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.