Using GitHub Copilot to Solve Simple Programming Problems
Using GitHub Copilot to Solve Simple Programming Problems
复制标题
DOI：
10.1145/3545945.3569830
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
M. Wermelinger
中科院分区：
文献类型：
--
作者：
M. Wermelinger
编程入门的教学和评估包括编写代码来解决由文本描述的问题。之前的研究发现，OpenAI的Codex是一种经过数十亿行代码训练的自然语言机器学习模型，在许多编程问题上表现良好，通常会生成正确且可读的Python代码。GitHub的Codex版本Copilot对学生免费开放。这引起了对教学和学术诚信的关注。教育工作者需要知道Copilot的能力，以便使他们的教学适应人工智能编程助手。以前的研究定量地评估了最有效的食品法典模型，例如，有多少问题至少有一个正确的建议通过了所有测试。在这里，我转而评估Copilot，看看它是否与Codex有什么不同，并定性地看待生成的建议，以了解Copilot的局限性。我还报告了使用Copilot完成编程课程中要求学生进行的其他活动的经验：解释代码、生成测试和修复bug。本文最后讨论了观察到的能力对编程教学的影响。
The teaching and assessment of introductory programming involves writing code that solves a problem described by text. Previous research found that OpenAI's Codex, a natural language machine learning model trained on billions of lines of code, performs well on many programming problems, often generating correct and readable Python code. GitHub's version of Codex, Copilot, is freely available to students. This raises pedagogic and academic integrity concerns. Educators need to know what Copilot is capable of, in order to adapt their teaching to AI-powered programming assistants. Previous research evaluated the most performant Codex model quantitatively, e.g. how many problems have at least one correct suggestion that passes all tests. Here I evaluate Copilot instead, to see if and how it differs from Codex, and look qualitatively at the generated suggestions, to understand the limitations of Copilot. I also report on the experience of using Copilot for other activities asked of students in programming courses: explaining code, generating tests and fixing bugs. The paper concludes with a discussion of the implications of the observed capabilities for the teaching of programming.