Big-Data Visual Code Completion Leveraging the Naturalness of Visual Source Code

点击翻译按钮获取中文摘要

We seek to innovate code completion for visual code. Code completion tools suggest what code comes next. While textual code completion has received extensive attention, visual code completion has not. To enable visual code completion, we will leverage language and graph models trained on big-data collections of source code. This research is important as it targets end-user programmers who are more numerous than professional programmers, and under served by software engineering (SE). Code completion uses the current code and context to show programmers what they can do next, enabling exploration and experimentation. Recently, Github Co-Pilot revolutionized textual code completion with language models trained by mining the millions of software repositories on Github. Yet not all programs are text, visual programming languages often represent source code visually as graphs of nodes with input and output ports, that you connect together graphically with arrows, connectors, or edges. These languages are promoted by the No-Code movement, supported by companies like Shopify, as they enable end-user programmers to automate workflows and program software without textual code. There are many visual programming languages in use today across many domains, with many example programs available online: * Code in game levels (Unreal Engine Blueprints); * Audio/Music data-flow based languages (pure-data, Max/MSP); * Workflows for image processing (Blender, Photoshop); * Model Driven Engineering (Eclipse Modelling Framework); * Teaching programming (Scratch); and * Web app design / Workflow automation with No-code (Zapier, Shopify Mesa). Many companies in Canada such as Electronic Arts (EA), Bioware, Epic, and Shopify use and develop visual programming languages. EA's and Epic's games rely on visual code to describe effects and events within games, often making up more than 60% of the code-base, yet visual code receives far less tool support and software quality support than textual code. Classical SE models that measure software quality and support code completion are missing from visual code environments. End-user programmers who produce visual code are often unaware of their options: what the common parameters are, what comes next, and what quality code looks like. Visual code completion helps end-user programmers by suggesting visual code solutions drawn from past successful programs. Thus we propose to improve the state of visual programming languages by adapting and inventing methods for visual code completion, to suggest what nodes, edges, or parameters come next, and visual code improvement, to warn programmers about style, modularity, and error-prone code. Applying what we learned about visual programs and the naturalness of software, we will introduce SE ideas missing in visual code environments. Visual code completion will leverage the success of the past visual programs to enable end-user programmers to build successful programs in the future.