Collaborative Research: FMitF: Track I: Usable Synthesis-based End-User Programming with Rich Interaction Modalities

编程，定义为将心理行动计划转换为可由计算机执行的表示的过程，已被越来越多的专业程序员以外的人认为是一种强大的，理想的，甚至是必要的技能。与此同时，为编程提供更容易、更支持的接口一直是计算机科学中的一个长期挑战，这对于那些想要利用编程能力但很少或没有受过正式培训的计算机用户来说可能特别有用。该项目旨在开发软件开发工具和方法，帮助人们使用工具提供的建议和反馈，交互式地创建数据科学和数据可视化程序。 这些工具将允许用户通过手势、语音和其他人们熟悉的与世界互动的方式来表达他们的目标，提供快速和持续的反馈，帮助人们专注于他们想要创建的程序。 为了实现这一目标，该项目结合了人机交互，编程语言和形式化方法的技术。一方面，该项目正在提供一个程序综合框架，支持丰富的互动模式，包括接受多模式的规格，并在综合过程中提供反馈。另一方面，该项目还提供了新颖的用户界面，使最终用户能够以不同的方式自然地表达他们的意图，更好地理解代表他们生成的程序的特征，并更容易地识别和更新他们的意图表达。通过协同结合这两个领域的优势，该项目旨在为可用的以人为中心的基于合成的交互系统奠定正式的方法基础。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Programming, defined as the process of transforming a mental plan of actions into a representation that can be executed by a computer, has been recognized as a powerful, desirable, and even necessary skill for an increasing number of people beyond professional programmers. In parallel, it has been a longstanding challenge in computer science to provide easier, more supportive interfaces for programming, which may be especially useful for computer users who want to leverage the power of programming but have little or no formal training. This project aims to develop software development tools and methods that help people to interactively create programs for data science and data visualization, using suggestions and feedback provided by the tools. The tools will allow users to express their goals through gestures, voice, and other familiar ways people interact with the world, providing rapid and ongoing feedback to help people zero in on the programs they are looking to create. To achieve this goal, this project is combining techniques from human-computer interaction, programming languages, and formal methods. On the one hand, this project is contributing a program-synthesis framework that supports rich interaction modalities, including accepting multi-modal specifications and providing feedback during the synthesis process. On the other hand, this project is also contributing novel user interfaces that allow end-users to naturally express their intent in different modalities, to better understand the characteristics of the program(s) generated on their behalf, and to more easily recognize and update their expression of intent. By synergistically combining strengths of these two areas, the project aims to lay the formal methods foundation for usable human-centered synthesis-based interactive systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.