Collaborative Research: FMitF: Track I: Usable Synthesis-based End-User Programming with Rich Interaction Modalities

合作研究：FMitF：第一轨：具有丰富交互方式的可用的基于综合的最终用户编程

• 批准号: 2123965
• 负责人: Elena Glassman
• 金额: $ 35万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2123965&HistoricalAwards=false
• 关键词: Collaborative; Research; FMitF; Track; Usable

Programming, defined as the process of transforming a mental plan of actions into a representation that can be executed by a computer, has been recognized as a powerful, desirable, and even necessary skill for an increasing number of people beyond professional programmers. In parallel, it has been a longstanding challenge in computer science to provide easier, more supportive interfaces for programming, which may be especially useful for computer users who want to leverage the power of programming but have little or no formal training. This project aims to develop software development tools and methods that help people to interactively create programs for data science and data visualization, using suggestions and feedback provided by the tools. The tools will allow users to express their goals through gestures, voice, and other familiar ways people interact with the world, providing rapid and ongoing feedback to help people zero in on the programs they are looking to create. To achieve this goal, this project is combining techniques from human-computer interaction, programming languages, and formal methods. On the one hand, this project is contributing a program-synthesis framework that supports rich interaction modalities, including accepting multi-modal specifications and providing feedback during the synthesis process. On the other hand, this project is also contributing novel user interfaces that allow end-users to naturally express their intent in different modalities, to better understand the characteristics of the program(s) generated on their behalf, and to more easily recognize and update their expression of intent. By synergistically combining strengths of these two areas, the project aims to lay the formal methods foundation for usable human-centered synthesis-based interactive systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

编程被定义为将心理行动计划转换为可以由计算机执行的表示形式的过程，已被认为是一种强大，可取的，甚至是必要的技能，对于越来越多的专业程序员以外的人来说。同时，在计算机科学方面，为编程提供更轻松，更支持性的接口是一个长期的挑战，这对于想要利用编程功能但几乎没有正式培训的计算机用户特别有用。该项目旨在开发软件开发工具和方法，以使用工具提供的建议和反馈来帮助人们交互式创建数据科学和数据可视化程序。 这些工具将使用户能够通过手势，声音和其他熟悉的方式来表达他们的目标，并提供与世界互动的方式，提供快速而持续的反馈，以帮助人们对他们想要创建的程序进行零。 为了实现这一目标，该项目是将人类计算机互动，编程语言和正式方法的技术结合在一起。一方面，该项目正在贡献一个程序合成框架，该框架支持丰富的交互方式，包括接受多模式规格并在合成过程中提供反馈。另一方面，该项目还贡献了新颖的用户界面，这些界面使最终用户可以自然地表达其意图，以更好地理解代表他们生成的程序的特征，并更容易地识别和更新其意图的表达。通过协同结合这两个领域的优势，该项目旨在为基于人类以人为中心的合成互动系统的形式基础。该奖项反映了NSF的法定任务，并被认为值得通过基金会的知识分子优点和更广泛的影响标准通过评估来进行评估。

项目成果

PaTAT: Human-AI Collaborative Qualitative Coding with Explainable Interactive Rule Synthesis

• DOI: 10.1145/3544548.3581352
• 发表时间: 2023-04
• 期刊: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
• 作者: Simret Araya Gebreegziabher;Zheng Zhang;Xiaohang Tang;Yihao Meng;Elena L. Glassman;Toby Jia-Jun Li

Assuage: Assembly Synthesis Using A Guided Exploration

Assuage：使用引导探索进行装配合成

• DOI: 10.1145/3472749.3474740
• 发表时间: 2021
• 期刊: UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology
• 作者: Hu, Jingmei;Vaithilingam, Priyan;Chong, Stephen;Seltzer, Margo;Glassman, Elena L.
• 通讯作者: Glassman, Elena L.

Reverse Sketching

逆向素描

• 发表时间: 2023
• 期刊: PLATEAU workshop
• 作者: Holloway, Tyler;Swoopes, Chelse;Arawjo, Ian;Peleg, Hila;Glassman, Elena
• 通讯作者: Glassman, Elena