CAREER: Interactive Program Synthesis for Web Automation

流程自动化仍然是数字化转型的主要驱动力。然而，从技术上讲，创建这样的自动化程序仍然非常苛刻。该项目旨在显著降低创建Web自动化程序的技术障碍。该项目的新颖性是一套新的基于重写的程序合成算法，可以根据用户演示自动生成程序。该项目的影响是使需要执行乏味但与网络相关的编程任务的非专家能够创建一个程序来自动化工作，即使他们在网络编程方面几乎没有或根本没有背景。这个项目的核心是开发算法，将用户演示作为输入-以用户动作(例如，点击按钮、抓取数据)的跟踪A的形式-并通过重写A到P来合成具有控制流结构的参数化程序P。该方法基于有限树自动机，涉及底层Web自动化语言中的神经和符号元素，目标是能够有效地推理网页内容，同时仍然利用底层网页结构。该项目正在策划一套新的网络自动化任务，这些任务对本项目以外的未来研究是独立有用的。这位研究人员正在与密歇根州的合作伙伴(例如，女性科学与工程组织(WISE)、密歇根州路易斯·斯托克斯少数群体参与联盟(MI-LSAMP)和工程学术成功中心(ECAS))合作，以增加来自代表性不足群体的学生对他的研究的参与。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Process Automation continues to be a main driver of digital transformation. Yet, it is still technically very demanding to create such automation programs. This project aims to significantly lower the technical barrier of creating web automation programs. The project’s novelties are a suite of new rewrite-based program synthesis algorithms that can automatically generate programs from user demonstrations. The project’s impacts are to enable non-experts in need of performing a tedious but programmatic web-related task to create a program to automate the work, even if they have little or no background in web programming. At the core, this project is developing algorithms that take as input a user demonstration – in the form of a trace A of user actions (e.g., clicking buttons, scraping data) – and synthesize a parameterized program P with control-flow structures by rewriting A to P. The approach is based on finite tree automata and involves both neural and symbolic elements in the underlying web automation language, with the goal of being able to effectively reason about the webpage contents while still leveraging the underlying webpage structure. The project is curating a new suite of web automation tasks that are independently useful for future research beyond this project. The investigator is collaborating with partners at Michigan (e.g., Women in Science and Engineering (WISE), Michigan Louis Stokes Alliance for Minority Participation (MI-LSAMP) and Engineering Center for Academic Success (ECAS)) to increase participation of students from underrepresented groups in his research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.