NSF Convergence Accelerator Track H: Determining Community Needs for Accessibility Tools that Facilitate Programming Education and Workforce Readiness for Persons with Disabilities

10亿人，占世界人口的15%，经历过残疾。残疾是进入劳动力市场的主要障碍；2021年，约80%的残疾人被排除在劳动力之外。与此同时，编程工作继续呈爆炸式增长，但在很大程度上是残疾人无法获得的。标准的编程接口——屏幕、键盘、鼠标——对许多残疾人来说很难操作。该项目支持身体残疾的个人，这些人在学习和参与编程方面存在障碍，阻碍了他们获得广泛可用的、有利可图的、向上流动的技术劳动力。这项工作将通过开发和评估多模态接口（例如，语音、眼动追踪、踏板）的原型，开发一种改善残疾人面临的负面劳动结果的方法，使残疾人能够学习、练习和利用编程技能。该项目还将为残疾人提供培训和进入编程队伍的途径，从而弥合阻碍大多数残疾人获得此类职业机会的职业差距。该项目的影响包括：1)增加了残疾人在STEM工作中的代表性，2)增加了残疾人的经济和个人福祉，3)提高了美国的经济竞争力，4)加强了研究和教育的基础设施。让残疾人士获得编程技能和就业机会，将促使大批残疾人士进入科技行业，增加STEM领域的多样性，并为残疾人士带来持续的就业和经济福祉。由于编程工作通常可以远程完成，因此这项工作将消除残疾人的交通障碍，特别是那些使用轮椅的残疾人。由于工作的稳定性是许多残疾人士最关心的问题，这将持续改善他们的生活质素。此外，增加会编程的劳动力将提高美国的经济竞争力，同时增加劳动力的多样性。最后，开发有效的、用户友好的、可持续的与计算机交互的方式也将有助于K-12和全国范围的研究。它将允许早期编程教育和计算机在K-12设置中使用，用于身体残疾的儿童，以及那些失去灵活性和精细运动控制的老年人。这项计划将带来多项技术上的进步和贡献，包括：1)就残疾人士分组的相关需求提供大量的见解，让他们参与学习和实践为工作做好准备的编程；2)共同设计可个性化的原型界面，特别设计为价格合理，可访问且跨平台工作；3)机器学习模型，使开发的工具能够个性化，以满足个人用户的需求；4)在共同设计会议期间，由顾问委员会和社区合作伙伴告知的包容性评估框架。这项工作结合了键盘和鼠标以外的输入设备，创建了一个现成的、个性化的、多模式的输入界面，用于教学，并为至少一个确定的具有相似身体能力的用户亚群提供工作准备编程。这些原型探索了由多模态、个性化机器学习模型处理的输入模式组合，将输入转换为输出按键和光标活动。原型将利用大规模预训练的语言模型来指导将用户输入转换为代码输出，并将使用具有集中输出空间的单一输入模式（即编写Python函数，操作符和特定于程序的变量）；将建立多个原型，并与相关的残疾人群体进行测试。来自每种模式的数据语料库将用于通过聚类技术识别具有相似能力的用户集，而集中式学习主干可以使用低参数微调方法（如基于转换器的适配器）对每个用户群进行微调。这些技术进步有可能大大扩大残疾人社区获得技术劳动力的机会，从而深刻影响身体感觉运动障碍患者的技术劳动力机会。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

One billion people, 15% of the world’s population, have experienced a disability. Disabilities present major barriers to entering the labor force; approximately 80% of persons with disabilities (PWDs) were excluded from the 2021 labor force. Meanwhile, programming jobs continue to grow explosively but are largely inaccessible to PWDs. Standard programming interfaces–-screens, keyboards, mice–-are difficult to operate for many PWDs with physical challenges. This project supports individuals with physical disabilities that result in barriers to learning and engaging in programming, blocking access to the widely available, lucrative, and upwardly mobile technology workforce. The work will develop a means of ameliorating negative labor outcomes faced by PWDs by developing and evaluating prototypes of multimodal interfaces (e.g., speech, eye tracking, pedals) that enable PWDs to learn, practice, and utilize programming skills. The project will also develop a path for PWDs to train for—and enter—the programming workforce, thereby bridging the career gap that blocks most PWDs from such career opportunities. Impacts of this project include: 1) increased representation of PWDs in STEM jobs, 2) increased economic and personal well-being for PWDs, 3) improved economic competitiveness of the U.S., and 4) enhanced infrastructure for research and education. Enabling PWDs access to programming skills and employment will produce an influx of PWDs in the technology sector, increase diversity in STEM, and lead to sustained employment and economic well-being for PWDs. Since programming jobs can often be done remotely, this work will remove transportation barriers for PWDs, especially those who use wheelchairs. As job stability is a primary concern for many PWDs, this will lead to sustained improvements in quality of life. Furthermore, increasing the workforce of people who can program will improve US economic competitiveness while increasing the diversity of the workforce. Finally, developing effective, user-friendly, and sustainable ways of interfacing with computers will also be useful in K-12 and in research nationwide. It will allow for early programming education and computer use in K-12 settings for children with physical disabilities, and for aging seniors who experience loss of dexterity and fine motor control.This project will produce multiple technical advances and contributions, including: 1) a large corpus of insights about the relevant PWD subpopulation needs for engaging in learning and practicing workforce-ready programming; 2) co-designed personalizable prototype interfaces specifically designed to be affordable, accessible, and work across platforms; 3) machine learning models that will enable personalization of the developed tools to meet individual user needs; and 4) an inclusive evaluation framework informed by the advisory board and community partners during the co-design sessions. This work combines input devices beyond keyboards and mice to create an off-the-shelf, personalizable, multimodal input interface for teaching and enabling workforce-ready programming for at least one identified subpopulation of users with similar physical abilities. The prototypes explore combinations of input modalities processed by multimodal, personalized machine learning models to translate inputs to output keystrokes and cursor activity. Prototypes will leverage large-scale pretrained language models to guide the translation of user input to code output and will use a single input modality with a focused output space (i.e., writing Python functions, operators, and program-specific variables); multiple prototypes will be created and tested with the relevant PWD populations. The corpus of data from each modality will be used to identify user sets with similar abilities through clustering techniques, while a centralized learning backbone can be fine-tuned per user population using low-parameter fine-tuning approaches such as Transformer-based Adapters. These technical advances have the potential to significantly expand access to the technology labor force for the PWD community, thereby profoundly impacting the technological workforce opportunities of individuals with physical sensorimotor disabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.