NSF Convergence Accelerator Track H: Determining Community Needs for Accessibility Tools that Facilitate Programming Education and Workforce Readiness for Persons with Disabilities

10亿人，占世界人口的15%，经历了残疾。残疾是进入劳动力的主要障碍；大约80%的残疾人被排除在2021年的劳动力之外。与此同时，编程工作继续爆炸性增长，但残疾人基本上无法接触到编程工作。标准的编程接口--屏幕、键盘、鼠标--对于许多面临物理挑战的残疾人来说很难操作。该项目支持身体残疾的个人，这些残疾导致学习和参与编程的障碍，阻碍了获得广泛可用的、有利可图的和向上流动的技术劳动力的机会。这项工作将开发一种方法，通过开发和评估多模式接口(例如，语音、眼球跟踪、踏板)的原型，使残疾人能够学习、练习和利用编程技能，从而改善残疾人面临的负面劳动结果。该项目还将为残疾人士开辟一条培训和进入编程工作队伍的途径，从而弥合阻碍大多数残疾人士获得此类职业机会的职业差距。该项目的影响包括：1)增加残疾人士在STEM工作中的比例，2)增加残疾人士的经济和个人福祉，3)提高美国的经济竞争力，4)加强研究和教育的基础设施。使残疾人能够获得编程技能和就业，将在技术部门产生大量残疾人，增加STEM的多样性，并为残疾人带来持续就业和经济福祉。由于编程工作通常可以远程完成，这项工作将为残疾人，特别是那些使用轮椅的残疾人消除交通障碍。由于工作稳定性是许多残疾人士最关心的问题，这将导致生活质量的持续改善。此外，增加能够编程的劳动力将提高美国的经济竞争力，同时增加劳动力的多样性。最后，开发有效的、用户友好的和可持续的计算机接口方式也将在K-12和全国范围内的研究中有用。这个项目将产生多项技术进步和贡献，包括：1)关于从事学习和练习劳动力准备编程的相关残障人群需求的大型语料库；2)共同设计的可个性化的原型界面，特别设计为负担得起、可访问和跨平台工作；3)机器学习模型，使开发的工具能够个性化，以满足个人用户的需求；以及4)由咨询委员会和社区合作伙伴在共同设计会议期间提供信息的包容性评价框架。这项工作将键盘和鼠标以外的输入设备结合在一起，创建了一个现成的、可个性化的多模式输入界面，用于教学并为至少一个确定的具有类似身体能力的用户亚群提供劳动力就绪的编程。原型探索了由多模式、个性化的机器学习模型处理的输入模式的组合，以将输入转换为输出击键和光标活动。原型将利用大规模预先训练的语言模型来指导用户输入到代码输出的转换，并将使用具有重点输出空间的单一输入通道(即，编写Python函数、运算符和特定于程序的变量)；将创建多个原型，并使用相关的PWD种群进行测试。来自每个通道的数据语料库将用于通过集群技术识别具有相似能力的用户集，而集中式学习主干可以使用低参数微调方法(如基于Transformer的适配器)针对每个用户群体进行微调。这些技术进步有可能显著扩大残疾人社区获得技术劳动力的机会，从而深刻影响身体感觉运动障碍个人的技术劳动力机会。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

One billion people, 15% of the world’s population, have experienced a disability. Disabilities present major barriers to entering the labor force; approximately 80% of persons with disabilities (PWDs) were excluded from the 2021 labor force. Meanwhile, programming jobs continue to grow explosively but are largely inaccessible to PWDs. Standard programming interfaces–-screens, keyboards, mice–-are difficult to operate for many PWDs with physical challenges. This project supports individuals with physical disabilities that result in barriers to learning and engaging in programming, blocking access to the widely available, lucrative, and upwardly mobile technology workforce. The work will develop a means of ameliorating negative labor outcomes faced by PWDs by developing and evaluating prototypes of multimodal interfaces (e.g., speech, eye tracking, pedals) that enable PWDs to learn, practice, and utilize programming skills. The project will also develop a path for PWDs to train for—and enter—the programming workforce, thereby bridging the career gap that blocks most PWDs from such career opportunities. Impacts of this project include: 1) increased representation of PWDs in STEM jobs, 2) increased economic and personal well-being for PWDs, 3) improved economic competitiveness of the U.S., and 4) enhanced infrastructure for research and education. Enabling PWDs access to programming skills and employment will produce an influx of PWDs in the technology sector, increase diversity in STEM, and lead to sustained employment and economic well-being for PWDs. Since programming jobs can often be done remotely, this work will remove transportation barriers for PWDs, especially those who use wheelchairs. As job stability is a primary concern for many PWDs, this will lead to sustained improvements in quality of life. Furthermore, increasing the workforce of people who can program will improve US economic competitiveness while increasing the diversity of the workforce. Finally, developing effective, user-friendly, and sustainable ways of interfacing with computers will also be useful in K-12 and in research nationwide. It will allow for early programming education and computer use in K-12 settings for children with physical disabilities, and for aging seniors who experience loss of dexterity and fine motor control.This project will produce multiple technical advances and contributions, including: 1) a large corpus of insights about the relevant PWD subpopulation needs for engaging in learning and practicing workforce-ready programming; 2) co-designed personalizable prototype interfaces specifically designed to be affordable, accessible, and work across platforms; 3) machine learning models that will enable personalization of the developed tools to meet individual user needs; and 4) an inclusive evaluation framework informed by the advisory board and community partners during the co-design sessions. This work combines input devices beyond keyboards and mice to create an off-the-shelf, personalizable, multimodal input interface for teaching and enabling workforce-ready programming for at least one identified subpopulation of users with similar physical abilities. The prototypes explore combinations of input modalities processed by multimodal, personalized machine learning models to translate inputs to output keystrokes and cursor activity. Prototypes will leverage large-scale pretrained language models to guide the translation of user input to code output and will use a single input modality with a focused output space (i.e., writing Python functions, operators, and program-specific variables); multiple prototypes will be created and tested with the relevant PWD populations. The corpus of data from each modality will be used to identify user sets with similar abilities through clustering techniques, while a centralized learning backbone can be fine-tuned per user population using low-parameter fine-tuning approaches such as Transformer-based Adapters. These technical advances have the potential to significantly expand access to the technology labor force for the PWD community, thereby profoundly impacting the technological workforce opportunities of individuals with physical sensorimotor disabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.