Enhancing Programming and Machine Learning Education for Students with Visual Impairments through the Use of Compilers, AI and Cloud Technologies

有吸引力的高薪和高度灵活的计算机科学职业应该更容易为失明或视觉障碍（BVI）的人获得。不幸的是，由于两个主要困难，教授BVI学生所需的计算机编程和数据科学技能极具挑战性。第一个困难来自当前屏幕阅读器的有限能力，无法正确读取由英文字母、数字和标点符号组成的计算机代码。编程中使用的一组特殊的按键也不便于屏幕阅读器读取（例如，空格和制表符）。第二个困难来自耗时和令人沮丧的代码导航，使用BVI的学生必须反复使用屏幕阅读器阅读每一行，以找到所需的行进行编辑。该项目将与圣安东尼奥盲人和视力受损灯塔合作，开发新的无障碍工具，包括程序语法和语义感知屏幕阅读器和基于语音命令的代码导航框架，以解决上述两个困难。这些辅助工具将通过基于云的网络界面提供给全国的学生和教育工作者。该项目的成功将提高对英属维尔京群岛学生的计算机编程和数据科学教学的有效性，这反过来将增加更多英属维尔京群岛个人参与计算科学的高收入职业机会，并可能导致更多样化的计算机科学劳动力。这些易访问性工具将使用编译器、人工智能（AI）和云技术，根据它们的含义读取计算机代码语句，而不是一次只读取一个字符。屏幕阅读器将清晰地表达初级程序员需要的必要信息，并帮助他们更容易地理解计算机编程和数据科学中使用的词汇和语义。基于语音命令的代码导航将采用语音识别和自然语言处理，以便学生能够使用他们的声音轻松定位代码中的特定语句（例如，变量声明）。这些辅助工具将集成到Jupyter笔记本中，并通过云提供给全国的学生和教育工作者。这种基于云的解决方案还允许使用复杂的人工智能模型，而不需要学生拥有强大而昂贵的计算机来运行这些辅助工具。该项目将采用单案例研究设计对这些辅助工具进行系统评估，以加深对如何将编译器、人工智能和云计算等技术应用于英属维尔京群岛学生的计算机科学技能教学的理解。评估还将对不同语音风格的有效性提供反馈，并为这些辅助工具的未来改进提供额外的反馈。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Attractive high-paying and highly flexible Computer Science careers should be more readily accessible for people with blindness or visual impairments (BVI). Unfortunately, teaching the required computer programming and data science skills to students with BVI is extremely challenging due to two major difficulties. The first difficulty comes from the limited capability of current screen readers to properly read computer codes that are a mix of English letters, digits, and punctuation marks. The specialized set of keystrokes used in programming is also not conveniently read by screen readers (e.g., spaces and tabs). The second difficulty comes from time-consuming and frustrating code navigation, whereby students with BVI must repeatedly use screen readers to read every line to locate the desired line for editing. Partnering with San Antonio Lighthouse for the Blind and Vision Impaired, the project will develop new accessibility tools, including a program syntax- and semantics-aware screen reader and a voice-command-based code navigation framework to address the above two difficulties. These accessibility tools will be offered through cloud-based web interfaces to provide nationwide access to students and educators. The success of this project will improve the effectiveness of teaching computer programming and data science to students with BVI, which in turn will increase accessibility for more individuals with BVI to participate in Computing Science with high-paying career opportunities and could lead to a more-diverse Computer Science workforce. These accessibility tools will use compilers, Artificial Intelligence (AI), and cloud technologies to read computer code statements based on their meanings, rather than only reading one character at a time. The screen reader will articulate the necessary information that beginning coders need and help them more easily understand the lexicon and semantics used in computer programming and data science. The voice-command-based code navigation will employ speech recognition and natural language processing so that students will be able to use their voice to easily locate a specific statement (e.g., a variable declaration) within their code. These accessibility tools will be integrated into Jupyter notebook and offered through the cloud which will give nationwide access to students and educators. This cloud-based solution will also allow sophisticated AI models to be employed without requiring the students to have powerful and expensive computers to run these accessibility tools. The project will conduct a systematic evaluation of these accessibility tools using single-case research design to deepen the understanding of how technologies, including compilers, AI, and cloud computing, can be applied to teaching Computer Science skills to students with BVI. The evaluation will also provide feedback on the effectiveness of different speech styles and provide additional feedback for future improvements of these accessibility tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.