CAREER: Fundamental Research, Design and Evaluation of Auditory and Multimodal Graphs

无论技术如何进步，对盲人学生的科学、数学和工程（SMET）教育及其职业生涯的持续挑战是检查和解释数据的任务，通常由视力正常的用户通过可视化图表完成。虽然有许多项目旨在通过以某种方式添加声音来加强盲人的SMET教育和科学获取，但关于如何准确地创建有效的听觉图表的系统研究却很少。如果没有这样的听觉显示工具（与视觉图形相结合），视障人士参与科学的机会就很小，特别是当合作讨论是学习或发现过程的一部分时。在这个项目中，PI将对设计和开发听觉图形的最有效方法进行研究和实证验证，培训用户解释它们，并评估它们与视觉图形一起的有效性和适当性。PI的初步研究结果表明，视力正常和视力受损的学生和科学家都可以利用定量信息的听觉展示，只要这种展示是复杂的、设计良好的，听众是经过训练的，并且设计师意识到不同的听众在解释他们所听到的内容时可能存在的差异。多模式显示应该更有效，特别是当教室或研究团队中既有视力正常的人也有盲人的时候。在这些见解的基础上，PI将首先进行基础实验，旨在确定为听觉图形提供上下文线索的最佳方法，类似于视觉图形中的轴或标记。一旦确定了这些听觉图形设计指南，开源软件就会被开发出来，同时生成听觉和视觉图形，这样教师或其他处理数据的人就不需要重复他们的工作。然后，PI将进行实验，以确定训练听众解释听觉图表的最有效方法，因为这对于克服大多数学生，无论视力正常或失明，对听觉显示缺乏经验至关重要。我们还将开发在线培训材料，以确保可能创建和利用这些复杂听觉图表的教师具备必要的知识和技能。培训方法和材料将在一个实验中进行评估和验证，并由与视力和视力受损学生一起工作的教师进行焦点小组讨论。最后，通过比较使用听觉和听觉+视觉图形学习科学概念的学生的学习成果，验证增强的听觉+视觉图形的有效性。所有这些研究都需要教师、视障学生和正常学生的参与。更广泛的影响：设计良好的听觉显示对于视力和视力受损的学生来说都是无价的。拥有多种格式的数据显示将允许所有用户最大限度地学习，并有一个共享的协作基础。正在进行的研究项目将与大学水平的课程挂钩，这样学生就可以通过参与研究工作来学习；这些结果和发现将被纳入计算机、心理学和辅助技术等后续课程的材料中。视障参与者将成为焦点小组、实验和原型评估的主要组成部分；这将鼓励他们参与未来的研发工作，重点关注他们使用和感兴趣的技术。项目成果将通过PI的各伙伴组织和国际研究合作者传播；网站将使软件、培训和研究材料在世界范围内免费提供。

Regardless of advances in technology, a continual challenge to science, math, and engineering (SMET) education for blind students, and for their careers, is the task of examining and interpreting data, typically accomplished with visual graphs by sighted users. While there have been many projects aimed at enhancing SMET education and scientific access for the blind by adding sound in some manner, there has been very little systematic study of how, exactly, to create effective auditory graphs. Without such auditory display tools (in combination with visual graphs), there is little chance for visually impaired individuals to participate in science, especially when collaborative discussions are part of the learning or discovery process. In this project, the PI will conduct a program of research and empirical validation into the most effective ways to design and develop auditory graphs, train users to interpret them, and evaluate their effectiveness and appropriateness alongside visual graphs. The PI's preliminary findings indicate that both sighted and visually impaired students and scientists can make use of the auditory display of quantitative information, so long as the displays are sophisticated and well designed, the listeners are trained, and the designer is aware of any differences that may exist in how the various listeners tend to interpret what they hear. A multimodal display should be even more effective, especially when there are both sighted and blind participants in a classroom or research team. Building on these insights, the PI will first perform fundamental experiments aimed at determining the optimal methods for providing contextual cues to auditory graphs, analogous to the axis or tick marks in a visual graph. Once these auditory graph design guidelines have been determined, open source software will be developed to produce both auditory and visual graphs simultaneously, so that teachers or others working with data need not duplicate their efforts. The PI will then conduct experiments to determine the most effective ways to train listeners to interpret the auditory graphs, since it is crucial to overcome the lack of experience that most students, sighted or blind, have with auditory displays. Online training materials will also be developed in order to ensure that the teachers who are likely to create and utilize these sophisticated auditory graphs have the requisite knowledge and skills. The training methods and materials will be evaluated and validated in an experiment and focus group with teachers who work with sighted and visually impaired students. Finally, the efficacy of the enhanced auditory+visual graphs will be validated by comparing the learning outcomes for students who are taught science concepts with the aid of auditory vs. auditory+visual graphs. All of these studies will involve the participation of teachers and both visually impaired and sighted students. Broader Impact: Well-designed auditory displays should be invaluable for both sighted and visually impaired students. Having multiple formats for the data displays will allow all users to maximize learning, and have a shared basis for collaboration. The ongoing research projects will be tied in to university level classes, so that students can learn by taking part in the research efforts; the results and findings will then be rolled into the materials for subsequent classes in computing, psychology, and assistive technology. Visually impaired participants will be a major constituent in focus groups, experiments, and prototype evaluation; this will encourage them to become involved in future R & D efforts that focus on technologies of use and of interest to them. Project outcomes will be disseminated through the PI's various partner organizations and international research collaborators; web sites will make the software, training, and research materials freely available worldwide.