CAREER: Fundamental Research, Design and Evaluation of Auditory and Multimodal Graphs

不管技术的进步如何，盲人学生的科学、数学和工程(SMET)教育以及他们的职业生涯一直面临着一个挑战，那就是检查和解释数据的任务，这通常是由有视力的用户通过视觉图形完成的。虽然已经有许多项目旨在通过以某种方式增加声音来加强SMET教育和盲人获得科学知识的机会，但关于如何准确地创建有效的听觉图形的系统研究很少。如果没有这样的听觉展示工具(与视觉图表相结合)，视障人士参与科学的机会很小，特别是当协作讨论是学习或发现过程的一部分时。在这个项目中，PI将对设计和开发听觉图形的最有效方法进行研究和实证验证，培训用户解释它们，并与视觉图形一起评估它们的有效性和适当性。PI的初步发现表明，视力和视力受损的学生和科学家都可以利用定量信息的听觉显示，只要显示是复杂和设计良好的，听者经过培训，设计师意识到不同的听者在如何解释他们听到的内容方面可能存在的任何差异。多模式展示应该更有效，特别是当教室或研究团队中既有视力也有盲人的参与者时。在这些见解的基础上，PI将首先进行基本实验，旨在确定向听觉图形提供上下文提示的最佳方法，类似于视觉图形中的轴或刻度线。一旦确定了这些听觉图形设计指南，将开发开放源码软件来同时产生听觉和视觉图形，这样教师或其他处理数据的人就不需要重复他们的工作。然后，PI将进行实验，以确定训练听者解读听觉图形的最有效方法，因为克服大多数学生，无论是视力正常还是失明的学生，缺乏听觉显示的经验是至关重要的。还将开发在线培训材料，以确保有可能制作和使用这些复杂听力图的教师具备必要的知识和技能。培训方法和材料将在实验和焦点小组中与从事视力和视障学生工作的教师一起进行评估和验证。最后，通过比较听觉图形和听觉+视觉图形对学习科学概念的学生的学习效果，验证增强的听觉+视觉图形的有效性。所有这些研究都会有教师和视障及视力受损的学生参与。更广泛的影响：设计良好的听觉展示对于视力障碍和视障的学生来说都应该是无价的。拥有多种格式的数据显示将允许所有用户最大限度地学习，并拥有共同的协作基础。正在进行的研究项目将与大学层面的课程捆绑在一起，这样学生就可以在参与研究的过程中学习；结果和发现将被整合到后续的计算机、心理学和辅助技术课程的材料中。视障参与者将成为焦点小组、实验和原型评估的主要组成部分；这将鼓励他们参与未来的研发工作，专注于他们所使用和感兴趣的技术。项目成果将通过PI的各种伙伴组织和国际研究合作者传播；网站将在全球范围内免费提供软件、培训和研究材料。

Regardless of advances in technology, a continual challenge to science, math, and engineering (SMET) education for blind students, and for their careers, is the task of examining and interpreting data, typically accomplished with visual graphs by sighted users. While there have been many projects aimed at enhancing SMET education and scientific access for the blind by adding sound in some manner, there has been very little systematic study of how, exactly, to create effective auditory graphs. Without such auditory display tools (in combination with visual graphs), there is little chance for visually impaired individuals to participate in science, especially when collaborative discussions are part of the learning or discovery process. In this project, the PI will conduct a program of research and empirical validation into the most effective ways to design and develop auditory graphs, train users to interpret them, and evaluate their effectiveness and appropriateness alongside visual graphs. The PI's preliminary findings indicate that both sighted and visually impaired students and scientists can make use of the auditory display of quantitative information, so long as the displays are sophisticated and well designed, the listeners are trained, and the designer is aware of any differences that may exist in how the various listeners tend to interpret what they hear. A multimodal display should be even more effective, especially when there are both sighted and blind participants in a classroom or research team. Building on these insights, the PI will first perform fundamental experiments aimed at determining the optimal methods for providing contextual cues to auditory graphs, analogous to the axis or tick marks in a visual graph. Once these auditory graph design guidelines have been determined, open source software will be developed to produce both auditory and visual graphs simultaneously, so that teachers or others working with data need not duplicate their efforts. The PI will then conduct experiments to determine the most effective ways to train listeners to interpret the auditory graphs, since it is crucial to overcome the lack of experience that most students, sighted or blind, have with auditory displays. Online training materials will also be developed in order to ensure that the teachers who are likely to create and utilize these sophisticated auditory graphs have the requisite knowledge and skills. The training methods and materials will be evaluated and validated in an experiment and focus group with teachers who work with sighted and visually impaired students. Finally, the efficacy of the enhanced auditory+visual graphs will be validated by comparing the learning outcomes for students who are taught science concepts with the aid of auditory vs. auditory+visual graphs. All of these studies will involve the participation of teachers and both visually impaired and sighted students. Broader Impact: Well-designed auditory displays should be invaluable for both sighted and visually impaired students. Having multiple formats for the data displays will allow all users to maximize learning, and have a shared basis for collaboration. The ongoing research projects will be tied in to university level classes, so that students can learn by taking part in the research efforts; the results and findings will then be rolled into the materials for subsequent classes in computing, psychology, and assistive technology. Visually impaired participants will be a major constituent in focus groups, experiments, and prototype evaluation; this will encourage them to become involved in future R & D efforts that focus on technologies of use and of interest to them. Project outcomes will be disseminated through the PI's various partner organizations and international research collaborators; web sites will make the software, training, and research materials freely available worldwide.