CAREER: Fundamental Research, Design and Evaluation of Auditory and Multimodal Graphs

无论技术如何进步，盲人学生的科学、数学和工程（SMET）教育及其职业生涯面临的一个持续挑战是检查和解释数据的任务，通常由视力正常的用户使用视觉图形完成。 虽然有许多项目旨在通过以某种方式添加声音来加强SMET教育和盲人的科学接触，但很少有系统的研究如何准确地创建有效的听觉图表。 如果没有这样的听觉显示工具（结合视觉图形），视障人士参与科学的机会就很小，特别是当协作讨论是学习或发现过程的一部分时。 在这个项目中，PI将进行一项研究和实证验证计划，以最有效的方式设计和开发听觉图表，培训用户解释它们，并评估它们的有效性和适当性以及视觉图表。 PI的初步研究结果表明，视力正常和视力受损的学生和科学家都可以利用定量信息的听觉显示，只要显示器是复杂的和精心设计的，听众是训练有素的，设计者意识到可能存在的任何差异，不同的听众往往如何解释他们听到的。 多模态显示应该更有效，特别是当教室或研究团队中既有视力正常的参与者，也有盲人参与者时。 基于这些见解，PI将首先进行基础实验，旨在确定为听觉图提供上下文线索的最佳方法，类似于视觉图中的轴或刻度线。 一旦确定了这些听觉图设计准则，将开发开源软件，同时生成听觉和视觉图，这样教师或其他处理数据的人就不必重复工作。 然后PI将进行实验，以确定最有效的方法来训练听众解释听觉图形，因为它是至关重要的，以克服缺乏经验，大多数学生，视力或盲人，与听觉显示。 还将开发在线培训材料，以确保可能创建和使用这些复杂听觉图表的教师具备必要的知识和技能。 培训方法和材料将在一个实验和焦点小组中进行评估和验证，该小组由与视力和视力受损学生一起工作的教师组成。 最后，增强的听觉+视觉图形的功效将通过比较谁是教科学概念的学生与听觉与听觉+视觉图形的帮助下的学习成果进行验证。 所有这些研究都将涉及教师和视力受损和视力正常的学生的参与。 更广泛的影响：精心设计的听觉显示器对视力正常和视力受损的学生都是非常宝贵的。 数据显示的多种格式将使所有用户能够最大限度地学习，并有一个共同的协作基础。 正在进行的研究项目将与大学水平的课程联系在一起，这样学生就可以通过参加研究工作来学习;结果和发现将被纳入计算机，心理学和辅助技术的后续课程的材料中。 视力受损的参与者将成为焦点小组、实验和原型评估的主要组成部分;这将鼓励他们参与未来的研发工作，重点是他们感兴趣的技术。 项目成果将通过PI的各种伙伴组织和国际研究合作者传播;网站将在全世界免费提供软件、培训和研究材料。

Regardless of advances in technology, a continual challenge to science, math, and engineering (SMET) education for blind students, and for their careers, is the task of examining and interpreting data, typically accomplished with visual graphs by sighted users. While there have been many projects aimed at enhancing SMET education and scientific access for the blind by adding sound in some manner, there has been very little systematic study of how, exactly, to create effective auditory graphs. Without such auditory display tools (in combination with visual graphs), there is little chance for visually impaired individuals to participate in science, especially when collaborative discussions are part of the learning or discovery process. In this project, the PI will conduct a program of research and empirical validation into the most effective ways to design and develop auditory graphs, train users to interpret them, and evaluate their effectiveness and appropriateness alongside visual graphs. The PI's preliminary findings indicate that both sighted and visually impaired students and scientists can make use of the auditory display of quantitative information, so long as the displays are sophisticated and well designed, the listeners are trained, and the designer is aware of any differences that may exist in how the various listeners tend to interpret what they hear. A multimodal display should be even more effective, especially when there are both sighted and blind participants in a classroom or research team. Building on these insights, the PI will first perform fundamental experiments aimed at determining the optimal methods for providing contextual cues to auditory graphs, analogous to the axis or tick marks in a visual graph. Once these auditory graph design guidelines have been determined, open source software will be developed to produce both auditory and visual graphs simultaneously, so that teachers or others working with data need not duplicate their efforts. The PI will then conduct experiments to determine the most effective ways to train listeners to interpret the auditory graphs, since it is crucial to overcome the lack of experience that most students, sighted or blind, have with auditory displays. Online training materials will also be developed in order to ensure that the teachers who are likely to create and utilize these sophisticated auditory graphs have the requisite knowledge and skills. The training methods and materials will be evaluated and validated in an experiment and focus group with teachers who work with sighted and visually impaired students. Finally, the efficacy of the enhanced auditory+visual graphs will be validated by comparing the learning outcomes for students who are taught science concepts with the aid of auditory vs. auditory+visual graphs. All of these studies will involve the participation of teachers and both visually impaired and sighted students. Broader Impact: Well-designed auditory displays should be invaluable for both sighted and visually impaired students. Having multiple formats for the data displays will allow all users to maximize learning, and have a shared basis for collaboration. The ongoing research projects will be tied in to university level classes, so that students can learn by taking part in the research efforts; the results and findings will then be rolled into the materials for subsequent classes in computing, psychology, and assistive technology. Visually impaired participants will be a major constituent in focus groups, experiments, and prototype evaluation; this will encourage them to become involved in future R & D efforts that focus on technologies of use and of interest to them. Project outcomes will be disseminated through the PI's various partner organizations and international research collaborators; web sites will make the software, training, and research materials freely available worldwide.