CAREER: Making Digital Imagery Accessible to Blind and Low-Vision Users via Audiohaptic Dioramas

职业：通过视听立体模型让盲人和弱视用户可以访问数字图像

• 批准号: 2339788
• 负责人: Brian Smith
• 金额: $ 79.84万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-06-01 至 2029-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339788&HistoricalAwards=false
• 关键词: CAREER; Making; Digital; Imagery; Accessible

Blind and low-vision people are largely left out of visual experiences on computers, affecting many aspects of daily life such as navigating, shopping, reading the news, and staying in touch with friends. Computers currently convey images to blind users via simple text descriptions, which does not allow users to form their own interpretations. This project seeks to enhance the quality of life for blind people by introducing the concept of audiohaptic dioramas, which allows users to learn what things look like in a way similar to real life: by exploring objects' three-dimensional structure via touch, gleaning physical details such as depth, shape, and material. Audiohaptic dioramas will work with everyday touchscreens, allowing blind users to touch or point at an image and sense its physical details via intuitive audio and haptic cues. This technology will give blind people greater access to images, films, video games, virtual reality, and the Web. The investigator will partner with leading technology companies to make audiohaptic dioramas available on mainstream consumer apps. The project's educational component will make K-12 textbooks accessible to blind students by supplementing printed images with audiohaptic dioramas. It will also train blind students to become technology designers, a field hard to enter without sight, catalyzing future efforts to make technology more inclusive.This research program advances a decades-long problem in accessibility research: understanding how to make images equivalently accessible to blind users. Through the concept of audiohaptic dioramas, this research will contribute the first techniques for conveying physical details of three-dimensional scenes via sound and haptics in a way that is compatible with everyday touchscreen devices. The research program will follow a user-centered design and evaluation process to ensure that blind users’ needs are met, involving blind community and lab members throughout and partnering with blind-serving organizations. This project will lead to several scientific contributions: (1) a framework for conveying still images’ depth, shape, material, and color intuitively via sound and haptics on a per-pixel basis as users explore them on a touchscreen; (2) techniques for enabling blind people to independently perform everyday visual tasks such as navigation, finding a building entrance, or finding a specific color shirt using their smartphone camera or smartglasses; and (3) techniques for enabling blind people to visualize interactive virtual environments from video games and virtual reality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

盲人和低视力人群在很大程度上无法获得计算机的视觉体验，从而影响了日常生活的许多方面，例如导航、购物、阅读新闻以及与朋友保持联系。目前，计算机通过简单的文本描述向盲人用户传达图像，这不允许用户形成自己的解释。该项目旨在通过引入视听立体模型的概念来提高盲人的生活质量，该概念允许用户以类似于现实生活的方式了解事物的样子：通过触摸探索物体的三维结构，收集深度、形状和材料等物理细节。视听立体模型将与日常触摸屏配合使用，允许盲人用户触摸或指向图像，并通过直观的音频和触觉提示感知其物理细节。这项技术将使盲人能够更好地接触图像、电影、视频游戏、虚拟现实和网络。研究人员将与领先的技术公司合作，在主流消费应用程序上提供视听立体模型。该项目的教育部分将通过用视听立体模型补充印刷图像，使 K-12 教科书可供盲人学生使用。它还将培训盲人学生成为技术设计师，这是一个没有视力就很难进入的领域，从而促进未来使技术更具包容性的努力。该研究项目推进了无障碍研究中长达数十年的问题：了解如何使盲人用户能够平等地访问图像。通过音频触觉立体模型的概念，这项研究将提供第一个通过声音和触觉传达三维场景的物理细节的技术，以与日常触摸屏设备兼容的方式。该研究计划将遵循以用户为中心的设计和评估流程，以确保盲人用户的需求得到满足，包括盲人社区和实验室成员的全程参与，并与盲人服务组织合作。该项目将带来几项科学贡献：（1）当用户在触摸屏上探索静态图像时，通过声音和触觉以每个像素为基础直观地传达静态图像的深度、形状、材质和颜色的框架； (2) 使盲人能够独立执行日常视觉任务的技术，例如导航、寻找建筑物入口或使用智能手机摄像头或智能眼镜找到特定颜色的衬衫； (3) 使盲人能够从视频游戏和虚拟现实中可视化交互式虚拟环境的技术。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。