Providing Access to Appliance Displays for Visually Impaired Users

描述(申请人提供)：为视障用户提供对家电显示器的访问摘要本项目的目标是开发一种在智能手机和平板电脑上运行的计算机视觉系统，使盲人和视障人士能够阅读家电显示器。这种能力对于使用微波炉和恒温器等配备显示器的各种家用和商业用具越来越必要，对于独立生活、教育和就业也是必不可少的。目前，诸如光学字符识别(OCR)之类的传统文本阅读系统不提供对该信息的访问，光学字符识别(OCR)旨在用于阅读打印文档。我们提出的系统作为软件运行在标准的现成智能手机或平板电脑上，使用计算机视觉算法来分析用户拍摄的视频图像，并检测和读取每个图像中的文本。对于盲人用户，该文本要么使用合成语音大声朗读，要么针对视力低下的用户，以放大、对比度增强的形式呈现(这最适合使用平板电脑)。我们建议在过去的原型显示阅读器工作的基础上，使用强大的新技术，使最终的系统能够阅读更多种类的LED/LCD显示文本字体，并更好地适应经常使阅读显示困难的条件，包括眩光、反射和对比度较差。这些技术包括专门适用于数字显示器领域的新颖字符识别技术；允许用户指向显示器中感兴趣的特定位置的手指检测；使用显示模板作为参考图像以与显示器的噪声图像相匹配并由此帮助解释该显示器的噪声图像；以及将显示器的多个视图集成到单个清晰视图中。特殊的用户界面功能，例如使用手指检测来帮助盲人用户将相机对准显示器，以及针对低视力用户的对比度增强功能，以满足不同用户的特定需求。盲人和视障受试者将定期测试系统，以提供反馈和性能数据，推动系统的发展和持续改进，并将以客观的性能衡量标准进行评估。由此产生的显示阅读器系统将作为智能手机和平板电脑的应用程序发布，任何人都可以免费下载和使用，也可以作为开放源码项目，可以自由构建或修改以用于第三方软件。

DESCRIPTION (provided by applicant): Providing Access to Appliance Displays for Visually Impaired Users Summary The goal of this project is to develop a computer vision system that runs on smartphones and tablets to enable blind and visually impaired persons to read appliance displays. This ability is increasingly necessary to use a variety of household and commercial appliances equipped with displays, such as microwave ovens and thermostats, and is essential for independent living, education and employment. No access to this information is currently afforded by conventional text reading systems such as optical character recognition (OCR), which is intended for reading printed documents. Our proposed system runs as software on a standard, off-the-shelf smartphone or tablet and uses computer vision algorithms to analyze video images taken by the user and to detect and read the text within each image. For blind users, this text is either read aloud using synthesized speech, or for low vision users, presented in magnified, contrast- enhanced form (which is most suited to the use of a tablet). We propose to build on our past work on a prototype display reader using powerful new techniques that will enable the resulting system to read a greater variety of LED/LCD display text fonts, and to better accommodate the conditions that often make reading displays difficult, including glare, reflections and poor contrast. These techniques include novel character recognition techniques adapted specifically to the domain of digital displays; finger detection to allow the user to point to a specific location of interest in the display; the use of display templates as reference images to match to, and thereby help interpret, noisy images of displays; and the integration of multiple views of the display into a single clear view. Special user interface features such as the use of finger detection to help blind users aim the camera towards the display and contrast-enhancement for low vision users address the particular needs of different users. Blind and visually impaired subjects will test the system periodically to provide feedback and performance data, driving the development and continual improvement of the system, which will be assessed with objective performance measures. The resulting display reader system will be released as an app for smartphones and tablets that can be downloaded and used by anybody for free, and also as an open source project that can be freely built on or modified for use in 3rd-party software.