VIDI: Visual Information Dissemination for Visually Impaired Individuals

这是三年持续奖励的第一年资助。PI的论点是，计算机视觉领域已经成熟到一个完全便携式的系统，能够在接近实时的环境中发现符号和其他文本事件，然后大声读出它们的内容，这是可行的。他在这个项目中的目标是为视障人士提供一种从环境中获取关键信息的方法，而这些信息现在是完全无法获得的（除非通过视力正常的人的干预）。在连接陀螺仪参考头戴式摄像头的可穿戴计算机上运行，设想的系统将以马赛克的形式检测用户周围连续视频的紧凑表示中的文本和符号。标志检测和识别系统将使用一组多尺度特征，这些特征已被证明对视角和照明的变化是不变的。识别将通过建立这些特征分布的多分辨率空间网络，并将它们与已知（根据地区定制）标志（例如国际公路标志）的数据库进行匹配来完成，使用粗到细的匹配技术，该技术既稳健又高效。在商业OCR系统的帮助下，使用相同的技术检测文本。PI还将检查OCR的中间表示，以及一个单词字典以容忍字符识别错误。他将利用现有的工作，从使用图像内容的大型数据库中检索图像，人脸识别，使用颜色属性的图像索引，以及复杂图像中的文本检测和识别。这项工作将分三个阶段进行，每个阶段大致相当于一年。系统设计和评估都将与专门从事视障人士康复培训的灯塔国际合作进行。在第一阶段的工作中，将开发包括相机稳定、马赛克、文本和符号识别在内的各个组成部分。在第二阶段，将开发头戴式单元、姿态推断、可穿戴计算机、摄像头、语音合成器和用户界面模型。此外，将研究对数极相机的使用，以进行多尺度特征的快速计算，以及同时解决视场和分辨率问题的有效性。第三阶段和最后阶段将包括一系列用户和系统评价和改进。毫无疑问，如果成功的话，这项研究将对视障人士的生活产生巨大的影响，并显著改善他们的生活。

This is the first year funding of a three year continuing award. The PI's contention is that the field of computer vision has matured to the point where a completely portable system capable of finding signs and other textual events in the environment in close to real time, and then reading their contents aloud, is feasible. His goal in this project is to provide the visually impaired with a way to obtain crucial information from the environment that is now totally inaccessible (except through intervention of a sighted person). Running on a wearable computer connected to a gyroscopically referenced head mounted camera, the envisaged system will detect text and signs within a compact representation of continuous video of the user's surroundings in the form of a mosaic. The sign detection and recognition system will use a set of multi-scale features that have been shown to be invariant to changes in viewing angle and illumination. Recognition will be accomplished by building multi-resolution spatial networks of distributions of these features and matching them against a database of known (customized to locale) signs (e.g. international highway signs) using a coarse to fine matching technique that is both robust and efficient. Text will be detected using the same techniques, with the aid of a commercial OCR system. The PI will also examine the OCR's intermediate representations, and a word dictionary for tolerance to character recognition errors. He will leverage existing work on image retrieval from large databases using image content, face recognition, image indexing using color properties, and text detection and recognition in complex images. The work will be performed in three phases roughly corresponding to a year each. System design and evaluation will both be conducted in conjunction with Lighthouse International, which specializes in rehabilitation training for the visually impaired. In the first phase of the work, individual components, which include camera stabilization, mosaics, text and sign recognition, will be developed. In the second phase, a head mounted unit, attitude inference, a wearable computer, a camera, speech synthesizer and user interface model will be developed. Additionally, the use of log-polar cameras will be examined for fast computation of multi-scale features and for its effectiveness in simultaneously addressing the field of view and resolution issues. The third and last phase will consist of a series of user and system evaluations and refinements. There is no doubt that, if successful, this research will have a tremendous impact on and dramatically improve the lives of visually impaired individuals.