Vision Without Sight: Exploring the Environment with a Portable Camera

描述（由申请人提供）：随着计算机视觉物体识别算法在准确性和速度上的提高，以及计算机变得更加强大和紧凑，在便携式设备（如具有摄像头的手机）上实现此类算法变得越来越实用。这种“移动视觉”方法允许正常视力的用户识别物体、标志、地点和环境中的其他特征，只需拍张照片，等待几秒钟的目标识别分析结果。这种方法对盲人或视障（VI）用户很有希望，因为他们可能没有其他方法来识别非视觉线索无法检测到的重要特征。然而，为了使该方法对VI用户实用，必须适当促进使用相机的用户和环境之间的交互。例如，由于用户可能事先不知道将相机指向目标的哪个方向，因此他或她必须能够左右移动相机以搜索目标，并在检测到目标时获得快速反馈。借鉴PI和他的合作者过去在VI用户对象识别系统方面的经验，我们建议研究使用移动视觉技术来探索环境中的特征，具体检查发现这些特征并获得对它们的指导的过程。我们的主要目标是调查这些技术的用户采用的策略，以加快探索过程，设计和测试与这些策略一致的最有效的用户界面，并评估和基准技术的效率。结果将是一套最小设计标准，将指定系统性能参数，用户界面功能和任何移动视觉对象识别系统为VI用户所必需的操作策略。

DESCRIPTION (provided by applicant): As computer vision object recognition algorithms improve in accuracy and speed, and computers become more powerful and compact, it is becoming increasingly practical to implement such algorithms on portable devices such as camera-enabled cell phones. This "mobile vision" approach allows normally sighted users to identify objects, signs, places and other features in the environment simply by snapping a photo and waiting a few seconds for the results of the object recognition analysis. The approach holds great promise for blind or visually impaired (VI) users, who may have no other means of identifying important features that are undetectable by non-visual cues. However, in order for the approach to be practical for VI users, the interaction between the user and the environment using the camera must be properly facilitated. For instance, since the user may not know in advance which direction to point the camera towards a desired target, he or she must be able to pan the camera left and right to search for it, and receive rapid feedback whenever it is detected. Drawing on past experience of the PI and his collaborators on object recognition systems for VI users, we propose to study the use of mobile vision technologies for exploring features in the environment, specific examining the process of discovering these features and obtaining guidance towards them. Our main objectives are to investigate the strategies adopted by users of these technologies to expedite the exploration process, devise and test maximally effective user interfaces consistent with these strategies, and to assess and benchmark the efficiency of the technologies. The result will be a set of minimum design standards that will specify the system performance parameters, the user interface functionality and the operational strategies necessary for any mobile vision object recognition system for VI users. PUBLIC HEALTH RELEVANCE: The ability to locate and identify objects, places and other features in the environment is taken for granted every day by the sighted, but this fundamental capability is missing or severely degraded in the approximately 10 million Americans with significant vision impairments and a million who are legally blind. The proposed research would investigate how computer vision object recognition technologies, which are now being implemented on mobile vision devices such as cell phones but are typically designed for normally sighted users, could be modified and harnessed to meet the special needs of blind and visually impaired persons. Such research could lead to new technologies to dramatically improve independence for this population