SeeStar a mobile pda phone based assistance system for visually impaired users

描述（由申请人提供）：拟议研究的目的是在第一阶段概念验证结果的基础上继续开发“SeeStar”系统。参与者在各种使用条件下使用SeeStar的经验证明了该系统的价值，并指出了需要进一步开发的地方。SeeStar为盲人提供“一双眼睛”，通过使用手机摄像头和数据链接到远程助手，后者可以通过手机摄像头查看实时图像，然后描述该人的周围环境，指出显著地标的位置和通往特定目的地的路径。此外，使用GPS，助手还可以使用谷歌地图将用户定位到更远的环境。第一阶段参与者以及远程操作员的经验清楚地表明，SeeStar的持续开发应该关注静态而不是动态使用场景——也就是说，不要试图在人行走时提供“逐个”的方向。此外，与会者明确表示，在没有远程助手的情况下，一种检测明显的视觉地标和特定目的地的自动化手段将是最有用的。当地标和文本识别算法在这方面进行演示时，与会者最感兴趣。SeeStar远程操作人员的经验还突出了第二阶段需要解决的具体问题：(1)由于数据信道容量波动导致的视频和双向语音接收延迟超过1秒，导致通信困难；(2)由于相机镜头的视野非常有限，难以定位到设置中；(3)视频图像分辨率低，难以轻松读取标志和识别物体；(4)GPS覆盖范围和误差。此外，参与者在他们是否觉得把相机拿在手里更容易，还是把它戴在脖子上，甚至戴在头上更容易的问题上产生了分歧。(6)参与者还希望有一种方法来验证他们的预定路线（室内和室外）和目的地，定位和识别人行横道按钮、公交车站标志、公共汽车或火车号码、电梯按钮、出口、浴室、办公室号码和街道标志。自动化能力的开发将包括大量现有的先进技术，基于图像的地标识别，文本符号检测和识别，以及自动全景创建软件。当参考GIS数据库时，这种地标和文本标志可以为GPS提供一种冗余的定位方法，而GPS可能受到覆盖和可用性问题的影响。物体和文本识别的结合还将在没有助手的情况下实现关键特征的定位和识别，例如定位和识别公交车号码、街道标志、电梯按钮或行人过街按钮。最后，将进行更广泛和统计相关的焦点小组和最终用户测试，以确保我们已经充分解决了这一非常多样化的人群所关注的问题。公共卫生相关性：从消费者的角度出发，SeeStar必须以一种尚未有证据的方式创造公平的竞争环境：“为了有效地在社会中发挥作用，所有人都必须有能力规划可理解的路线，遵循路线，并在途中保持方向。”人们还必须能够穿越任何给定的地点，不被移动的车辆撞到，不被新的建筑工地弄糊涂，也不绊倒在腐朽的人行道或道路上。视力受损的人在实现独立行动方面面临着独特的挑战，因为他们没有简单的方法来识别潜在的危险，或者在没有辅助技术设备或视力正常的人的帮助下立即获取周围环境的信息”[T2RERC的利益相关者论坛，2003]。

DESCRIPTION (provided by applicant): The purpose of the proposed research is to continue development of the "SeeStar" system based on Phase 1 proof of concept results. Participant experiences with SeeStar under a variety of use conditions demonstrated the value of the system, and indicated where further development efforts will be required. SeeStar offers persons who are blind a "pair of eyes" via the use of a cell phone camera and data link to a remote assistant who can look at live images through that cell phone camera and then describe the person's surroundings, indicating the location of salient landmarks and pathways to specific destinations. Also, using GPS, the assistant can employ Google Maps to orient the person to their farther reaching surroundings. The experiences of Phase 1 participants, as well as remote operators, made it clear that continued SeeStar development should focus on static rather than dynamic use scenarios -- that is, not attempt to provide "blow by blow" directions as the person is walking. Further, participants made it clear that in the absence of an available remote assistant, an automated means of detecting salient visual landmarks and specific destination points would be most useful. The participants were most enthusiastic when landmark and text recognition algorithms were demonstrated in this regard. The experience of the remote operators of SeeStar also highlighted specific issues to be addressed in Phase 2: (1) difficulties in communicating caused by video and two-way voice reception latencies greater than 1 second caused by fluctuations in data channel capacity, (2) difficulty in orienting to a setting given the very restricted field of view of the camera lens, (3) low video image resolutions made it difficult to read signs and recognize objects easily, and (4) GPS coverage and errors. (5)Further, participants were split over whether they felt it was easier to hold the camera in their hands, or wear it around their neck, or perhaps even on their head. (6)Participants also wanted a means of verifying their intended routes (indoor and outdoor) and destinations, locating and recognizing pedestrian crossing buttons, bus stop signs, bus or train numbers, elevator buttons, exits, bathrooms, office numbers, and street signs. The development the automated capabilities will include a significant amount of existing of state of the art but off the shelf, image based landmark recognition, text sign detection and recognition, and automatic panorama creation software. Such landmarks and text signs when referenced to a GIS database can provide a redundant method of positioning to GPS which may suffer from coverage and availability issues. The combination of object and text recognition will also enable location and identification of critical salient features in the absence of an assistant such locating and recognizing bus numbers, street signs, elevator buttons, or pedestrian crossing buttons. Finally, more extensive and statistically relevant focus group and end user tests will be conducted to insure we have adequately addressed issues of concern for this very heterogeneous population. PUBLIC HEALTH RELEVANCE: Using a from the consumer's perspective, SeeStar must level the playing field in a manner not yet in evidence for: "In order to effectively function within society, all people must have the ability to plan comprehensible routes, follow routes, and maintain orientation along the way. People must also be able to traverse any given location without being struck by moving traffic, confounded by new construction sites, or stumbling upon a decaying sidewalk or roadway. People who have visual impairments face unique challenges in achieving independent mobility, as there is no easy way for them to identify potential hazards, or to immediately access information about their surroundings without an assistive technology device or assistance from a sighted person" [T2RERC's Stakeholder Forum, 2003].