SeeStar a mobile pda phone based assistance system for visually impaired users

描述（由申请人提供）：拟议研究的目的是根据第1阶段概念验证结果继续开发“SeeStar”系统。与会者在各种使用条件下使用SeeStar的经验证明了该系统的价值，并指出了需要进一步开发的地方。SeeStar为盲人提供了一双“眼睛”，通过使用手机摄像头和与远程助手的数据链接，远程助手可以通过手机摄像头查看实时图像，然后描述该人的周围环境，指出显著地标的位置和通往特定目的地的路径。此外，使用GPS，助手可以使用谷歌地图来定位人到他们更远的环境。第1阶段参与者以及远程操作员的经验表明，SeeStar的持续开发应该专注于静态而不是动态的使用场景-也就是说，不要试图在人走路时提供“一吹一吹”的方向。此外，与会者明确表示，在没有远程助理的情况下，最有用的是采用自动化手段来探测显著的视觉地标和具体的目的地点。当地标和文本识别算法在这方面得到演示时，与会者表现得最为热情。SeeStar远程操作员的经验也突出了第二阶段需要解决的具体问题：（1）由数据信道容量的波动引起的大于1秒的视频和双向语音接收时延引起的通信困难，（2）在给定摄像机透镜的非常受限的视场的情况下，难以定向到设置，（3）低视频图像分辨率使得难以容易地读取标志和识别物体，以及（4）GPS覆盖和误差。（5）此外，参与者对他们是否觉得把相机拿在手里更容易，还是戴在脖子上更容易，甚至是戴在头上更容易产生分歧。（6）参与者还希望有一种方法来验证他们的预期路线（室内和室外）和目的地，定位和识别行人过路按钮，公共汽车站牌，公共汽车或火车号码，电梯按钮，出口，浴室，办公室号码和街道标志。自动化功能的开发将包括大量现有的最先进但现成的、基于图像的地标识别、文本标志检测和识别以及自动全景创建软件。当参考GIS数据库时，这样的地标和文本标志可以向GPS提供冗余的定位方法，这可能受到覆盖和可用性问题的影响。对象和文本识别的组合还将在没有助理的情况下实现关键显著特征的定位和识别，例如定位和识别公共汽车号码、街道标志、电梯按钮或行人过街按钮。最后，将进行更广泛和统计相关的焦点小组和最终用户测试，以确保我们充分解决了这个非常异质的人群的问题。 公共卫生相关性：从消费者的角度来看，SeeStar必须以一种尚不明显的方式创造公平的竞争环境：“为了有效地在社会中发挥作用，所有人都必须有能力规划可理解的路线，遵循路线，并保持沿着方向前进。人们还必须能够穿越任何给定的位置，而不会被移动的交通所撞击，被新的建筑工地所迷惑，或者在腐烂的人行道或道路上绊倒。有视力障碍的人在实现独立行动方面面临着独特的挑战，因为他们没有简单的方法来识别潜在的危险，或者在没有辅助技术设备或视力正常的人的帮助下立即获得有关其周围环境的信息”[T2 RERC的利益相关者论坛，2003年]。

DESCRIPTION (provided by applicant): The purpose of the proposed research is to continue development of the "SeeStar" system based on Phase 1 proof of concept results. Participant experiences with SeeStar under a variety of use conditions demonstrated the value of the system, and indicated where further development efforts will be required. SeeStar offers persons who are blind a "pair of eyes" via the use of a cell phone camera and data link to a remote assistant who can look at live images through that cell phone camera and then describe the person's surroundings, indicating the location of salient landmarks and pathways to specific destinations. Also, using GPS, the assistant can employ Google Maps to orient the person to their farther reaching surroundings. The experiences of Phase 1 participants, as well as remote operators, made it clear that continued SeeStar development should focus on static rather than dynamic use scenarios -- that is, not attempt to provide "blow by blow" directions as the person is walking. Further, participants made it clear that in the absence of an available remote assistant, an automated means of detecting salient visual landmarks and specific destination points would be most useful. The participants were most enthusiastic when landmark and text recognition algorithms were demonstrated in this regard. The experience of the remote operators of SeeStar also highlighted specific issues to be addressed in Phase 2: (1) difficulties in communicating caused by video and two-way voice reception latencies greater than 1 second caused by fluctuations in data channel capacity, (2) difficulty in orienting to a setting given the very restricted field of view of the camera lens, (3) low video image resolutions made it difficult to read signs and recognize objects easily, and (4) GPS coverage and errors. (5)Further, participants were split over whether they felt it was easier to hold the camera in their hands, or wear it around their neck, or perhaps even on their head. (6)Participants also wanted a means of verifying their intended routes (indoor and outdoor) and destinations, locating and recognizing pedestrian crossing buttons, bus stop signs, bus or train numbers, elevator buttons, exits, bathrooms, office numbers, and street signs. The development the automated capabilities will include a significant amount of existing of state of the art but off the shelf, image based landmark recognition, text sign detection and recognition, and automatic panorama creation software. Such landmarks and text signs when referenced to a GIS database can provide a redundant method of positioning to GPS which may suffer from coverage and availability issues. The combination of object and text recognition will also enable location and identification of critical salient features in the absence of an assistant such locating and recognizing bus numbers, street signs, elevator buttons, or pedestrian crossing buttons. Finally, more extensive and statistically relevant focus group and end user tests will be conducted to insure we have adequately addressed issues of concern for this very heterogeneous population. PUBLIC HEALTH RELEVANCE: Using a from the consumer's perspective, SeeStar must level the playing field in a manner not yet in evidence for: "In order to effectively function within society, all people must have the ability to plan comprehensible routes, follow routes, and maintain orientation along the way. People must also be able to traverse any given location without being struck by moving traffic, confounded by new construction sites, or stumbling upon a decaying sidewalk or roadway. People who have visual impairments face unique challenges in achieving independent mobility, as there is no easy way for them to identify potential hazards, or to immediately access information about their surroundings without an assistive technology device or assistance from a sighted person" [T2RERC's Stakeholder Forum, 2003].