SeeStar a mobile pda phone based assistance system for visually impaired users

SeeStar 是一款针对视障用户的基于移动 PDA 电话的辅助系统

• 批准号: 7483480
• 负责人: TIM DORCEY
• 金额: $ 40.6万
• 依托单位: IVISIT, LLC
• 依托单位国家: 美国
• 财政年份: 2005
• 资助国家: 美国
• 起止时间: 2005-09-30 至 2010-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7483480
• 关键词: Address; Advanced Development; Algorithms; Arts; Awareness; Behavior; Books; Cellular Phone; Code; Collaborations; Communication; Computer software; Computer-Assisted Image Analysis; Computers; Condition; Data; Databases; Depth Perception; Destinations; Detection; Development; Drops; Elevator; Employee Strikes; Engineering; Evaluation; Evolution; Eye; Face; Facility Construction Funding Category; Focus Groups; Funding; Generic Drugs; Geographic Information Systems; Goals; Hand; Head; Human; Image; Lead; Learning; Life; Light; Lighting; Link; Location; Maps; Methods; Neck; Numbers; Operating System; Operative Surgical Procedures; Pan Genus; Participant; Pathway interactions; Performance; Persons; Phase; Play; Population; Population Heterogeneity; Positioning Attribute; Principal Investigator; Printed Media; Process; Protocols documentation; Public Health; Purpose; Qualifying; Rate; Reading; Recommendation; Rehabilitation Research; Relative (related person); Research; Research Personnel; Research Project Grants; Resolution; Resource Development; Robotics; Route; Safety; Scanning; Scientist; Site; Societies; Solutions; Specialist; Speech; Speed; System; Tactile; Telephone; Testing; Text; Time; Training; Travel; Trees; Update; Veterans; Vision; Visual; Visual impairment; Voice; Walking; Work; Writing; assistive device/technology; base; blind; concept; design; desire; evaluation/testing; experience; hazard; improved; interest; lens; multidisciplinary; novel; object recognition; object shape; optical character recognition; preference; size; software development; trafficking; transmission process; visual information

DESCRIPTION (provided by applicant): The purpose of the proposed research is to continue development of the "SeeStar" system based on Phase 1 proof of concept results. Participant experiences with SeeStar under a variety of use conditions demonstrated the value of the system, and indicated where further development efforts will be required. SeeStar offers persons who are blind a "pair of eyes" via the use of a cell phone camera and data link to a remote assistant who can look at live images through that cell phone camera and then describe the person's surroundings, indicating the location of salient landmarks and pathways to specific destinations. Also, using GPS, the assistant can employ Google Maps to orient the person to their farther reaching surroundings. The experiences of Phase 1 participants, as well as remote operators, made it clear that continued SeeStar development should focus on static rather than dynamic use scenarios -- that is, not attempt to provide "blow by blow" directions as the person is walking. Further, participants made it clear that in the absence of an available remote assistant, an automated means of detecting salient visual landmarks and specific destination points would be most useful. The participants were most enthusiastic when landmark and text recognition algorithms were demonstrated in this regard. The experience of the remote operators of SeeStar also highlighted specific issues to be addressed in Phase 2: (1) difficulties in communicating caused by video and two-way voice reception latencies greater than 1 second caused by fluctuations in data channel capacity, (2) difficulty in orienting to a setting given the very restricted field of view of the camera lens, (3) low video image resolutions made it difficult to read signs and recognize objects easily, and (4) GPS coverage and errors. (5)Further, participants were split over whether they felt it was easier to hold the camera in their hands, or wear it around their neck, or perhaps even on their head. (6)Participants also wanted a means of verifying their intended routes (indoor and outdoor) and destinations, locating and recognizing pedestrian crossing buttons, bus stop signs, bus or train numbers, elevator buttons, exits, bathrooms, office numbers, and street signs. The development the automated capabilities will include a significant amount of existing of state of the art but off the shelf, image based landmark recognition, text sign detection and recognition, and automatic panorama creation software. Such landmarks and text signs when referenced to a GIS database can provide a redundant method of positioning to GPS which may suffer from coverage and availability issues. The combination of object and text recognition will also enable location and identification of critical salient features in the absence of an assistant such locating and recognizing bus numbers, street signs, elevator buttons, or pedestrian crossing buttons. Finally, more extensive and statistically relevant focus group and end user tests will be conducted to insure we have adequately addressed issues of concern for this very heterogeneous population. PUBLIC HEALTH RELEVANCE: Using a from the consumer's perspective, SeeStar must level the playing field in a manner not yet in evidence for: "In order to effectively function within society, all people must have the ability to plan comprehensible routes, follow routes, and maintain orientation along the way. People must also be able to traverse any given location without being struck by moving traffic, confounded by new construction sites, or stumbling upon a decaying sidewalk or roadway. People who have visual impairments face unique challenges in achieving independent mobility, as there is no easy way for them to identify potential hazards, or to immediately access information about their surroundings without an assistive technology device or assistance from a sighted person" [T2RERC's Stakeholder Forum, 2003].

描述（由申请人提供）：拟议研究的目的是基于概念结果的第一阶段证明，继续开发“ Seestar”系统。参与者在各种使用条件下的Seestar经验证明了系统的价值，并指出了需要进一步开发工作的何处。 Seestar通过使用手机摄像头和与远程助手的数据链接可以通过手机摄像机查看实时图像，然后描述该人的周围环境，这表明明显地标和通往特定目的地的途径的位置。此外，使用GPS，助手可以使用Google地图将其定向到更远的环境。第1阶段参与者以及远程运营商的经验清楚地表明，持续的Seestar开发应集中在静态而不是动态使用方案上 - 也就是说，随着人的行走，不要试图提供“打击”方向。此外，参与者明确表示，在没有可用的远程助手的情况下，一种自动化的方法来检测显着的视觉地标和特定的目的地是最有用的。当在这方面证明了地标和文本识别算法时，参与者最热情。 Seestar的远程操作员的经验还强调了在第2阶段中要解决的特定问题：（1）由视频和双向语音接收潜伏期所引起的沟通困难，这是由于数据渠道容量波动的波动而导致的1秒钟，（2）在将摄像机镜头的视野和范围的范围限制为限制的情况下，（3）难以识别的镜头和（3）启用摄像机和读者的范围（3）读取的距离（3）读取的镜头（3）读者的读者（3）读取（3）读取的对象（3）（3）读取的对象（3）（3）读取的对象（3）（3）读取（3）（3）读取的对象（3）（3）错误。 （5）此外，参与者是否认为将相机握在手中，或将其戴在脖子上，甚至在头上更容易。 （6）参与者还希望一种验证其预期路线（室内和室外）和目的地，定位和识别行人交叉按钮，公共汽车停车标志，公共汽车或火车号码，电梯按钮，出口，浴室，办公室号码和街道标志的方法。自动化功能的开发将包括大量现有的最新技术，但在架子上，基于图像的地标识别，文本标志检测和识别以及自动全景创建软件。当引用GIS数据库时，此类地标和文本标志可以为GPS提供冗余的定位方法，该方法可能会遇到覆盖范围和可用性问题。在没有助手的情况下，对象和文本识别的组合还将启用关键特征的位置和识别。最后，将进行更广泛和统计相关的焦点小组和最终用户测试，以确保我们已经充分解决了这个非常异构的人群的关注问题。 公共卫生相关性：使用从消费者的角度来看，Seestar必须以尚未证据的方式平衡竞争环境：“为了有效地在社会内部运作，所有人都必须能够规划可理解的途径，遵循途径并沿途遵循途径。障碍在实现独立流动性方面面临着独特的挑战，因为他们没有简单的方法可以识别潜在的危害，或者在没有辅助技术设备或视力的人的援助的情况下立即获取有关其周围环境的信息” [T2rerc的利益相关者论坛，2003年]。