Development of an AI Assistant for Individuals with Low Vision and Blindness

国家眼科研究所估计，目前在美国合法的有100多万人 到2050年，这一数字将增加到约400万。据退伍军人事务部盲人办公室称 康复服务，大约有13万美国退伍军人在法律上失明，而且不止一人 数百万人因为视力低下而失去了执行日常任务的能力。其中一些人可能是 借助相对简单和便宜的设备，如放大镜和拐杖。然而，随着更严重的 视力丧失时，人们可能会失去独立性，生活质量下降。失明有一种 对退伍军人和其他人的重大影响，这一事实证明了这一事实，70%以上的劳动适龄人口 严重视力障碍的人都失业了。 虽然有一些失明的原因可以很容易地纠正，但在许多情况下，没有有效的 治疗。一种解决方案是通过手术在视网膜或视觉皮质放置电极来恢复视力，并 根据头戴式摄像机记录的光线水平对大脑进行电刺激。这样的系统 类型正在开发中，它们具有巨大的潜力。然而，植入物也有缺点。 包括手术风险、高成本和一种严重限制视力的系统 正常的感觉。还有各种各样的设备和智能手机应用程序可以执行专门的 功能(文本阅读器、障碍物探测器、颜色识别器)。 这项拟议研究的目标是开发和测试一种新型的视觉辅助设备，名为 A-Eyes，适用于失明和视力低下的人。它利用了人工智能领域的最新进展 智能，包括计算机视觉、机器学习、光学字符识别、语音识别和 3D声音渲染，使用户能够识别、定位和与中的对象和人进行交互 他们就在附近。换句话说，能够造福低视力人群的AI技术已经存在，而且将会存在 在一种为视力障碍患者的需求量身定做的设备中实施。该系统将是直观的 并且易于学习和交互，因为它理解口头指令并与用户交谈。AEyes有 基于需求分析和来自焦点小组的反馈、一对一访谈和 在退伍军人中进行的原型测试在退伍军人普罗维登斯医疗系统进行。 目标1涉及设备开发。基本实现了系统架构和算法 和功能，但对软件进行了一系列改进，并集成了独立的功能模块 进入整个系统是必要的。这些改进包括升级到更高分辨率的摄像头， 重新训练识别神经网络，并将人脸识别、手部跟踪和语音 将识别模块整合到中央架构中。 目标2包括对视力低下的受试者进行的设备测试 包括无光感、视网膜色素变性、青光眼、黄斑变性和糖尿病的情况 视网膜病变。要进行的测试包括用户识别和本地化人员的能力，联系 抓取物体，阅读手持纸、计算机显示器和标志上的文本，并训练系统 认识新的人。比较仅有残余视力和增加AEyes的情况下的表现 确定设备作为视觉助手的有效性，并指出需要改进的地方。

The National Eye Institute estimates that in the United States there are presently over 1 million legally blind people and this number will increase to about 4 million by 2050. According to the VA Office of Blind Rehabilitation Services, there are approximately 130,000 US Veterans who are legally blind and more than one million who have lost the ability to perform daily tasks because of low vision. Some of these people can be helped by relatively simple and inexpensive devices such as magnifiers and canes. However, with more severe vision loss, people are likely to experience loss of independence and reduced quality of life. Blindness has a major impact on Veterans and others, as evidenced by the fact that over 70% of working-age people with significant visual impairment are unemployed. While there are causes of blindness that can be readily corrected, in many cases there is no effective treatment. One solution is to restore vision by surgically placing electrodes in the retina or visual cortex and electrically stimulating the brain based on the light level recorded by a head-mounted camera. Systems of this type are being developed and they have great potential. However, there are downsides to the implanted systems that include the risks of surgery, high cost, and a form of vision that is severely limited compared to normal perception. There is also a wide array of devices and smartphone apps that perform specialized functions (text readers, obstacle detectors, color identifiers). The goal of the proposed research is to develop and test a new type of visual assistive device, named AEyes, for individuals with blindness and low vision. It takes advantage of recent advances in artificial intelligence including computer vision, machine learning, optical character recognition, speech recognition, and 3D sound rendering to give the user the ability to recognize, localize, and interact with objects and people in their vicinity. In other words, the AI technology that can benefit people with low vision already exists and it will be implemented in a device tailored to the needs of people with visual disorders. The system will be intuitive and easy to learn and interact with as it understands spoken instructions and it speaks to the user. AEyes has been developed based on a needs-analysis and feedback from focus groups, one-on-one interviews, and prototype tests with Veterans conducted at the VA Providence Healthcare System. Aim 1 concerns device development. The system architecture and algorithms are largely implemented and functional, but a range of refinements to the software and the integration of separate functional modules into the overall system are needed. These improvements include upgrading to a higher resolution camera, retraining the recognition neural network, and integrating the face recognition, hand tracking, and voice recognition modules into the central architecture. Aim 2 consists of device tests that will be conducted with subjects having a range of low vision conditions that include no light sense, retinitis pigmentosa, glaucoma, macular degeneration, and diabetic retinopathy. The tests to be conducted include the ability of a user to recognize and localize people, reach for and grasp objects, read text on handheld paper, computer displays, and signs, and train the system to recognize new people. Comparing performance with residual vision alone and with the addition of AEyes will establish the device’s effectiveness as a visual assistant and indicate where improvements are needed.