Development of an AI Assistant for Individuals with Low Vision and Blindness

国家眼科研究所估计，在美国，目前有超过100万的合法 到2050年，这一数字将增加到约400万。根据VA盲人办公室的数据， 康复服务，大约有130，000名美国退伍军人谁是法律上的盲人和一个以上的 数百万人因视力低下而失去了执行日常任务的能力。有些人可能 借助相对简单和廉价的设备，如放大镜和手杖。然而，随着更严重的 视力丧失，人们可能会失去独立性，生活质量下降。失明有一个 对退伍军人和其他人的重大影响，事实证明，超过70%的工作年龄的人， 严重视力障碍者失业。 虽然有一些致盲原因可以很容易地纠正，但在许多情况下， 治疗一种解决方案是通过手术将电极放置在视网膜或视觉皮层中来恢复视力， 基于由头戴式摄像机记录的光水平电刺激大脑。该系统 类型正在开发中，具有很大的潜力。然而，植入的药物也有缺点， 系统，包括手术的风险，高成本，和一种形式的视力，是严重限制相比， 正常的感知。还有各种各样的设备和智能手机应用程序， 功能（文本阅读器，障碍物探测器，颜色识别器）。 这项研究的目的是开发和测试一种新型的视觉辅助设备， A-Eyes，适用于失明和低视力的个人。它利用了人工智能的最新进展， 智能，包括计算机视觉、机器学习、光学字符识别、语音识别和 3D声音渲染，使用户能够识别，定位，并与对象和人互动， 他们的附近。换句话说，能够让低视力人群受益的AI技术已经存在， 可以在针对视觉障碍患者的需求量身定制的设备中实现。该系统将是直观的 并且易于学习和交互，因为它理解口头指令并且它向用户说话。Aeyes拥有 根据需求分析和焦点小组的反馈，一对一访谈， 在VA普罗维登斯医疗保健系统对退伍军人进行原型测试。 目标1涉及器械开发。实现了系统的体系结构和算法 和功能，但对软件进行了一系列改进，并集成了单独的功能模块 整个系统都需要。这些改进包括升级到更高分辨率的摄像头， 对识别神经网络进行再训练，并将人脸识别、手部跟踪和语音 识别模块到中央架构。 目标2包括将对具有一定范围低视力的受试者进行的器械测试 包括无光感、色素性视网膜炎、青光眼、黄斑变性和糖尿病在内的病症 视网膜病变要进行的测试包括用户识别和定位人的能力， 并抓住物体，阅读手持纸张上的文本，计算机显示器和标志，并训练系统 认识新的人比较单独剩余视力和添加AEyes的性能， 确定设备作为视觉助手的有效性，并指出需要改进的地方。

The National Eye Institute estimates that in the United States there are presently over 1 million legally blind people and this number will increase to about 4 million by 2050. According to the VA Office of Blind Rehabilitation Services, there are approximately 130,000 US Veterans who are legally blind and more than one million who have lost the ability to perform daily tasks because of low vision. Some of these people can be helped by relatively simple and inexpensive devices such as magnifiers and canes. However, with more severe vision loss, people are likely to experience loss of independence and reduced quality of life. Blindness has a major impact on Veterans and others, as evidenced by the fact that over 70% of working-age people with significant visual impairment are unemployed. While there are causes of blindness that can be readily corrected, in many cases there is no effective treatment. One solution is to restore vision by surgically placing electrodes in the retina or visual cortex and electrically stimulating the brain based on the light level recorded by a head-mounted camera. Systems of this type are being developed and they have great potential. However, there are downsides to the implanted systems that include the risks of surgery, high cost, and a form of vision that is severely limited compared to normal perception. There is also a wide array of devices and smartphone apps that perform specialized functions (text readers, obstacle detectors, color identifiers). The goal of the proposed research is to develop and test a new type of visual assistive device, named AEyes, for individuals with blindness and low vision. It takes advantage of recent advances in artificial intelligence including computer vision, machine learning, optical character recognition, speech recognition, and 3D sound rendering to give the user the ability to recognize, localize, and interact with objects and people in their vicinity. In other words, the AI technology that can benefit people with low vision already exists and it will be implemented in a device tailored to the needs of people with visual disorders. The system will be intuitive and easy to learn and interact with as it understands spoken instructions and it speaks to the user. AEyes has been developed based on a needs-analysis and feedback from focus groups, one-on-one interviews, and prototype tests with Veterans conducted at the VA Providence Healthcare System. Aim 1 concerns device development. The system architecture and algorithms are largely implemented and functional, but a range of refinements to the software and the integration of separate functional modules into the overall system are needed. These improvements include upgrading to a higher resolution camera, retraining the recognition neural network, and integrating the face recognition, hand tracking, and voice recognition modules into the central architecture. Aim 2 consists of device tests that will be conducted with subjects having a range of low vision conditions that include no light sense, retinitis pigmentosa, glaucoma, macular degeneration, and diabetic retinopathy. The tests to be conducted include the ability of a user to recognize and localize people, reach for and grasp objects, read text on handheld paper, computer displays, and signs, and train the system to recognize new people. Comparing performance with residual vision alone and with the addition of AEyes will establish the device’s effectiveness as a visual assistant and indicate where improvements are needed.