NSF Convergence Accelerator Track H: Phase II Smart Wearables for Expanding Workplace Access for People with Blindness and Low Vision

全世界有2.85亿人患有失明和低视力（pBLV）。视力丧失的一个显著相关因素是失业，城市环境中的失业率高达81%。就业差距在很大程度上是由于上下班交通困难以及在工作场所本身找路困难。该项目通过一个强大的辅助导航平台，即空间智能和车载导航视障智能服务系统（VIS 4 ION），在解决pBLV的就业挑战方面迈出了基础性的一步。VIS 4 ION是一个谨慎的，装有仪器的背包，带有一系列集成到背带中的微型传感器，这些传感器连接到嵌入式系统进行计算分析。实时音频和触觉反馈通过双耳骨传导耳机和可选的重新配置的腰带转向触觉接口提供。一个原型平台，部分通过第一阶段融合加速器赠款开发，为pBLV提供了有价值的微服务，包括实时导航，场景理解和避障，所有这些都可以改变pBLV的工作场所体验。这个第二阶段的项目旨在将目前的VIS 4 ION原型引入商业系统，以在真实的复杂的工作环境中产生影响。这项工作将利用对话式人工智能、机器视觉、网络和触觉的新发展，沿着用户的广泛试验。为了实现这一多方面项目的目标，组建了一个跨学科团队，涵盖技术开发、工程、转化医学和业务管理，并得到了私营企业（Google、AT& T和高通）、pBLV非营利组织（灯塔协会和愿景），以及政府机构的运输和残疾人（MTA、DOT和纽约市残疾人办公室）。该翻译项目将在四个相互关联的任务中执行：（1）为智能可穿戴设备提供独立的基于AI的微服务，特别关注支持室内和室外旅程的导航管道，我们的目标是：（a）实现零基础设施要求;（B）通过定制AI集成和对话AI增强协调的微服务;（c）通过独特的视频速率自适应算法构建网络连接弹性;（d）在交通枢纽、医院和工作场所执行广泛的系统可用性测试。除了对医学，科学，工程和改善pBLV生活的预期影响外，该计划还将为研究生，本科生，医学生和高中生提供临床和基础STEM/STEAM研究的正式和非正式学习机会。这些培训机会将促进多管齐下和多学科的团队方法，这对复杂的现实世界问题的创新解决方案的发展至关重要。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

There are 285 million people with blindness and low vision (pBLV) worldwide. One dramatic correlate of vision loss is unemployment, with rates as high as 81% in urban environments. Employment disparities stem, in large part, from difficulties with transportation to and from work and wayfinding within the workplace itself. This project takes a fundamental step in addressing the employment challenges of pBLV through a powerful assistive-navigation platform, the Visually Impaired Smart Service System for Spatial Intelligence and Onboard Navigation (VIS4ION). VIS4ION is a discreet, instrumented backpack with an array of miniaturized sensors integrated into straps that connect to an embedded system for computational analysis. Real-time audio and tactile feedback are provided through a binaural bone conduction headset and an optional reconfigured waist strap turned haptic interface. A prototype platform, partly developed through the Phase 1 Convergence Accelerator Grant, provides valuable microservices for pBLV, including real-time navigation, scene understanding, and obstacle avoidance, all of which could transform workplace experiences for pBLV. This translational Phase 2 project broadly seeks to bring the current VIS4ION prototype to a commercial system for impact in real complex workplace environments. The work will leverage new developments in conversational AI, machine vision, networking, and haptics, along with extensive trials with users. To realize the goals for this multi-faceted project, a transdisciplinary team has been assembled that spans technology development, engineering, translational medicine, and business management, supported by partners in private industry (Google, AT&T, and Qualcomm), non-profits for pBLV (Lighthouse Guild and VISIONS), and government agencies for transport and disabilities (MTA, DOT, and NYC office for persons with disabilities).This translational project will be executed in four inter-connected tasks: (1) boosting independent AI-based microservices for smart wearables with a specific focus on a navigation pipeline that supports indoor and outdoor journeys, with zero infrastructural requirements; (b) enhancing coordinated microservices through custom AI ensembles and conversational AI; (c) building network connectivity resilience through unique video rate-adaptation algorithms; and (d) executing extensive system usability testing in transit hubs, hospitals, and workplaces. Beyond the envisioned impact on medicine, science, engineering, and improving the lives of pBLV, this program will foster formal and informal learning opportunities for graduate, undergraduate, medical, and high school students in clinical and fundamental STEM/STEAM research. These training opportunities will foster multi-pronged and multi-disciplinary team approaches critical to the development of innovative solutions for complex real-world problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.