SBIR Phase I: Narrative interface technology to support two-way human-computer interaction for the disabled community

这个小型企业创新研究（SBIR）第一阶段项目的更广泛的影响/商业潜力是为大约6100万美国残疾人改善软件的可访问性。交互式技术目前尚未标准化，最终用户必须依赖软件中内置的可访问性功能，这些功能不一致或不存在。拟议的无障碍界面将使最终用户能够利用80%的商业软件，提高残疾人的生活水平，并使他们能够在线专业和社交。同时，这项创新将使软件分销商能够提供低成本的可访问性解决方案，完成所需的任务，并增加其产品的使用和收入。这项创新将克服以前对最先进的无障碍解决方案的限制，因为它将能够适应增强现实（AR）等高强度软件。它还将向后兼容，为旧平台添加功能，并提供强大而持久的竞争优势。这项小型企业创新研究（SBIR）第一阶段项目旨在为盲人、聋人和身体残疾的用户提供交互式技术无障碍，而无需额外的无障碍硬件。机器学习（ML）引擎将视觉数据转换为通过文本或文本到语音传递的文本块，并通过嵌入数据识别视觉信息。自然语言处理（NLP）将用于将动作绑定到口语短语，允许最终用户完全控制集成软件。该项目将测试该界面与个人生产力软件、Web浏览器和视频游戏集成的能力，目标是使用户能够在市场上80%的软件中实现95%的功能。研究和开发将涉及：i）训练ML引擎使用嵌入式数据将视觉数据重新解释为文本，并产生列出用户影响范围内的所有交互对象的动态创建的句子，以及iii）通过创建和实现ML协议来扩展接口的多功能性，以向任何现有或未来的软件添加可访问性功能。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project is to improve software accessibility for the roughly 61 million Americans living with a disability. Interactive technology is not currently standardized, and end-users must rely on accessibility features built into software which are inconsistent or nonexistent. The proposed accessibility interface will enable end-users to utilize 80% of commercially-available software, improving the standard of living for individuals with disabilities and allowing them to engage online professionally and socially. Simultaneously, the innovation will allow software distributors to provide low-cost accessibility solutions, fulfill required mandates, and increase their products’ use and revenue. The innovation will overcome previous limitations for state-of-the-art accessibility solutions as it will be able to accommodate high-intensity software such as augmented reality (AR). It will also be backward-compatible, adding functionality to older platforms and providing a robust and enduring competitive advantage. US-based medical technology companies, game developers, and organizations with accessibility mandates will be initially targeted.This Small Business Innovation Research (SBIR) Phase I project seeks to provide interactive technology accessibility for blind, deaf, and physically disabled users without necessitating additional accessibility hardware. A machine learning (ML) engine will translate visual data into text blocks delivered via text or text-to-speech, with visual information recognized through embedded data. Natural language processing (NLP) will be used to bind actions to spoken phrases, allowing the end-user complete control of the integrated software. This project will test the interface’s ability to integrate with personal productivity software, web browsers, and a video game, with the goal of enabling users to attain 95% functionality across 80% of software on the market. The research and development will involve: i) training the ML engine to use embedded data to reinterpret visual data as text and produce dynamically-created sentences listing all interactive objects within the user’s field of influence, ii) developing the controller interface, which will be driven by user input, listen for approved phrases, and activate desired software controls, and iii) expanding the interface’s versatility by creating and implementing ML protocols to add accessibility features to any existing or future software. The result will be an early prototype with moderate-to-full functionality across selected software.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.