Semantics-Preserving Virtualization: A Computing System Framework to Run Any Screen Reader on Any Device with Easy Customization

保留语义的虚拟化：一种计算系统框架，可在任何设备上运行任何屏幕阅读器并轻松定制

• 批准号: 9360624
• 负责人: Donald Elliott Porter
• 金额: $ 30.53万
• 依托单位: UNIV OF NORTH CAROLINA CHAPEL HILL
• 依托单位国家: 美国
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-03-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9360624
• 关键词: Adopted; Age; Android; Apple; Case Study; Categories; Cellular Phone; Client; Computer Systems; Computer software; Computers; Custom; Development; Devices; Drops; Employment; Encapsulated; Engineering; Face; Future; Generic Drugs; Goals; Investments; Lead; Libraries; Licensing; Linux; Memory; Modeling; Modernization; Muscle; Occupations; Operating System; Phonation; Process; Productivity; Reader; Reading; Research; Running; Self-Help Devices; Semantics; Speed; System; Tablets; Technology; Text; Translating; Translations; Vision; Visual impairment; Work; Writing; base; blind; cloud based; cost; design; disability; empowered; experience; improved; insight; laptop; new technology; novel; platform-independent; productivity loss; public health relevance; tool; trend; two-dimensional; ubiquitous computing; usability; virtual

 DESCRIPTION (provided by applicant): Computer users increasingly have multiple devices, including desktops, phones, tablets, and laptops, and often use applications designed for different operating systems (OSes). For instance, a Mac user may access a cloud-based Windows remote desktop to run an application required for her job. The trend toward multiple devices and OSes creates a burden for users with visual impairments: these users' productivity depends on features of a specific screen reader, and these readers are tied to a specific OS. The only current option is to run a different screen reader on each platform-harming productivity incurring additional expense. The goal of the proposed work is to decouple unmodified, binary screen readers from the original OS platform, giving users with visual impairments a seamless experience when accessing assorted applications running on any platform and any device without having to switch their preferred screen reader. This proposal is built around the insight that this point of similarity is the semantics of application User In- terfaces (UIs); applications on every OS consist of similar widgets such as buttons, drop-down menus, and text fields. The primary obstacle to porting screen readers across platforms are the degree to which accessibility APIs are idiosyncratic, cumbersome, and varied across platforms. The novel concept is semantics-preserving virtualization, which is designed around a simple model of application UIs that is both minimal and sufficiently expressive to represent any application's UI. This generic UI model is used to drive cross-platform translations, from, say Windows' UI model into the generic model, and then to the Mac UI model such that Apple's native screen reader, VoiceOver, can read a Windows application as if it were running on the Mac. We will integrate these translations into both remote desktop clients and OS emulators to decouple screen readers from the OS. The specific aims of this project include: 1) Developing a computing systems framework to virtualize user interfaces at the semantic level, 2) Developing multiple proof-of-concept implementations where a screen reader designed for one OS is used to make applications running on a different OS accessible, 4) Exploring a simple programming model for writing customizations and extensions to assistive technologies, and 3) Validating the usability of the system with end-user case studies. This project will empower users with visual impairments to use any device without a disproportionate loss of productivity-ultimately allowing them to lead more independent lives with improved employment prospects. Thus, this project advances the vision of Universal Accessibility, that anyone should be able to reap the benefits of the information age with any computing device, unconstrained by any disability. In addition to the practical impact for users, the project will have a research impacton the field of Computer Accessibility and Operating Systems by developing a novel model for application user interfaces and removing systems-level obstacles that needlessly delay advances in accessibility software.

 描述（由申请人提供）：计算机用户越来越多地拥有多种设备，包括台式机、电话、平板电脑和笔记本电脑，并且经常使用为不同操作系统（OS）设计的应用程序。 例如，Mac用户可以访问基于云的Windows远程桌面来运行其工作所需的应用程序。 多设备和操作系统的趋势给有视觉障碍的用户带来了负担：这些用户的工作效率取决于特定屏幕阅读器的功能，而这些阅读器与特定操作系统绑定。目前唯一的选择是在每个平台上运行不同的屏幕阅读器，这会损害工作效率，产生额外的费用。 拟议工作的目标是将未经修改的艾德二进制屏幕阅读器与原始操作系统平台解耦，为视觉障碍用户提供无缝体验，使其在访问任何平台和任何设备上运行的各种应用程序时，无需切换其首选屏幕阅读器。 这个建议是建立在这样一个观点的基础上的，即这个相似点是应用程序用户界面（UI）的语义;每个操作系统上的应用程序都由类似的小部件组成，如按钮、下拉菜单和文本字段。 跨平台移植屏幕阅读器的主要障碍是可访问性API的特殊性、繁琐性和跨平台变化的程度。 新概念是语义保留虚拟化，它是围绕一个简单的应用程序UI模型设计的，该模型既最小又足够表达任何应用程序的UI。 这个通用UI模型用于驱动跨平台转换，从Windows的UI模型转换为通用模型，然后转换为Mac UI模型，这样Apple的原生屏幕阅读器VoiceOver就可以像在Mac上运行一样读取Windows应用程序。 我们将把这些翻译集成到远程桌面客户端和操作系统模拟器中，以将屏幕阅读器与操作系统分离。该项目的具体目标包括：1）开发计算系统框架以在语义级别虚拟化用户界面，2）开发多个概念验证实现，其中针对一个OS设计的屏幕阅读器用于使在不同OS上运行的应用程序可访问，4）探索一个简单的编程模型，用于编写辅助技术的定制和扩展，以及3）通过最终用户案例研究验证系统的可用性。 该项目将使视障用户能够使用任何设备，而不会造成不成比例的生产力损失，最终使他们能够过上更独立的生活，改善就业前景。 因此，该项目推进了普遍无障碍的愿景，即任何人都应该能够通过任何计算设备获得信息时代的贝内，不受任何残疾的限制。 除了对用户的实际影响外，该项目还将通过开发应用程序用户界面的新模型和消除不必要地延迟无障碍软件进步的系统级障碍，对计算机无障碍和操作系统领域产生研究影响。