Semantics-Preserving Virtualization: A Computing System Framework to Run Any Screen Reader on Any Device with Easy Customization

 描述(申请人提供)：计算机用户越来越多地拥有多种设备，包括台式机、手机、平板电脑和笔记本电脑，并且经常使用为不同操作系统(OS)设计的应用程序。例如，Mac用户可以访问基于云的Windows远程桌面，以运行其工作所需的应用程序。多设备和操作系统的趋势给有视觉障碍的用户带来了负担：这些用户的工作效率取决于特定fic屏幕阅读器的功能，而这些阅读器与特定fic操作系统捆绑在一起。目前唯一的选择是在每个平台上运行不同的屏幕阅读器-这损害了生产率，导致了额外的费用。拟议工作的目标是将未经修改的二进制屏幕阅读器与原始OS平台分离，为有视觉障碍的用户在访问运行于任何平台和任何设备上的各种应用程序时提供无缝体验，而无需切换他们的首选屏幕阅读器。这一建议建立在这样一种见解之上，即这一相似点是应用程序用户界面(UI)的语义；每个操作系统上的应用程序都由类似的小部件组成，如按钮、下拉菜单和文本fi字段。跨平台移植屏幕阅读器的主要障碍是辅助功能API的独特性、繁琐和跨平台变化的程度。新的概念是保持语义的虚拟化，它是围绕一个简单的应用程序UI模型而设计的，该模型既是最小的，又是超fi的，可以很好地表示任何应用程序的UI。这个通用的用户界面模型被用来驱动跨平台的转换，比如从Windows的用户界面模型到通用模型，然后再到Mac用户界面模型，这样苹果的原生屏幕阅读器VoiceOver就可以读取Windows应用程序，就像它在Mac上运行一样。我们将把这些翻译集成到远程桌面客户端和操作系统模拟器中，以将屏幕阅读器从操作系统中分离出来。该项目的具体目标包括：1)开发一个计算系统框架，以在语义层上虚拟化用户界面；2)开发多个概念验证实现，其中使用为一个操作系统设计的屏幕阅读器来使在不同操作系统上运行的应用程序可以访问；4)探索一个简单的编程模型，用于编写对辅助技术的定制和扩展；以及3)通过最终用户案例研究来验证系统的可用性。该项目将使有视觉障碍的用户能够使用任何设备，而不会造成不成比例的生产力损失-最终使他们能够过上更独立的生活，改善就业前景。因此，该项目推进了普遍可访问性的愿景，即任何人都应该能够使用任何计算设备获得信息时代的好处，不受任何残疾的限制。除了对用户的实际影响外，该项目还将通过开发一种新的应用程序用户界面模型并消除不必要地阻碍无障碍软件进步的系统级障碍，对计算机可访问性和操作系统的fi领域产生研究影响。

 DESCRIPTION (provided by applicant): Computer users increasingly have multiple devices, including desktops, phones, tablets, and laptops, and often use applications designed for different operating systems (OSes). For instance, a Mac user may access a cloud-based Windows remote desktop to run an application required for her job. The trend toward multiple devices and OSes creates a burden for users with visual impairments: these users' productivity depends on features of a specific screen reader, and these readers are tied to a specific OS. The only current option is to run a different screen reader on each platform-harming productivity incurring additional expense. The goal of the proposed work is to decouple unmodified, binary screen readers from the original OS platform, giving users with visual impairments a seamless experience when accessing assorted applications running on any platform and any device without having to switch their preferred screen reader. This proposal is built around the insight that this point of similarity is the semantics of application User In- terfaces (UIs); applications on every OS consist of similar widgets such as buttons, drop-down menus, and text fields. The primary obstacle to porting screen readers across platforms are the degree to which accessibility APIs are idiosyncratic, cumbersome, and varied across platforms. The novel concept is semantics-preserving virtualization, which is designed around a simple model of application UIs that is both minimal and sufficiently expressive to represent any application's UI. This generic UI model is used to drive cross-platform translations, from, say Windows' UI model into the generic model, and then to the Mac UI model such that Apple's native screen reader, VoiceOver, can read a Windows application as if it were running on the Mac. We will integrate these translations into both remote desktop clients and OS emulators to decouple screen readers from the OS. The specific aims of this project include: 1) Developing a computing systems framework to virtualize user interfaces at the semantic level, 2) Developing multiple proof-of-concept implementations where a screen reader designed for one OS is used to make applications running on a different OS accessible, 4) Exploring a simple programming model for writing customizations and extensions to assistive technologies, and 3) Validating the usability of the system with end-user case studies. This project will empower users with visual impairments to use any device without a disproportionate loss of productivity-ultimately allowing them to lead more independent lives with improved employment prospects. Thus, this project advances the vision of Universal Accessibility, that anyone should be able to reap the benefits of the information age with any computing device, unconstrained by any disability. In addition to the practical impact for users, the project will have a research impacton the field of Computer Accessibility and Operating Systems by developing a novel model for application user interfaces and removing systems-level obstacles that needlessly delay advances in accessibility software.