CHS: Medium: Improving the Accessibility of Mobile Applications by Enabling Third-Party Assessment, Repair, and Enhancement

许多移动的应用程序（app）没有正确地实现帮助残疾人使用计算机的无障碍服务，从而有效地拒绝了许多弱势群体对应用程序和相关服务的访问。该项目将开发新的方法来评估，修复和增强大量应用程序的可访问性，而无需访问应用程序或手机操作系统的源代码。为此，研究团队将开发新的应用程序接口像素级分析，以评估它们是否正确实现了手机平台提供的支持可访问性的应用程序编程接口（API），然后发布数据集，以了解整个应用程序生态系统的可访问性。该团队还将开发新方法，允许第三方构建在现有应用程序上运行的替代“代理”界面，并实现可访问性API，以及针对特定人的能力和需求个性化这些代理的新方法。该团队将在工具的设计和评估方面与为残疾人服务的合作伙伴密切合作，为这些合作伙伴带来直接利益，并通过积极传播工具，方法和数据集来支持其他人进行无障碍研究，从而带来更广泛的社会效益。该项目还将为从小学到格拉德生院的学生提供研究和教育机会，该团队专门针对来自计算研究中代表性不足的群体的学生，包括残疾人。为了进行评估，该团队将开发在移动的手机模拟器中执行应用程序的方法，使用常见任务导航其界面，捕获所述接口及其可访问性API表示的图像，以及评估从所捕获的图像恢复的基于像素的接口层次结构是否正确地对应于所述API表示。该团队将与残障人士合作，开发有效的分析摘要和演示文稿，开发可扩展的基础设施和有效的分析，以寻找可访问性故障类型的模式，这些故障如何在应用程序和开发人员中分布，以及它们如何受到应用程序更新的影响。对于修复和增强，该团队将开发交互代理，利用平台的可访问性服务来创建覆盖底层应用界面的覆盖窗口，并与之交换事件。他们还将开发设计模式，将常见的可访问性故障与修复策略联系起来，提供库以在交互代理中快速实现这些模式，以支持第三方-并最终自动修复这些常见故障。最后，团队将探索如何通过交互代理广泛部署响应个人能力的新交互技术，使用模型-视图/控制器范式来管理将特定增强应用于特定接口的组合复杂性。

Many mobile applications (apps) do not properly implement accessibility services that help people with disabilities use computers, effectively denying access to apps and associated services for many vulnerable people. This project will develop new approaches to assess, repair, and enhance accessibility across large numbers of apps without needing access to source code of an app or the phone's operating system. To do this the research team will develop new pixel-level analyses of app interfaces to assess whether they correctly implement application programming interfaces (APIs) that the phone platform provides to support accessibility, then release datasets that give a picture of accessibility across the ecosystem of apps. The team will also develop new approaches that allow third parties to build alternate "proxy" interfaces that run on top of an existing app and implement accessibility APIs, as well as new approaches for those proxies to be personalized to a particular person's abilities and needs. The team will work closely with partners who serve people with disabilities in both design and evaluation of the tools, bringing direct benefits to those partners, as well as broader social benefits through actively disseminating the tools, methods, and datasets to support others in doing accessibility research. The project will also provide research and education opportunities for students from grade to grad school, with the team specifically targeting students from groups that are underrepresented in computing research, including those with disabilities.For assessment, the team will develop methods for executing an application in a mobile phone emulator, navigating its interface using common tasks, capturing both images of the interface and its accessibility API representation, and evaluating whether a recovered pixel-based interface hierarchy from the captured image correctly corresponds to the API representation. Working with people with disabilities to develop effective summaries and presentations of the analyses, the team will develop scalable infrastructure and efficient analyses to look for patterns in types of accessibility failures, how these failures are distributed across apps and developers, and how they are affected by app updates. For repair and enhancement, the team will develop interaction proxies that leverage the platform's accessibility services to create overlay windows that cover the underlying app interface and exchange events with it. They will also develop design patterns that link common accessibility failures with repair strategies, providing libraries to quickly implement these patterns in interaction proxies to support both third-party and eventually automatic repair of these common failures. Finally, the team will explore how novel interaction techniques that respond to individual abilities could be broadly deployed via interaction proxies, using a Model-View/Controller paradigm to manage the combinatorial complexity of applying particular enhancements to particular interfaces.