CHS: Medium: Improving the Accessibility of Mobile Applications by Enabling Third-Party Assessment, Repair, and Enhancement

许多移动应用程序（app）没有正确实现帮助残疾人使用计算机的无障碍服务，从而使许多弱势群体无法使用应用程序和相关服务。该项目将开发新的方法来评估、修复和增强大量应用程序的可访问性，而无需访问应用程序或手机操作系统的源代码。为此，研究团队将开发新的像素级应用程序接口分析，以评估它们是否正确实现了手机平台提供的支持可访问性的应用程序编程接口（api），然后发布数据集，提供整个应用程序生态系统的可访问性图片。该团队还将开发新的方法，允许第三方构建在现有应用程序之上运行的替代“代理”接口，并实现可访问性api，以及为这些代理提供个性化的新方法，以满足特定个人的能力和需求。该团队将在工具的设计和评估方面与为残疾人提供服务的合作伙伴密切合作，通过积极传播工具、方法和数据集来支持其他人进行无障碍研究，为这些合作伙伴带来直接利益，并带来更广泛的社会效益。该项目还将为从小学到研究生的学生提供研究和教育机会，该团队特别针对那些在计算机研究中代表性不足的群体的学生，包括残疾学生。为了评估，该团队将开发在移动电话模拟器中执行应用程序的方法，使用常见任务导航其界面，捕获界面图像及其可访问性API表示，并评估从捕获的图像中恢复的基于像素的界面层次结构是否正确地对应于API表示。与残障人士一起开发分析的有效总结和演示，团队将开发可扩展的基础设施和有效的分析，以寻找可访问性故障类型的模式，这些故障如何分布在应用程序和开发人员之间，以及它们如何受到应用程序更新的影响。为了修复和增强，团队将开发交互代理，利用平台的可访问性服务来创建覆盖底层应用程序界面的覆盖窗口，并与之交换事件。他们还将开发设计模式，将常见的可访问性故障与修复策略联系起来，提供库来在交互代理中快速实现这些模式，以支持第三方并最终自动修复这些常见故障。最后，该团队将探索如何通过交互代理广泛部署响应个人能力的新颖交互技术，使用模型-视图/控制器范式来管理将特定增强应用于特定接口的组合复杂性。

Many mobile applications (apps) do not properly implement accessibility services that help people with disabilities use computers, effectively denying access to apps and associated services for many vulnerable people. This project will develop new approaches to assess, repair, and enhance accessibility across large numbers of apps without needing access to source code of an app or the phone's operating system. To do this the research team will develop new pixel-level analyses of app interfaces to assess whether they correctly implement application programming interfaces (APIs) that the phone platform provides to support accessibility, then release datasets that give a picture of accessibility across the ecosystem of apps. The team will also develop new approaches that allow third parties to build alternate "proxy" interfaces that run on top of an existing app and implement accessibility APIs, as well as new approaches for those proxies to be personalized to a particular person's abilities and needs. The team will work closely with partners who serve people with disabilities in both design and evaluation of the tools, bringing direct benefits to those partners, as well as broader social benefits through actively disseminating the tools, methods, and datasets to support others in doing accessibility research. The project will also provide research and education opportunities for students from grade to grad school, with the team specifically targeting students from groups that are underrepresented in computing research, including those with disabilities.For assessment, the team will develop methods for executing an application in a mobile phone emulator, navigating its interface using common tasks, capturing both images of the interface and its accessibility API representation, and evaluating whether a recovered pixel-based interface hierarchy from the captured image correctly corresponds to the API representation. Working with people with disabilities to develop effective summaries and presentations of the analyses, the team will develop scalable infrastructure and efficient analyses to look for patterns in types of accessibility failures, how these failures are distributed across apps and developers, and how they are affected by app updates. For repair and enhancement, the team will develop interaction proxies that leverage the platform's accessibility services to create overlay windows that cover the underlying app interface and exchange events with it. They will also develop design patterns that link common accessibility failures with repair strategies, providing libraries to quickly implement these patterns in interaction proxies to support both third-party and eventually automatic repair of these common failures. Finally, the team will explore how novel interaction techniques that respond to individual abilities could be broadly deployed via interaction proxies, using a Model-View/Controller paradigm to manage the combinatorial complexity of applying particular enhancements to particular interfaces.