CHS: Medium: Improving the Accessibility of Mobile Applications by Enabling Third-Party Assessment, Repair, and Enhancement

CHS：中：通过启用第三方评估、修复和增强来提高移动应用程序的可访问性

• 批准号: 1702751
• 负责人: James Fogarty
• 金额: $ 119.85万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1702751&HistoricalAwards=false
• 关键词: CHS; Medium; Improving; Accessibility; Mobile

Many mobile applications (apps) do not properly implement accessibility services that help people with disabilities use computers, effectively denying access to apps and associated services for many vulnerable people. This project will develop new approaches to assess, repair, and enhance accessibility across large numbers of apps without needing access to source code of an app or the phone's operating system. To do this the research team will develop new pixel-level analyses of app interfaces to assess whether they correctly implement application programming interfaces (APIs) that the phone platform provides to support accessibility, then release datasets that give a picture of accessibility across the ecosystem of apps. The team will also develop new approaches that allow third parties to build alternate "proxy" interfaces that run on top of an existing app and implement accessibility APIs, as well as new approaches for those proxies to be personalized to a particular person's abilities and needs. The team will work closely with partners who serve people with disabilities in both design and evaluation of the tools, bringing direct benefits to those partners, as well as broader social benefits through actively disseminating the tools, methods, and datasets to support others in doing accessibility research. The project will also provide research and education opportunities for students from grade to grad school, with the team specifically targeting students from groups that are underrepresented in computing research, including those with disabilities.For assessment, the team will develop methods for executing an application in a mobile phone emulator, navigating its interface using common tasks, capturing both images of the interface and its accessibility API representation, and evaluating whether a recovered pixel-based interface hierarchy from the captured image correctly corresponds to the API representation. Working with people with disabilities to develop effective summaries and presentations of the analyses, the team will develop scalable infrastructure and efficient analyses to look for patterns in types of accessibility failures, how these failures are distributed across apps and developers, and how they are affected by app updates. For repair and enhancement, the team will develop interaction proxies that leverage the platform's accessibility services to create overlay windows that cover the underlying app interface and exchange events with it. They will also develop design patterns that link common accessibility failures with repair strategies, providing libraries to quickly implement these patterns in interaction proxies to support both third-party and eventually automatic repair of these common failures. Finally, the team will explore how novel interaction techniques that respond to individual abilities could be broadly deployed via interaction proxies, using a Model-View/Controller paradigm to manage the combinatorial complexity of applying particular enhancements to particular interfaces.

许多移动应用程序（应用程序）没有正确实施帮助残疾人使用计算机的辅助服务，从而有效地拒绝了许多弱势群体对应用程序和相关服务的访问。该项目将开发新方法来评估、修复和增强大量应用程序的可访问性，而无需访问应用程序的源代码或手机操作系统。为此，研究团队将开发新的应用程序界面像素级分析，以评估它们是否正确实现了手机平台提供的支持可访问性的应用程序编程接口（API），然后发布数据集，以描述整个应用程序生态系统的可访问性。该团队还将开发新方法，允许第三方构建在现有应用程序之上运行的替代“代理”界面并实现可访问性 API，以及使这些代理根据特定人的能力和需求进行个性化的新方法。该团队将与为残疾人服务的合作伙伴密切合作，设计和评估工具，为这些合作伙伴带来直接利益，并通过积极传播工具、方法和数据集来支持其他人进行无障碍研究，从而带来更广泛的社会效益。该项目还将为从小学生到研究生院的学生提供研究和教育机会，该团队专门针对计算研究中代表性不足的群体的学生，包括残疾人。为了进行评估，该团队将开发在手机模拟器中执行应用程序的方法，使用常见任务导航其界面，捕获界面图像及其可访问性 API 表示，并评估是否可以从恢复的基于像素的界面层次结构中恢复。 捕获的图像正确对应于 API 表示。该团队将与残疾人合作，制定有效的分析摘要和演示文稿，开发可扩展的基础设施和高效的分析，以寻找可访问性故障类型的模式、这些故障如何在应用程序和开发人员之间分布，以及它们如何受到应用程序更新的影响。为了修复和增强，该团队将开发交互代理，利用平台的辅助服务来创建覆盖底层应用程序界面并与其交换事件的覆盖窗口。他们还将开发将常见可访问性故障与修复策略联系起来的设计模式，提供库在交互代理中快速实现这些模式，以支持第三方并最终自动修复这些常见故障。最后，该团队将探索如何通过交互代理广泛部署响应个人能力的新颖交互技术，使用模型-视图/控制器范例来管理将特定增强功能应用于特定接口的组合复杂性。

项目成果

Jod: Examining Design and Implementation of a Videoconferencing Platform for Mixed Hearing Groups

Jod：检查混合听力群体视频会议平台的设计和实现

• DOI: 10.1145/3597638.3608382
• 发表时间: 2023
• 期刊: Proceedings of the ACM Conference on Computers and Accessibility
• 作者: Mittal, Anant;Gupta, Meghna;Poddar, Roshni;Naik, Tarini;Kuppuraj, Seethalakshmi;Fogarty, James;Kumar, Pratyush;Jain, Mohit
• 通讯作者: Jain, Mohit

GestureCalc: An Eyes-Free Calculator for Touch Screens

• DOI: 10.1145/3308561.3353783
• 发表时间: 2019-10
• 期刊: Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility
• 作者: Bindita Chaudhuri;Leah Perlmutter;Justin Petelka;Philip E. Garrison;J. Fogarty;J. Wobbrock;R. Ladner

Crowdsourcing Similarity Judgments for Agreement Analysis in End-User Elicitation Studies

最终用户启发研究中协议分析的众包相似性判断

• DOI: 10.1145/3242587.3242621
• 发表时间: 2018
• 期刊: Proceedings of the ACM Symposium on User Interface Software and Technology
• 作者: Ali, Abdullah X.;Morris, Meredith Ringel;Wobbrock, Jacob O.
• 通讯作者: Wobbrock, Jacob O.

Quantifying Touch: New Metrics for Characterizing What Happens During a Touch

量化触摸：描述触摸过程中发生的情况的新指标

• DOI: 10.1145/3517428.3544804
• 发表时间: 2022
• 期刊: Proceedings of the ACM Conference on Computers and Accessibility
• 作者: Kong, Junhan;Zhong, Mingyuan;Fogarty, James;Wobbrock, Jacob O.
• 通讯作者: Wobbrock, Jacob O.

Distributed interaction design: designing human-centered interactions in a time of social distancing

分布式交互设计：在社交距离时代设计以人为本的交互

• DOI: 10.1145/3447790
• 发表时间: 2021
• 期刊: Interactions
• 作者: Ali, Abdullah X.;Morris, Meredith Ringel;Wobbrock, Jacob O.
• 通讯作者: Wobbrock, Jacob O.