CHS: Small: Improving Web Accessibility Through Multi-Resolution Mixed-Initiative Interaction Tools

这项研究将创造强大的新方法，弥合视觉障碍者每天在网络上面临的关键无障碍差距，使他们能够更独立和更容易获得信息。有视觉障碍的用户很难或不可能通过屏幕阅读器访问许多网站。其中许多障碍目前已被证明，如果没有视力正常的同事的帮助，是无法克服的。该项目设想的未来是，随着时间的推移，Web可以变得更具导航性和可访问性，同时在第一次遇到访问差距时，使用最少的人力来弥合它们，然后自动为未来的交互提供桥接支持。这是通过最终用户、网络自动化工具和远程员工之间在多个交互解决方案级别上的混合主动交互来实现的。这使得开发的系统不仅每次都能为最终用户工作(甚至在系统以前从未见过的网页上)，而且还能更有效地利用Web上的常见设计元素，以减少成功学习新任务所需的总人工工作量，显著降低成本并提高可伸缩性。在该项目开发的方法中，当视觉障碍的Web用户遇到无法访问的站点时，他们将能够用自然语言指定他们的高级意图，然后自动宏和远程助手的组合将帮助他们完成任务。不仅网站有很大的不同，而且网站经常改变其底层结构，用户的目标也会改变。Web自动化脚本通常无法推断如何在这样的动态条件下实现最终用户的高级意图。另一方面，人类群体工作人员擅长计算出应该采取哪些行动来实现网站上的给定目标。这项工作将解决使用自动化的一些基本挑战，方法是雇佣人类来填补自动化失败的空白，并了解何时页面工作方式发生了语义上有意义的变化。研究分为两个阶段：第一阶段专注于开发创建宏的方法，第二阶段将专注于如何利用群组助手来使这些宏更健壮。这两个阶段都将推进Web自动化的技术水平。该系统将逐步通过一系列实验室和实地研究进行评估。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research will create powerful new ways to bridge critical accessibility gaps that people with visual impairments face on the Web every day, enabling greater independence and access to information. Many websites are difficult or impossible for users with visual impairments to access through screen readers. Many of these accessibility barriers have been shown currently to be insurmountable without help from a sighted colleague. This project envisions a future in which the Web can be made more navigable and accessible over time while using minimal human effort to bridge access gaps the first time they are encountered, and then automating this bridging support for future interactions. This is done through a mixed-initiative interaction between end users, web automation tools, and remote human workers, at multiple levels of interaction resolution. This allows development of a system that will not only work every time for end users (even on web pages the system has never seen before), but also to leverage the common design elements on the Web more effectively in order to reduce the total amount of human effort needed to successfully learn a new task, significantly reducing cost and improving scalability.In the approach to be developed by this project, when Web users with visual impairments encounter an inaccessible site, they will be able to specify their high-level intent in natural language and then a combination of automated macros and remote helpers will help them achieve their task. Not only do websites differ greatly, but websites often change their underlying structure, and users' goals change as well. Web automation scripts are often unable to infer how to achieve the end user's high-level intent under such dynamic conditions. Human crowd workers, on the other hand, are adept at figuring out which actions to take to achieve a given goal on a website. This work will address some of the fundamental challenges of using automation by employing humans to fill in the gaps where automation fails, and to understand when semantically-meaningful changes have occurred in how a page works. The research is divided into two phases: the first phase focuses on developing methods for creating macros, and the second phase will focus on ways to leverage crowd helpers to make these macros more robust. Both phases will advance the state of the art for Web automation. Progressively the system will be evaluated through a set of laboratory and field studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.