HCC-Large: Using the Internet without using the Eyes: Models of Online Transactions for Non-Visual Interaction

互联网已成为获取信息和进行多种在线交易的主要媒介，包括购物、支付账单、制定旅行计划、申请大学或就业以及参与公民活动。 互联网上的主要交互模式是通过为视觉导航而设计的图形浏览器。这严重限制了视力受损或失明的人的机会，而这一人口数量庞大，而且还在不断增加。 现有的非视觉互联网访问辅助技术通常迫使有视觉障碍的用户进入一种低效的、顺序的信息访问模式。 为了做得更好，需要两种模型。 首先，我们需要建立计算模型来表示网页和在线交易的结构，并使用非视觉方式有效地呈现它们。 其次，我们需要更好地理解用户在线交易的心理模型是如何建立和利用的;然后我们需要将计算模型与用户的心理模型相结合，以便联合收割机结合它们的优势，并显着提高非视觉交互的效率。 在以前的工作中，PI开发了HearSay非视觉Web浏览器，允许用户执行基本的非视觉Web浏览和搜索，上下文浏览和在线表单填写。 然而，道听途说并没有充分利用互动环境或视觉障碍者独特的感知和处理能力。 在目前的项目中，PI寻求将联合收割机基础计算和心理学研究相结合，旨在产生体现计算建模和用户心理模型协同作用的无障碍技术。 在计算研究方面，PI将：（i）自动跟踪用户浏览行为的交互上下文;（ii）自动为用户在线执行的交易构建模型;（iii）开发用户可以通过非视觉方式高效和有效地与交易模型交互的方法。 在心理学研究方面，将进行用户研究，以检查（i）人们如何建立在线交易的心理模型，以及（ii）他们如何使用特定于模态的线索和自己的短期记忆来利用这些心理模型。 PI将把这些用户研究的结果纳入在线交易处理的计算模型中，使其与用户的心理模型相一致。更广泛的影响：PI研究的最终目标是使视障人士能够借助互联网过上完全独立的生活。 为此，公共信息机构计划开展广泛的宣传活动，包括举办讲习班、与为视力障碍者提供服务的机构合作以及在线宣传HearSay原型和HearSay组件技术。 HearSay原则上还将为任何希望非视觉互联网访问的人提供一种手段（例如，在开车时收听互联网内容）。

The Internet has become the primary medium for accessing information and for conducting many types of online transactions, including shopping, paying bills, making travel plans, applying for college or employment, and participating in civic activities. The primary mode of interaction over the Internet is via graphical browsers designed for visual navigation. This seriously limits the access of people with impaired vision or blindness, a population that is large and growing ever larger. Existing assistive technology for non-visual Internet access typically forces users with visual impairments into an inefficient, sequential mode of information access. To do better, two kinds of models are needed. First, we need to build computational models to represent the structure of web pages and online transactions, and to present them effectively using non-visual modalities. Second, we need to better understand how users' mental models for online transactions are built and utilized; we then need to align the computational models with the users' mental models, so as to combine their strengths and significantly improve the efficiency of non-visual interactions. In previous work, the PI developed the HearSay non-visual web browser, which permits users to perform basic non-visual web browsing and search, contextual browsing, and online form-filling. However, HearSay does not take full advantage of the interaction context or the unique perceptual and processing strengths of people with visual impairments. In the current project, the PI seeks to combine basic computational and psychological research designed to produce accessibility technology embodying the synergy of computational modeling and users' mental models. In terms of computational research, the PI will: (i) automatically track the interaction context of user browsing actions; (ii) automatically build models for transactions that users perform online; and (iii) develop ways in which users can interact with transaction models through non-visual modalities efficiently and effectively. In terms of psychological research, user studies will be conducted to examine (i) how people build mental models for online transactions, and (ii) how they use modality-specific cues and their own short-term memory to utilize these mental models. The PI will incorporate the findings from these user studies into the computational models for online transaction processing, so as to align them with the users' mental models.Broader Impacts: The ultimate goal of the PI's research is to empower people with visual impairments to lead completely independent lives with the help of the Internet. To this end, the PI has planned an extensive dissemination campaign involving workshops, collaborations with institutions that serve people who have visual impairments, and online dissemination of HearSay prototypes and HearSay component technologies. HearSay will also provide a means, in principle, for anyone who wishes to have non-visual Internet access (e.g., listening to Internet content while driving).