EAGER: VizWiz - Enabling Blind People to Answer Visual Questions On-the-Go with Remote Automatic and Human-Powered Services

缺乏文字标签、图标和颜色等视觉信息让盲人感到沮丧，并严重削弱了他们的独立性。目前的接入技术使用全自动方法来解决该领域的一些问题，但容易出错、范围有限且成本高昂。有能力这样做的盲人必须携带多种具有不同音频和触觉接口的特殊用途便携式设备，以便通过GPS访问有关其环境的关键数据，如条形码中的产品信息和位置信息。如果这些设备的功能更强，故障更少，那么它们可能会被更频繁地使用。通过让咨询人类助手变得容易来提供后备可能是解决方案的一部分。PI用于手机的Talking VizWiz应用程序就是这样一个原型，它将盲人与远程人类工作人员联系起来，后者回答用户视觉环境的一般问题。VizWiz目前允许盲人用户拍照、提问，并快速收到回复。初步研究结果表明，将人类包括在环路中可以帮助克服视觉问题，这些问题仍然太难通过自动方法来解决，但这些方法的有效性、隐私、速度和成本仍然存在疑问。在这个项目中，PI将通过对盲人进行VizWiz的纵向研究来寻求其中一些问题的答案，以更好地了解该应用程序可能如何适应他们的日常生活。他将努力确定如何利用用户现有的社交网络作为Facebook和Twitter应用程序的答案来源。他将寻求定义具有适当接口的新服务，让用户在自动和人力远程来源之间进行调解以获得答案。同时使用自动化和人工Web服务的移动辅助功能解决方案代表了辅助功能方面的重大进步，但也带来了具有挑战性的用户界面问题。了解上述问题对于接受人力服务作为辅助技术的一部分是必要的。更广泛的影响：这项探索性研究代表了人机交互的一种新范式，在这种范式中，人类既是客户又是提供者。VizWiz具有提高盲人独立性的潜力，而且可能比目前的无障碍解决方案更便宜，更可持续。这个项目将提高我们对盲人有用的工具类型的理解，无论今天使用自动计算机视觉有什么可能，并将帮助我们更好地理解如何在尊重提问者的价值观的同时招募人员回答问题。这项研究将涉及所有盲人；由此产生的界面和功能将由世界各地的盲人在日常生活中进行评估。作为该项目的一部分，创建和改进的接口、应用程序和框架将以开源形式发布，以便其他研究人员可以在PI的结果基础上进行改进。项目成果将广泛适用于自动化解决方案可能偶尔需要人工干预的其他问题。

The lack of access to visual information like text labels, icons, and colors frustrates blind people and severely decreases their independence. Current access technology uses fully-automatic approaches to address some problems in this space but is error-prone, limited in scope, and expensive. Blind people who can afford to do so must carry multiple special-purpose portable devices with different audio and tactile interfaces in order to access critical data about their environment such as product information from bar codes and location information via GPS. These devices would likely be used more often if they had greater functionality and failed less often. Providing a fallback by making it easy to consult a human assistant could be part of the solution. The PI's talking VizWiz application for mobile phones is one such prototype that connects blind people to remote human workers who answer general questions about the users' visual environments. VizWiz currently allows blind users to take a picture, speak a question, and receive answers back quickly. Preliminary findings have demonstrated the potential advantages of including humans in the loop to help overcome visual problems that are still too difficult to be solved by automatic approaches alone, but questions remain about the efficacy, privacy, speed, and cost of these approaches. In this project the PI will seek answers to some of these questions, by conducting a longitudinal study of VizWiz with blind people to better understand how the application might fit into their everyday lives. He will endeavor to determine how users' existing social networks might be employed as a source of answers using applications for Facebook and Twitter. And he will seek to define new services with appropriate interfaces that let users mediate between automatic and human-powered remote sources for answers. A mobile accessibility solution using both automated and human Web services represents a significant advance in accessibility but presents challenging user interface questions. Understanding issues such as those enumerated above is necessary for human-powered services to be accepted as part of assistive technology. Broader Impacts: This exploratory research represents a new paradigm in human-computer interaction in which humans are both clients and providers. VizWiz has the potential to improve the independence of blind people, and may be both less expensive and more sustainable than current accessibility solutions. This project will improve our understanding of the types of tools that would be useful for blind people regardless of what is possible today with automatic computer vision, and will help us better understand how to recruit people to answer questions while respecting the asker's values. The research will involve blind people throughout; the resulting interfaces and functionality will be evaluated by blind people in the world going about their everyday lives. The interfaces, applications, and framework created and improved as part of this project will be released as open source so other researchers may build on the PI's results. Project outcomes will be broadly applicable to other problems where automated solutions may occasionally need human intervention.