EAGER: VizWiz - Enabling Blind People to Answer Visual Questions On-the-Go with Remote Automatic and Human-Powered Services

缺乏视觉信息，如文本标签，图标和颜色，使盲人感到沮丧，并严重降低了他们的独立性。 当前的接入技术使用全自动的方法来解决该领域中的一些问题，但是容易出错，范围有限并且昂贵。 有能力这样做的盲人必须携带多个具有不同音频和触觉接口的专用便携式设备，以便访问有关其环境的关键数据，例如条形码的产品信息和通过GPS的位置信息。 如果这些设备具有更强大的功能并且故障频率更低，则它们可能会更频繁地使用。 通过让咨询人类助手变得容易来提供后备可能是解决方案的一部分。 PI为移动的手机开发的Talking VizWiz应用程序就是这样一个原型，它将盲人与远程工作人员联系起来，后者回答有关用户视觉环境的一般问题。 VizWiz目前允许盲人用户拍照，说出问题，并迅速获得答案。 初步的研究结果已经证明了将人类纳入循环的潜在优势，以帮助克服视觉问题，这些问题仍然很难单独通过自动方法解决，但这些方法的有效性，隐私，速度和成本仍然存在问题。 在这个项目中，PI将通过对盲人进行VizWiz的纵向研究来寻求其中一些问题的答案，以更好地了解该应用程序如何融入他们的日常生活。 他将奋进确定如何使用Facebook和Twitter的应用程序将用户现有的社交网络用作答案的来源。 他将寻求定义具有适当接口的新服务，让用户在自动和人工远程源之间进行调解以获得答案。 使用自动化和人工Web服务的移动的可访问性解决方案代表了可访问性方面的重大进步，但也提出了具有挑战性的用户界面问题。 理解诸如上述列举的问题对于人力服务被接受为辅助技术的一部分是必要的。 更广泛的影响：这项探索性研究代表了人机交互的新范式，其中人类既是客户端，也是提供者。 VizWiz有可能提高盲人的独立性，并且可能比目前的无障碍解决方案更便宜，更可持续。 这个项目将提高我们对盲人有用的工具类型的理解，无论今天自动计算机视觉的可能性如何，并将帮助我们更好地了解如何招募人来回答问题，同时尊重提问者的价值观。 这项研究将涉及盲人;由此产生的界面和功能将由世界上的盲人进行日常生活评估。 作为该项目的一部分，创建和改进的接口，应用程序和框架将作为开源发布，以便其他研究人员可以在PI的结果上进行构建。 项目成果将广泛适用于自动化解决方案偶尔需要人工干预的其他问题。

The lack of access to visual information like text labels, icons, and colors frustrates blind people and severely decreases their independence. Current access technology uses fully-automatic approaches to address some problems in this space but is error-prone, limited in scope, and expensive. Blind people who can afford to do so must carry multiple special-purpose portable devices with different audio and tactile interfaces in order to access critical data about their environment such as product information from bar codes and location information via GPS. These devices would likely be used more often if they had greater functionality and failed less often. Providing a fallback by making it easy to consult a human assistant could be part of the solution. The PI's talking VizWiz application for mobile phones is one such prototype that connects blind people to remote human workers who answer general questions about the users' visual environments. VizWiz currently allows blind users to take a picture, speak a question, and receive answers back quickly. Preliminary findings have demonstrated the potential advantages of including humans in the loop to help overcome visual problems that are still too difficult to be solved by automatic approaches alone, but questions remain about the efficacy, privacy, speed, and cost of these approaches. In this project the PI will seek answers to some of these questions, by conducting a longitudinal study of VizWiz with blind people to better understand how the application might fit into their everyday lives. He will endeavor to determine how users' existing social networks might be employed as a source of answers using applications for Facebook and Twitter. And he will seek to define new services with appropriate interfaces that let users mediate between automatic and human-powered remote sources for answers. A mobile accessibility solution using both automated and human Web services represents a significant advance in accessibility but presents challenging user interface questions. Understanding issues such as those enumerated above is necessary for human-powered services to be accepted as part of assistive technology. Broader Impacts: This exploratory research represents a new paradigm in human-computer interaction in which humans are both clients and providers. VizWiz has the potential to improve the independence of blind people, and may be both less expensive and more sustainable than current accessibility solutions. This project will improve our understanding of the types of tools that would be useful for blind people regardless of what is possible today with automatic computer vision, and will help us better understand how to recruit people to answer questions while respecting the asker's values. The research will involve blind people throughout; the resulting interfaces and functionality will be evaluated by blind people in the world going about their everyday lives. The interfaces, applications, and framework created and improved as part of this project will be released as open source so other researchers may build on the PI's results. Project outcomes will be broadly applicable to other problems where automated solutions may occasionally need human intervention.