HCC-Small: TextSL: A Virtual World Interface for the Visually Impaired

身临其境的图形、大量的用户生成内容以及更复杂的虚拟世界提供的社交机会正在导致它们的受欢迎程度爆炸性增长，有数百万用户参与其中。然而，不幸的是，视障用户无法访问虚拟世界。在这个项目中，PI的目标是通过为视障用户开发一个名为TextSL的虚拟世界界面来缓解这一缺陷，该界面能够从虚拟世界中提取可用屏幕阅读器阅读的文本表示。用户通过受多用户地下城游戏启发的基于命令的机制与TextSL交互。该系统的一个原型表明，这种方法是可行的。该系统允许以一种基本形式进入流行的虚拟世界第二人生。但原型缺乏虚拟世界提供的两个最重要的功能：创造内容的能力，以及与互动对象互动的能力。PI的初步工作发现的其他问题是，在Second Life中，只有不到一半的物体具有有意义的描述性名称，这使得大多数物体对视障用户来说是看不见的，而且虚拟世界包含如此大量的内容，以至于仅仅将其以某种方式转换为文本描述可能很容易让用户不知所措。为了应对这些挑战，PI将开发一款基于“有目的的游戏”模式的游戏，允许有视力的用户通过标记没有名字的对象来使视障用户能够访问虚拟世界，同时还建立了可以用来提供更可用的反馈形式的对象的分类。他将创建一种新的3D物体识别算法，该算法基于“特征袋”方法，利用第二人生中物体的独特表现方式，可以高效、自动地识别大量没有名字的物体。他还将开发一些技术，这些技术能够在不向用户提供大量信息的情况下提供对用户环境的反馈，能够以文本形式传达用户与交互对象的交互，并支持使用基于命令或触觉的方法创建3D对象。布罗德影响：这项研究将通过让数百万视障人士更充分地参与信息社会，提高他们的生活质量。特别是，虚拟世界提供的社交互动将使视障人士以及其他残疾人士受益，他们的残疾往往会导致身体和社会孤立。此外，随着虚拟世界越来越多地被用作教育平台，让所有人都能接触到这项技术变得加倍关键。虚拟世界的交互模型类似于游戏的交互模型，因此项目成果也可能为如何让视障人士访问游戏提供新的见解。

The immersive graphics, the large amount of user-generated content, and the social interaction opportunities offered by the greater sophistication of virtual worlds are leading to an explosive growth in their popularity, with millions of participating users. Unfortunately, however, virtual worlds are not accessible to users who are visually impaired. In this project the PI's goal is to alleviate this deficiency by developing a virtual world interface for visually impaired users called TextSL, which is able to extract a textual representation from a virtual world that can be read with a screen reader. Users interact with TextSL by means of a command-based mechanism inspired by multi-user dungeon games. A prototype of the system that allows for a basic form of access to the popular virtual world of Second Life has shown that such an approach is feasible. But the prototype lacks two of the most important features offered by virtual worlds: the ability to create content, and the ability to interact with interactive objects. Additional problems identified by the PI's preliminary work are that less than half of the objects in Second Life have a meaningful descriptive name, which makes the majority of objects invisible to visually impaired users, and that virtual worlds contain such vast amounts of content that merely converting it somehow into a textual description may easily overwhelm the user. To address such challenges, the PI will develop a game based on the "games-with-a-purpose" paradigm that allows sighted users to make virtual worlds accessible to visually impaired users by labeling objects without a name, while also establishing a taxonomy of objects that can be used to provide a more usable form of feedback. He will create a new 3D object recognition algorithm based on the "bag-of-features" approach that takes advantage of the unique way objects are represented in Second Life, and which can efficiently and automatically recognize large numbers of objects without a name. And he will also develop techniques that are able to provide feedback on a user's environment without overwhelming the user with large amounts of information, that are able to convey textually a user's interaction with an interactive object, and that support creation of 3D objects using command or haptic based methods.Broader Impacts: This research will increase the quality of life for the millions of people who are visually impaired, by allowing them to participate more fully in the information society. In particular, the social interaction offered by virtual worlds will benefit people who are visually impaired as well as other individuals whose disabilities often lead to both physical and social isolation. Furthermore, as virtual worlds are increasingly used as educational platforms, access for all to this technology is doubly critical. The interaction models of virtual worlds resemble those of games, so project outcomes may also provide new insights into how to make games accessible to the visually impaired.