III: Small: Creating Natural Data Visualization and Analysis Environments

在当前数据丰富的时代，数据可视化和探索能力越来越广泛地应用于各种学科，包括商业、卫生、教育和公共政策等。目前，人们在台式机和笔记本电脑上使用可视化系统，通常通过键盘和鼠标进行交互。这种互动虽然有用，但与《少数派报告》（Minority Report）和《钢铁侠》（Iron Man）等未来主义电影中呈现的自然、流畅的互动相比，就显得苍白无力了。在这些电影中，角色通过语言、凝视和手势与巨大的投影墙壁显示器进行互动。为了实现这种未来的界面，我们必须开发新形式的自然用户界面（NUIs），采用多模态交互，如语音、笔、触摸、手势、凝视、头部和身体动作。虽然没有特定的交互模式可以提供所有所需的功能，但模式的组合（例如，语音，凝视和笔）可以理想地提供更自然，直观和集成的界面体验。该项目将探索、设计、开发和评估用于数据可视化和可视化分析的NUIs。开发技术和系统，为数据的多种表示提供自然的、富有表现力的、多模态的输入和交互，有可能广泛影响几乎无限数量的学科和社会领域。此外，该项目将为研究生和本科生提供教育经验和研究培训，以帮助他们的职业发展。借鉴之前对自然语言和多点触控界面可视化的研究，该研究旨在实现下一代强大、富有表现力和自然的系统，促进与数据可视化表示的流畅交互。虽然这些相互作用模式蕴藏着巨大的潜力，但仍存在许多研究挑战，必须加以解决。例如，系统应该如何处理格式错误、不完整或不明确的语音输入？如果一个意图被误解了怎么办？关于多点触控手势输入，在这些界面中使用的“最佳”触控手势是什么？这些手势能很好地映射到不同类型的可视化和不同的显示类型，大小和类型吗？我们如何让这些手势更容易发现和学习？结合多种输入模式可以让系统平衡一种模式的弱点和另一种模式的优势（例如，通过语音选择的模糊性可以通过触摸的精确性来平衡），促进自然的用户体验。项目目标包括设计、实现和评估数据可视化系统的多模态接口。特别是，该研究将调查不同的交互方法如何影响和实现数据分析和探索。该项目将为特定的数据领域创建应用程序，并为其他研究人员在他们自己的工作中使用开源工具包。用户研究将确定不同类型的交互（语音、触摸、手势等）是否以及如何使分析更快、更容易学习和更强大。项目网站（http://www.cc.gatech.edu/gvu/ii/naturalvis）将包括项目信息、相关出版物的链接、视频和制作的开源软件。

In the current data-rich era, data visualization and exploration capabilities are becoming more widely used in a variety of disciplines including business, health, education, and public policy, among others. Currently, people use visualization systems on desktop and laptop computers and typically interact via keyboard and mouse. Such interactions, while useful, pale in comparison to the natural, fluid interactions presented in futuristic feature films such as "Minority Report" and "Iron Man" where characters interact with large, projected wall displays through speech, gaze, and gesture. To move towards such futuristic interfaces, we must develop new forms of Natural User Interfaces (NUIs) employing multimodal interactions such as speech, pen, touch, gestures, gaze, and head and body movements. While no specific interaction modality may provide all desired capabilities, combinations of modalities (e.g., speech, gaze, and pen) could ideally provide a more natural, intuitive, and integrated interface experience. This project will explore, design, develop, and evaluate NUIs for data visualization and visual analytics. Developing techniques and systems that provide natural, expressive, multimodal input and interaction for multiple representations of data has the potential to broadly impact a virtually unlimited number of disciplines and areas of society. Additionally, the project will provide educational experience and research training for graduate and undergraduate students to help assist them in their careers. Drawing upon prior research on natural language and multi-touch interfaces for visualization, the research seeks to enable a next generation of powerful, expressive, and natural systems that facilitate fluid interaction with visual representations of data. While these interaction modalities harbor great potential, many research challenges exist and must be addressed. For instance, how should a system handle speech input that is ill-formed, incomplete, or ambiguous? What if an intention is misinterpreted or misunderstood? Regarding multi-touch gesture input, what are the "best" touch gestures to use in these interfaces? Do those gestures map well to different types of visualizations and to different display types and sizes and types? How do we make these gestures easier to discover and learn? Combining multiple input modalities may allow systems to counterbalance the weaknesses of one modality with the strengths of another (e.g., ambiguity in selection via speech can be balanced by the preciseness of touch), facilitating a naturalistic user experience. Project objectives include the design, implementation, and evaluation of multimodal interfaces to data visualization systems. In particular, the research will investigate how different interaction methods affect and enable data analysis and exploration. The project will create applications for particular data domains as well as open-source toolkits for other researchers to use in their own work. User studies will identify if and how different types of interaction (speech, touch, gesture, etc.) make analysis faster, easier to learn, and more powerful. The project website (http://www.cc.gatech.edu/gvu/ii/naturalvis) will include project information, links to resulting publications, videos, and open-source software that is produced.