Vision-Based Hand Gesture Analysis in a Multimodal Interface for Controlling Virtual Environments

用于控制虚拟环境的多模态界面中基于视觉的手势分析

• 批准号: 9634618
• 负责人: Thomas Huang
• 金额: $ 35.76万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 1996
• 资助国家: 美国
• 起止时间: 1996-08-01 至 2000-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9634618&HistoricalAwards=false
• 关键词: Vision; Based; Hand; Gesture; Analysis

The most natural means of human communication is multimodal, involving a mixture of speech, hand and body movement, facial expression, and eye motion. A multimodal framework is developed where several interaction modes are explored as a means of manipulating a 3D virtual display. The main focus of this project is exploring the use of free hand gestures for manipulating virtual objects in an effort to advance the goal of developing the fundamental theory behind hand gestures as a way of manipulating graphical objects together with speech, gaze direction, scene content, and graphical feedback. Existing multi-user virtual reality setups are used as experimental testbeds for validating the usefulness of methodologies for building useful multimodal human-computer interfaces for virtual reality. Since the development of natural human-computer interaction techniques are perceived to be a major hurdle in a greater impact of the information revolution, the results of this projects have wide-reaching potential applications.

人类最自然的交流方式是多模态的，包括语音、手部和身体运动、面部表情和眼部运动的混合。 开发了一个多模态框架，其中几种交互模式被探索作为操纵3D虚拟显示的手段。 该项目的主要重点是探索使用自由手势来操纵虚拟对象，以推进手势背后的基本理论的发展目标，作为操纵图形对象以及语音，凝视方向，场景内容和图形反馈的一种方式。 现有的多用户虚拟现实设置被用作实验测试平台，用于验证构建有用的多模态虚拟现实人机界面的方法的有用性。 由于自然人机交互技术的发展被认为是信息革命产生更大影响的主要障碍，因此该项目的结果具有广泛的潜在应用。