GEMINI: Gestural multimodal interaction for intelligent networked building environments

现代网络技术为在新兴的电子社会中实现各种以人为中心的无处不在的计算机应用提供了新的强大的解决方案，如智能网络生活空间（智能住宅）、工作空间、健康和养老空间、购物空间、娱乐空间等。为了增强这些密集网络应用的功能和用户满意度，我们必须开发更易于访问和不引人注目的、以用户为导向的和用户友好的。人机交互模式允许人类随时随地无缝访问信息，并与这些新的智能空间进行自然和直观的交互。智能建筑中最热门的领域之一是数字标牌。这个地区具有很强的工业和商业吸引力。动态数字标牌行业是一个快速增长的行业，投资来自IBM、3M数字标牌和思科等重要公司。数字标牌系统使用等离子和lcd屏幕，通常是集中控制的，内容多变，用于广告、电视直播、新闻、指路和其他与财产相关的信息。虽然数字标牌在许多方面都是一个成熟的行业，但交互式数字标牌仍处于起步阶段。该项目的目标是开发新的计算机视觉和语音识别技术，通过手势（身体姿势和手势）和语音人机界面将多模式交互功能添加到网络数字标牌系统中。应用潜力跨越智能建筑、美术馆博物馆、影城、图书馆、购物中心、智能健康养老设施、智能家居等多种智能网络化建筑环境。

Modern networking technologies provide new powerful solutions for the implementation of a large variety ofhuman-centered ubiquitous computer applications in the emergent e-Society such as intelligent networkedspaces for living (smart-houses), working, health and elderly care, shopping, entertainment, etc.In order to augment the functionality and user-satisfaction of these intensive-networked applications we have todevelop more accessible and unobtrusive, user-oriented and user-friendly, human-computer interactionmodalities allowing humans to seamlessly access information anytime and anywhere and to interact naturallyand intuitively with these new intelligent spaces.One of the hottest areas of interest in intelligent buildings is digital signage. This area has a strong industrialand commercial appeal. The dynamic digital signage industry is a fast-growing industry with investmentscoming from important companies such as IBM, 3M Digital Signage, and Cisco. Using plasma and LCDscreens that are usually centrally controlled and have changing contents, digital signage systems are used foradvertisements, live television, news, way-finding and other property-related information.While digital signage is, in many ways, a mature industry, the interactive digital signage is still in its infancy.The objective of this project is to develop new computer vision and voice recognition techniques that allowmultimodal interactive capabilities to be added to networked digital signage systems through gestural (bodyposture and hand gesture) and voice human-computer interfaces.The application potential spans a variety of intelligent networked building environments such as intelligentbuildings, art galleries and museums, movie complexes, libraries, shopping centers, smart health and elderlycare facilities, smart homes, etc.