GEMINI: Gestural multimodal interaction for intelligent networked building environments

现代网络技术为在新兴的e社会中实现各种以人为中心的无所不在的计算机应用提供了新的强大的解决方案，如智能住宅、工作、健康和老年护理、购物、娱乐等智能网络空间。为了增强这些密集网络应用的功能和用户满意度，我们必须开发更易访问和不引人注目的、面向用户和用户友好的人机交互方式，使人类可以随时随地无缝地访问信息，并与这些新的智能空间进行自然和直观的交互。数字标牌是智能建筑中最热门的领域之一。这个地区具有很强的工业和商业吸引力。动态数字标牌行业是一个快速增长的行业，投资来自IBM、3M Digital Signage和思科等重要公司。数字标牌系统通常是中央控制的，内容不断变化，数字标牌系统用于广告、直播电视、新闻、寻路和其他与财产相关的信息。虽然数字标牌在许多方面是一个成熟的行业，但交互式数字标牌仍处于起步阶段。该项目的目标是开发新的计算机视觉和语音识别技术，允许通过手势(身体姿势和手势)和语音人机接口将多模式交互功能添加到联网的数字标牌系统中。应用潜力跨越各种智能网络建筑环境，如智能建筑、美术馆和博物馆、电影综合体、图书馆、购物中心、智能健康和养老设施、智能家居等。

Modern networking technologies provide new powerful solutions for the implementation of a large variety ofhuman-centered ubiquitous computer applications in the emergent e-Society such as intelligent networkedspaces for living (smart-houses), working, health and elderly care, shopping, entertainment, etc.In order to augment the functionality and user-satisfaction of these intensive-networked applications we have todevelop more accessible and unobtrusive, user-oriented and user-friendly, human-computer interactionmodalities allowing humans to seamlessly access information anytime and anywhere and to interact naturallyand intuitively with these new intelligent spaces.One of the hottest areas of interest in intelligent buildings is digital signage. This area has a strong industrialand commercial appeal. The dynamic digital signage industry is a fast-growing industry with investmentscoming from important companies such as IBM, 3M Digital Signage, and Cisco. Using plasma and LCDscreens that are usually centrally controlled and have changing contents, digital signage systems are used foradvertisements, live television, news, way-finding and other property-related information.While digital signage is, in many ways, a mature industry, the interactive digital signage is still in its infancy.The objective of this project is to develop new computer vision and voice recognition techniques that allowmultimodal interactive capabilities to be added to networked digital signage systems through gestural (bodyposture and hand gesture) and voice human-computer interfaces.The application potential spans a variety of intelligent networked building environments such as intelligentbuildings, art galleries and museums, movie complexes, libraries, shopping centers, smart health and elderlycare facilities, smart homes, etc.