Gestural Interaction Paradigms for Smart Spaces

手势有很大的潜力来塑造与普适计算环境的交互，并可能构成未来除触摸之外的主要交互范例。虽然3D手势交互已经被探索了很多年，但关于它们的句法结构和跨不同普适计算环境的适用性的科学知识仍然是支离破碎的，局限于特定的受限领域。这使得普适计算环境的交互设计者很难决定在哪些情况和环境中应该使用哪些特定手势。为了缓解这一问题，该提案旨在了解普适计算环境中手势的共同特征，以详细说明它们跨情景和跨领域“成功”的因素。基于这样的观察，在行为层面上识别手势的健壮技术将很快被广泛使用(通过各种传感器，例如深度摄像头)，我们的目标是开发和评估覆盖手势的所有相关句法和语义方面的模型，这些方面可以被认为与在普适计算环境中成功应用手势相关。为此，我们计划开发一个通用的手势词汇表和一个跨域通用的手势语法。这些将基于两个领域的大规模启发性研究：智能家居和智能零售环境，这将导致大规模手势语料库的进一步分析。将提供一种新的手势符号，以帮助对语料库进行注释，并复制手势以供进一步评估。预期的手势模型将基于四个维度，与手势相关的社会、空间、身体和认知需求相关。在项目过程中，将制定相应的衡量标准，以便对手势进行信息性比较。这些指标还将有助于对语料库进行注释。我们的发现将在大规模的现场和实验室研究中得到经验验证。作为该项目的进一步成果，我们计划提供一个软件套件，允许其他研究人员运行和复制我们的研究。在项目期间，我们将开发并陆续使用该软件套件的各个模块，并提供我们的启发和注释手势语料库。这将包括开发将用于注释语料库的手势符号编辑器、将促进上述四个维度的手势注释的注释编辑器、为手势序列提供语义的语法编辑器、以及支持对给定手势语料库进行在线用户研究的评估工具包。这些工具将被集成到一个工具链中，不仅将在项目中使用，还将向社区公开提供。

Freehand gestures have great potential to shape the interaction with pervasive computing environments and might constitute, beside touch, the main interaction paradigm in the future. While 3D gestural interaction has been explored for many years now, the scientific knowledge on their syntactical structure and the applicability across different pervasive computing environments is still fragmented and bound to specific confined domains. This makes it very difficult for interaction designers of pervasive computing environments to decide which specific gestures should be used in which situations and environments. To alleviate this problem, this proposal aims to understand the common characteristics of gestures in pervasive computing environments to elaborate on factors of their “success” across situations and domains. Based on the observation that robust techniques to recognize gestures on a behavioural level will be soon broadly available (through various sensors, e.g. depth-cameras), we aim to develop and evaluate models that cover all relevant syntactic and semantic aspects of gestures that can be considered relevant for the successful application of gestures in pervasive computing environments. For this purpose we plan to develop a general vocabulary of gestures and a gesture grammar that generalizes over domains. These will be based upon a large-scale elicitation study in two domains: a smart home and an intelligent retail environment that will result in a large gesture corpus for further analysis. A novel gesture notation will be provided to help annotate the corpus and to reproduce freehand gestures for further evaluation purposes. The intended gestural model will be based on four dimensions, related to the social, spatial, physical and cognitive demands associated with freehand gestures. In the course of the project corresponding metrics will be developed to allow for an informative comparison of gestures. The metrics will also help to annotate the corpus. Our findings will be validated empirically in large-scale field as well as lab studies. As a further result of the project, we plan to provide a software suite that allows other researchers to run and replicate our studies. During the project we will develop and successively use individual modules of the software suit and make our elicited and annotated gesture corpus available. This will include the development of a gesture notation editor, which will be used to annotate the corpus, a annotation editor that will facilitate annotation of gesture along the four dimensions described above, a grammar editor to provide semantics to gesture sequences, and an evaluation toolkit to support online user studies with a given gesture corpus. The tools will be integrated into a tool chain, that will be not only used within the project, but made publicly available to the community.