3D story cube: an interactive tangible user interface for storytelling with 3D graphics and audio
3D story cube: an interactive tangible user interface for storytelling with 3D graphics and audio
复制标题
3D 故事立方体：交互式有形用户界面，用于通过 3D 图形和音频讲述故事
DOI：
--
复制
发表时间：
2004
影响因子：
--
通讯作者：
J. Pan
中科院分区：
文献类型：
--
作者：
Zhiying Zhou;Adrian David Cheok;J. Pan
无论是在娱乐还是在教育方面，讲故事都是人类文化的重要组成部分。传统上，讲故事可以带来多感官体验，包括音频（说话者的叙述）、视觉（书中的文字和图片）和触觉（翻页）。随着多媒体技术的发展，在个人计算机上讲述故事的数字电影成为可能。然而，从人机交互的角度来看，基于屏幕的讲故事界面将用户限制在平面矩形屏幕上。这不仅将故事与人类和物理世界分开，而且还限制了触摸的使用（从而限制了交互模式）。 “魔法书”[1] 技术鼓励与增强现实世界进行 3D 交互，但由于其平面 2D 配置，所使用的卡片和桨并没有太大改善触摸的使用。如图 1 所示，用户仅限于平面 2D 触摸，这与人类交互的自然现实世界 3D 对象不同。我们通过应用有形且直观的物理对象——可折叠 3D 立方体来解决这个问题。该系统的物理设置如图 2 所示。这使得人类能够感受到多种多感官叙事模式（音频、视觉和 3D 物理触摸）。我们的初步用户研究表明，与被动地获得视频剪辑和魔法书相比，大多数用户在能够实际操作和探索立方体以观看和听到故事时感到非常有趣。我们的故事立方体应用简单的状态转换模型来进行交互式讲故事。当用户将立方体展开到特定状态时，会按照预定义的顺序播放适当的音频和 3D 动画片段。仅当当前状态的内容已播放时才会调用状态转换。状态转换是单向的，因此只会回放第一个未使用的故事片段，使故事保持连续的叙事进程。我们的故事魔方讲述了一个著名的圣经故事——诺亚方舟，它被选为早期圣经的代表性故事之一，如图3所示。总而言之，这项工作探索了魔方界面在互动讲故事领域的应用。通过引入 3D 图形和声音以及 3D 操作和 3D 触觉，叙述似乎更加有吸引力和易于理解。我们的应用表明，增强现实技术可用于补充有形用户界面，以获得更令人兴奋和娱乐的体验。
Storytelling is an important part of human culture, both in entertainment and in education. Traditionally, storytelling enables multisensory experiences, including audio (speaker’s narration), vision (text and pictures in the book) and touch (turning the pages). With the development of multimedia technologies, digital movies for storytelling on the personal computer become possible. However, from the HCI viewpoint, the screenbased storytelling interfaces constrain users to flat rectangular screens. This not only separates the story from the humans and the physical world, but it also limits the use of touch (and, thus, limits the interaction modes). ‘‘Magic book’’ [1] technology encourages a 3D interaction with the augmented reality world, but the cards and paddles used do not improve the use of touch very much, due to their flat 2D configuration. As shown in Fig. 1, the users are limited to planar 2D touch, which is not like the natural real world 3D objects that humans interact with. We address this problem by applying a tangible and intuitive physical object—a foldable 3D cube. The physical setup of the system is shown in Fig. 2. This allows multiple modes of multisensory storytelling to be felt by the humans (audio, vision and 3D physical touch). Our preliminary user study shows that most of the users feel very entertained when they can physically manipulate and explore the cubes to see and hear the story, compared to being passively given a video clip and a magic book. Our story cube applies a simple state transition model for interactive storytelling. Appropriate segments of audio and 3D animation are played in a pre-defined sequence when the user unfolds the cube into a specific state. The state transition is invoked only when the contents of current state have been played. The state transition is of a one-way manner, thus, only the first unused story segment will be played back, allowing the story to maintain a continuous narrative progression. Our story cube tells a famous Bible story, Noah’s Ark, which is selected to be one of representative stories of the early Bible, as shown in Fig. 3. To summarize, this work explores the application of a cube interface to the field of interactive storytelling. The narrative appears to be much more attractive and understandable by introducing 3D graphics and sound, as well as the 3D manipulation and 3D sense of touch. Our application shows that augmented reality technologies can be used to complement tangible user interfaces for a more exciting and entertaining experience.