Creation and Evaluation of Novel Media Experiences

该提案有两条线索，研究利用人类感知和认知能力来创造和评估下一代媒体体验的新方法。两种媒体体验将被研究：一种新颖的3D显示和教育背景下的互动视频。***对于第一个线程，我计划推进3D视角校正球形（3DPS）显示器的艺术状态。触摸和手势输入、高分辨率和多人功能将被添加到一个球形的3D显示器中，目标是3D行为的两个方面：交互和协作。使用具有低延迟反馈和快速刷新率的多个投影仪具有高分辨率，使物体在球体中的物理感觉成为可能。有了这种现象，再加上球形显示器上和周围的手势和触摸交互，可以探索高保真、协同3D交互的潜力。利用现有的与体积、圆柱形和立方显示的交互工作，将探索指向、选择、翻译、旋转和雕刻内容的新交互语义。由于该显示器支持多路复用视频、新颖的协同定位和远程协作，将创建和评估三维交互语义。****第二个线程推进交互式视频（IV）的创作，导航和共享在教育环境的状态。本文所追求的主要见解是将时空视频内容的界面设计与人的认知和情感机制相匹配，并将其应用于教育视频情境。目前的视频播放器主要针对相对较长的视频内容的线性观看，导致组织部分观看的视频或突出内容的机制较差。然而，随着在线视频内容变得无处不在和越来越短，消费变得更加非线性，视频观看量至少增加了一个数量级。寻找特定的视频或视频中的突出内容是一个重大挑战，特别是当现有的机制倾向于语义记忆而不是情节记忆时。此外，用户生成的内容受到一个将创作与观看和分享分开的模型的影响，阻碍了非专家（如教育工作者和学习者）创建视频内容。探索其他机制，如情感、时间和空间线索将解决这一挑战。**

This proposal has two threads investigating novel means of leveraging human perceptive and cognitive abilities to create and evaluate next generation media experiences. Two media experiences will be investigated: a novel 3D display and interactive video in an educational context. ***For the first thread, I plan to advance the state of the art of 3D perspective-corrected spherical (3DPS) displays. Touch and gesture input, high-resolution, and multi-person functionality will be added to a spherical, 3D display targeting two aspects of behaviour in 3D: interaction and collaboration. Having high-resolution using multiple projectors with low-latency feedback and fast refresh rate, enables the sensation of an object being physically in the sphere. With this phenomenon, along with gesture and touch interaction on and around the spherical display, the potential for high-fidelity, collaborative 3D interaction can be explored. Leveraging off existing work on interaction with volumetric, cylindrical and cubic displays, new interaction semantics for pointing, selecting, translating, rotating, and sculpting content will be explored. As the display supports multiplexed video, novel co-located and remote collaborative, 3D interaction semantics will be created and evaluated.****The second thread advances the state of interactive video (IV) authoring, navigation and sharing in educational environments. The main insight pursued matches interface design for spatio-temporal video content with peoples' cognitive and affective mechanisms, as and applies it to educational video contexts. Current video players target mostly linear viewing of relatively long video content leading to poor mechanisms for organizing partially viewed video or salient content. However, with online video content becoming ubiquitous and shorter, consumption has become more non-linear and the amount of video viewing has increased by at least an order of magnitude. Finding particular videos, or salient content within videos are significant challenges, especially as existing mechanisms tend to be geared towards semantic memory rather than episodic. Furthermore, user generated content suffers from a model that separates authoring from viewing and sharing, impeding video content creation by non-experts, such as educators and learners. Exploring additional mechanisms, such as affective, temporal and spatial cueing will address this challenge. **