Creation and Evaluation of Novel Media Experiences

该提案有两条线索，研究利用人类感知和认知能力来创建和评估下一代媒体体验的新方法。两个媒体的经验将被调查：一个新的3D显示和互动视频在教育方面。 对于第一个线程，我计划推进3D透视校正球形（3DPS）显示器的最新技术。触摸和手势输入、高分辨率和多人功能将被添加到球形3D显示器中，目标是3D行为的两个方面：交互和协作。使用具有低延迟反馈和快速刷新率的多台投影仪获得高分辨率，使物体能够感觉到物理上处于球体中。利用这种现象，沿着在球形显示器上和周围的手势和触摸交互，可以探索高保真协作3D交互的潜力。利用现有的工作互动与体积，圆柱形和立方体显示器，新的互动语义指向，选择，翻译，旋转和雕刻内容将被探索。由于显示器支持多路复用视频，新颖的协同定位和远程协作，3D交互语义将被创建和评估。 第二个线程推进教育环境中的交互式视频（IV）创作、导航和共享的状态。主要的见解追求匹配的时空视频内容的界面设计与人们的认知和情感机制，并将其应用到教育视频环境。当前的视频播放器主要以相对长的视频内容的线性观看为目标，导致用于组织部分观看的视频或显著内容的不良机制。然而，随着在线视频内容变得无处不在和更短，消费变得更加非线性，视频观看量至少增加了一个数量级。找到特定的视频或视频中的突出内容是一个重大挑战，特别是因为现有的机制往往面向语义记忆，而不是情节记忆。此外，用户生成的内容受到将创作与观看和共享分离的模型的影响，从而阻碍了诸如教育者和学习者的非专家的视频内容创建。探索其他机制，如情感，时间和空间线索将解决这一挑战。

This proposal has two threads investigating novel means of leveraging human perceptive and cognitive abilities to create and evaluate next generation media experiences. Two media experiences will be investigated: a novel 3D display and interactive video in an educational context. For the first thread, I plan to advance the state of the art of 3D perspective-corrected spherical (3DPS) displays. Touch and gesture input, high-resolution, and multi-person functionality will be added to a spherical, 3D display targeting two aspects of behaviour in 3D: interaction and collaboration. Having high-resolution using multiple projectors with low-latency feedback and fast refresh rate, enables the sensation of an object being physically in the sphere. With this phenomenon, along with gesture and touch interaction on and around the spherical display, the potential for high-fidelity, collaborative 3D interaction can be explored. Leveraging off existing work on interaction with volumetric, cylindrical and cubic displays, new interaction semantics for pointing, selecting, translating, rotating, and sculpting content will be explored. As the display supports multiplexed video, novel co-located and remote collaborative, 3D interaction semantics will be created and evaluated. The second thread advances the state of interactive video (IV) authoring, navigation and sharing in educational environments. The main insight pursued matches interface design for spatio-temporal video content with peoples’ cognitive and affective mechanisms, as and applies it to educational video contexts. Current video players target mostly linear viewing of relatively long video content leading to poor mechanisms for organizing partially viewed video or salient content. However, with online video content becoming ubiquitous and shorter, consumption has become more non-linear and the amount of video viewing has increased by at least an order of magnitude. Finding particular videos, or salient content within videos are significant challenges, especially as existing mechanisms tend to be geared towards semantic memory rather than episodic. Furthermore, user generated content suffers from a model that separates authoring from viewing and sharing, impeding video content creation by non-experts, such as educators and learners. Exploring additional mechanisms, such as affective, temporal and spatial cueing will address this challenge.