Creation and Evaluation of Novel Media Experiences

这项提议有两条主线，调查利用人类感知和认知能力创造和评估下一代媒体体验的新方法。将调查两种媒体体验：新颖的3D显示和教育背景下的互动视频。*对于第一条线索，我计划推进3D透视校正球面(3DPS)显示器的最新技术。触摸和手势输入、高分辨率和多人功能将添加到球形3D显示中，目标是3D行为的两个方面：交互和协作。使用具有低延迟反馈和快速刷新率的多个投影仪具有高分辨率，使物体能够物理地在球体中的感觉。通过这种现象，以及球形显示器上和周围的手势和触摸交互，可以探索高保真、协作式3D交互的潜力。利用与体积、圆柱体和立方体显示交互的现有工作，将探索用于指向、选择、平移、旋转和雕刻内容的新的交互语义。由于显示器支持多路视频、新颖的协同定位和远程协作，因此将创建和评估3D交互语义。*第二个线程推进了教育环境中交互式视频(IV)的创作、导航和共享的状态。所追求的主要洞察力将时空视频内容的界面设计与人们的认知和情感机制相匹配，并将其应用于教育视频情境。当前的视频播放器主要以相对较长的视频内容的线性观看为目标，导致用于组织部分观看的视频或显著内容的机制较差。然而，随着在线视频内容变得无处不在、变得更短，消费变得更加非线性，视频浏览量至少增长了一个数量级。寻找特定的视频或视频中的显著内容是一个巨大的挑战，特别是在现有的机制倾向于面向语义记忆而不是情节的情况下。此外，用户生成的内容受到创作与观看和共享分离的模式的影响，阻碍了非专家(如教育工作者和学习者)的视频内容创作。探索其他机制，如情感、时间和空间线索，将解决这一挑战。**

This proposal has two threads investigating novel means of leveraging human perceptive and cognitive abilities to create and evaluate next generation media experiences. Two media experiences will be investigated: a novel 3D display and interactive video in an educational context. ***For the first thread, I plan to advance the state of the art of 3D perspective-corrected spherical (3DPS) displays. Touch and gesture input, high-resolution, and multi-person functionality will be added to a spherical, 3D display targeting two aspects of behaviour in 3D: interaction and collaboration. Having high-resolution using multiple projectors with low-latency feedback and fast refresh rate, enables the sensation of an object being physically in the sphere. With this phenomenon, along with gesture and touch interaction on and around the spherical display, the potential for high-fidelity, collaborative 3D interaction can be explored. Leveraging off existing work on interaction with volumetric, cylindrical and cubic displays, new interaction semantics for pointing, selecting, translating, rotating, and sculpting content will be explored. As the display supports multiplexed video, novel co-located and remote collaborative, 3D interaction semantics will be created and evaluated.****The second thread advances the state of interactive video (IV) authoring, navigation and sharing in educational environments. The main insight pursued matches interface design for spatio-temporal video content with peoples' cognitive and affective mechanisms, as and applies it to educational video contexts. Current video players target mostly linear viewing of relatively long video content leading to poor mechanisms for organizing partially viewed video or salient content. However, with online video content becoming ubiquitous and shorter, consumption has become more non-linear and the amount of video viewing has increased by at least an order of magnitude. Finding particular videos, or salient content within videos are significant challenges, especially as existing mechanisms tend to be geared towards semantic memory rather than episodic. Furthermore, user generated content suffers from a model that separates authoring from viewing and sharing, impeding video content creation by non-experts, such as educators and learners. Exploring additional mechanisms, such as affective, temporal and spatial cueing will address this challenge. **