Personalized Perception and Interaction in Mixed Reality Environments

我将开发一种潜在的颠覆性创新叫做魔镜。目前的原型包括一个深度传感器（Kinect）、一个显示器和定制的现场可视化软件，使其成为一种低成本的混合现实（MR）技术。在这个为期5年的项目中，我将通过研究新的机器学习方法来解决魔镜目前的局限性，这些方法将使系统变得个性化，从而适应任何用户的感知和交互需求。我的研究工作将进一步集中于开发独特的方法，包括人机交互、用户界面、感知、多模态传感器分析、生物力学建模和协作混合现实环境的设计。事实上，混合现实是协作接口的理想选择，因为它解决了计算机支持的协作工作中的两个主要问题——无缝性和增强真实感。这个范例的核心是增强现实技术，它可以用于开发面对面和远程协作的根本不同的接口。在项目结束时，我将通过魔镜在混合现实环境中提供个性化感知和交互，从而为不同场景中的不同问题提供可扩展和通用的解决方案，例如老年技术（远程康复/远程护理）领域，以及其他重要领域，如安全（人员跟踪，视频监控，特别是辅助生活），军事弹性和人员健康保护，或者军用网真。

I will develop a potential Disruptive Innovation known as the Magic Mirror. The current prototype comprises a depth sensor (Kinect), a display, and custom in-situ visualization software making it a low-cost mixed-reality (MR) technology. Over the course of the 5-year Program, I will address the current limitations of the Magic Mirror by investigating novel machine learning methods that will enable the system to become personalized so that it will fit the perception and interaction requirements of any user. My research work will further focus on developing unique methods that encompass human-computer interaction, user interfaces, perception, multimodal sensor analysis, biomechanical modeling, and the design of a collaborative mixed-reality environment. Indeed, mixed reality is ideal for collaborative interfaces because it addresses two major issues in computer supported collaborative work - seamlessness and enhancing reality. Central to this paradigm are augmented reality techniques which can be used to develop fundamentally different interfaces for face-to-face and remote collaboration. At the conclusion of the Program, I will have delivered Personalized Perception and Interaction in Mixed Reality Environments, via the Magic Mirror, thereby providing extensible and versatile solutions to different problems in distinct scenarios, e.g., in the field of gerontechnology (telerehabilitation/telecare), as well as in other important areas, such as security (people tracking, video-surveillance particularly in assisted living), military resilience and health protection of personnel, or military telepresence.