Personalized Perception and Interaction in Mixed Reality Environments

我将开发一种名为魔镜的潜在颠覆性创新。目前的原型包括一个深度传感器(Kinect)、一个显示器和定制的现场可视化软件，使其成为一种低成本的混合现实(MR)技术。在五年计划的过程中，我将通过研究新的机器学习方法来解决Magic Mirror目前的局限性，这些方法将使系统变得个性化，从而满足任何用户的感知和交互要求。我的研究工作将进一步专注于开发独特的方法，包括人机交互、用户界面、感知、多模式传感器分析、生物力学建模以及协作混合现实环境的设计。事实上，混合现实是协作界面的理想选择，因为它解决了计算机支持的协作工作中的两个主要问题--无缝和增强现实。这种模式的核心是增强现实技术，这种技术可以用来开发面对面和远程协作的根本不同的界面。在方案结束时，我将通过魔镜在混合现实环境中提供个性化的感知和互动，从而为不同场景中的不同问题提供可扩展和多功能的解决方案，例如在老年技术领域(远程康复/远程护理)，以及在其他重要领域，例如安全(人员跟踪、视频监控，特别是在辅助生活中)、军事复原力和人员的健康保护，或军事远程呈现。

I will develop a potential Disruptive Innovation known as the Magic Mirror. The current prototype comprises a depth sensor (Kinect), a display, and custom in-situ visualization software making it a low-cost mixed-reality (MR) technology. Over the course of the 5-year Program, I will address the current limitations of the Magic Mirror by investigating novel machine learning methods that will enable the system to become personalized so that it will fit the perception and interaction requirements of any user. My research work will further focus on developing unique methods that encompass human-computer interaction, user interfaces, perception, multimodal sensor analysis, biomechanical modeling, and the design of a collaborative mixed-reality environment. Indeed, mixed reality is ideal for collaborative interfaces because it addresses two major issues in computer supported collaborative work - seamlessness and enhancing reality. Central to this paradigm are augmented reality techniques which can be used to develop fundamentally different interfaces for face-to-face and remote collaboration. At the conclusion of the Program, I will have delivered Personalized Perception and Interaction in Mixed Reality Environments, via the Magic Mirror, thereby providing extensible and versatile solutions to different problems in distinct scenarios, e.g., in the field of gerontechnology (telerehabilitation/telecare), as well as in other important areas, such as security (people tracking, video-surveillance particularly in assisted living), military resilience and health protection of personnel, or military telepresence.