Personalized Perception and Interaction in Mixed Reality Environments

我将开发一种潜在的颠覆性创新，称为魔镜。目前的原型包括深度传感器（Kinect），显示器和定制的原位可视化软件，使其成为一种低成本的混合现实（MR）技术。在为期5年的项目中，我将通过研究新的机器学习方法来解决魔镜目前的局限性，这些方法将使系统变得个性化，从而满足任何用户的感知和交互需求。我的研究工作将进一步侧重于开发独特的方法，包括人机交互，用户界面，感知，多模态传感器分析，生物力学建模和协作混合现实环境的设计。事实上，混合现实是协作界面的理想选择，因为它解决了计算机支持的协作工作中的两个主要问题-无缝和增强现实。这种模式的核心是增强现实技术，可用于开发面对面和远程协作的根本不同的界面。在项目结束时，我将通过魔镜在混合现实环境中提供个性化感知和交互，从而为不同场景中的不同问题提供可扩展和通用的解决方案，例如，在老年人技术领域（远程康复/远程护理），以及在其他重要领域，如安全（人员跟踪、视频监控，特别是在辅助生活方面）、军事复原力和人员健康保护或军事远程呈现。

I will develop a potential Disruptive Innovation known as the Magic Mirror. The current prototype comprises a depth sensor (Kinect), a display, and custom in-situ visualization software making it a low-cost mixed-reality (MR) technology. Over the course of the 5-year Program, I will address the current limitations of the Magic Mirror by investigating novel machine learning methods that will enable the system to become personalized so that it will fit the perception and interaction requirements of any user. My research work will further focus on developing unique methods that encompass human-computer interaction, user interfaces, perception, multimodal sensor analysis, biomechanical modeling, and the design of a collaborative mixed-reality environment. Indeed, mixed reality is ideal for collaborative interfaces because it addresses two major issues in computer supported collaborative work - seamlessness and enhancing reality. Central to this paradigm are augmented reality techniques which can be used to develop fundamentally different interfaces for face-to-face and remote collaboration. At the conclusion of the Program, I will have delivered Personalized Perception and Interaction in Mixed Reality Environments, via the Magic Mirror, thereby providing extensible and versatile solutions to different problems in distinct scenarios, e.g., in the field of gerontechnology (telerehabilitation/telecare), as well as in other important areas, such as security (people tracking, video-surveillance particularly in assisted living), military resilience and health protection of personnel, or military telepresence.