Cloudfinity - Phase 1 Recognition of cockpit controls manipulation using COTS sensors

在培训期间，培训者和受训者应位于同一地点。 一种分布式模拟器训练技术，不需要受训者/教练员共存，飞行 仿真培训硬件/软件与客户端培训平台不在同一位置，驾驶舱 训练平台的环境通过虚拟现实(而不是拥有物理驾驶舱)进行渲染 专家小组)将产生重大的经济影响，因为培训师可以跟随许多人的培训课程 受训人员同时位于偏远地区。要在分布式环境中实现这一点， 受训者必须被抓获并展示给教练员。重要的活动类型是用户在 虚拟操作平台的不同控件时执行的平台和手势。在 从长远来看，在具有代表性的飞机模拟器虚拟驾驶舱中跟踪活动应该是可能的。第一步 要实现这一长期目标，就在于监控学员在与选定的 飞机面板上的典型控件(或这些面板的虚拟表示)，并将此活动呈现给 身临其境的环境中的实习生/培训师。 该项目的目标是准确地识别飞行员在操纵时的手势 各种控制，定位和检测哪个控制被操纵，建立最佳传感器配置以 检测放置在头顶、玻璃罩和底座上的面板上的操作，并使用虚拟 这些操作的面板表示法。 一个挑战是，传感器和显示器必须是低成本的COTS设备，必须是非侵入性的，并且在

the trainer and the trainee be co-located at the same site during a training session. A distributed simulator training technology for which co-location of the trainee/trainer is not required, flight simulation training hardware/software is not co-located with client training platforms, and the cockpit environment of the training platform is rendered via virtual reality (as opposed to having physical cockpit panels) would have a significant economic impact since a trainer could follow the training sessions of many trainees located at remote locations simultaneously. To achieve this in a distributed environment, the activity of the trainee must be captured and displayed to the trainer. Important types of activity are the pose of the user in the platform and hand gestures performed when virtually manipulating different controls of the platform. In the long term, activity tracking in a representative aircraft simulator virtual cockpit should be possible. A first step for reaching this long term objective consists in monitoring trainee activity when interacting with a selection of typical controls on aircraft panels (or a virtual representation of these panels) and rendering this activity to the trainee/trainer in an immersive environment. The goals of the project are to recognize with good accuracy the hand gestures of the pilot when manipulating various controls, locate and detect which control is manipulated, establish optimal sensor configuration to detect manipulations on panels positioned on the overhead, glareshield and pedestal and use a virtual representation of panels for these manipulations. A challenge is that the sensors and displays must be low-cost COTS devices, be non-intrusive and function in