Cloudfinity - Phase 1 Recognition of cockpit controls manipulation using COTS sensors

点击翻译按钮获取中文摘要

the trainer and the trainee be co-located at the same site during a training session. A distributed simulator training technology for which co-location of the trainee/trainer is not required, flight simulation training hardware/software is not co-located with client training platforms, and the cockpit environment of the training platform is rendered via virtual reality (as opposed to having physical cockpit panels) would have a significant economic impact since a trainer could follow the training sessions of many trainees located at remote locations simultaneously. To achieve this in a distributed environment, the activity of the trainee must be captured and displayed to the trainer. Important types of activity are the pose of the user in the platform and hand gestures performed when virtually manipulating different controls of the platform. In the long term, activity tracking in a representative aircraft simulator virtual cockpit should be possible. A first step for reaching this long term objective consists in monitoring trainee activity when interacting with a selection of typical controls on aircraft panels (or a virtual representation of these panels) and rendering this activity to the trainee/trainer in an immersive environment. The goals of the project are to recognize with good accuracy the hand gestures of the pilot when manipulating various controls, locate and detect which control is manipulated, establish optimal sensor configuration to detect manipulations on panels positioned on the overhead, glareshield and pedestal and use a virtual representation of panels for these manipulations. A challenge is that the sensors and displays must be low-cost COTS devices, be non-intrusive and function in