AuMoCap: Augmented, Portable, Real-Time, Markerless Motion Capture for Digital Humans

点击翻译按钮获取中文摘要

The Imager Group needs a markerless motion capture (MoCap) system to support fundamental research in computer vision (CV), computer graphics (CG), and human-computer interaction (HCI). For CV, it will serve three purposes: as a capture modality for multi-view reconstruction, a reference system for validating new algorithms, and a source for recording training datasets for deep learning algorithms. For CG, it will facilitate research on more natural and immersive telepresence and digital human avatars. In HCI, it will provide a full-body input sensor system for prototyping new interaction techniques. Moreover, it will enable applications outside Computer Science, such as automating quantitative life science studies, as marker-based solutions do, but without hampering or influencing the subjects' natural behavior.******The equipment required includes the hardware and software to 1) stream eight HD videos to a central processing unit; 2) automated calibration; 3) reconstruction of the user's motion as the 3D position of a virtual skeleton over time, including the details of gaze direction; and 4) augmented reality to simulate real environments in controlled conditions.******It will be part of a new, dedicated augmented reality studio of the Imager group and provide the technical infrastructure for running multi-modal, embodied, and immersive telepresence research. Nevertheless, it is designed to be portable, thereby allowing MoCap in outdoor environments and setups in dedicated labs with immobile equipment, such as heavy force plates. Concrete collaborations have been set up with the co-applicant and Imager members in HCI, and the Psychology and Physiotherapy labs at UBC.******The system has the following unique properties which complement existing marker-based solutions:***1) It is nonintrusive, whether physically or psychologically. This is important for sports, where sweat alone poses a challenge to markers, and for running psychometrics studies for eliminating external influence.***2) Outdoor capture at full sunlight and against an uncontrolled moving background.***3) Capturing subjects with loose apparel, such as skirts, as no markers must be attached.***4) Quick setup time, as no instrumentation is necessary.***5) Controlled illumination and augmentation for training and testing computer vision algorithms.***6) Recording the complete appearance, unlike sparse marker-based solutions.***At the same time, it provides real-time motion trajectories of multiple people at an accuracy of 1cm, which is close to the accuracy of marker-based solutions.******My students will use the system regularly, at all stages of their research: at development, training, and test time. Additionally, 5 Imager students in HCI have semi-regular need of this motion tracking system as part of their user studies, and occasional, yet, large-scale studies in Psychology and Physiotherapy will be supported. All this will help ensure that UBC and Canada remain leaders in the human-centered AI field.