Mixed reality in pediatric rehabilitation

点击翻译按钮获取中文摘要

Mixed reality (MR) video games, wherein real-life objects are used to interact with virtual environments, can provide children with disabilities with engaging, home-based opportunities to practice fine motor skills needed for everyday function. The ability to accurately track objects is key to the design and widespread use of MR video games. Cameras are often used to track objects in MR as they are low-cost, compact and widely available. However, camera-based approaches to object tracking have limitations objects often need to be tagged with special markers/identifiers, tracking performance is sensitive to the environment (e.g. lighting) and prone to errors for fast-moving or occluded objects. This research program will address current challenges in object tracking for MR applications by combining information from many different sources and sensors. For instance, tracking movements of the player can be indicative of interactions with, and movements of, the targeted object. Additionally, the tasks presented by the MR game can be predictive of when/where to expect changes in object position. Likewise, relying on diverse sensors (e.g. colour, depth, inertial) can provide greater informational content regarding an object's movements and the ability to better adapt to changes in operating conditions. Novel contributions of this research will include methods to: (1) determine when to rely on different information sources/sensors, (2) adapt the system dynamically to changes in operating conditions (e.g. proximity to sensor, occlusions) that may impact the reliability of different information sources, (3) detect movements/gestures associated with object interactions for children with a range of abilities, and (4) generalize the approaches developed for diverse MR applications. In the course of this research, three novel MR therapy games will be designed with high potential for commercialization and translation. Eight graduate and four undergraduate students will be provided with high quality, interdisciplinary training in computer vision, digital media, and user-centred design with opportunities to interact with industry leaders in digital media and with healthcare providers in clinical contexts. Trainees will be well-positioned to pursue careers in academia, industry, and entrepreneurship in the fields of digital media and health technology, two of Canada's most high demand sectors.