Plausible multisensory modeling of 3D deformable objects for interactive applications

点击翻译按钮获取中文摘要

The introduction on the market of 3D enabled technologies, such as 3D displays, 3D glasses and of various means for interaction with virtual environments (gloves, tactile feedback equipment, etc.) places an increasing pressure on the research community to develop enabling technologies for 3D contents. To fully reach their potential, interactive environments need to go beyond the modeling of software generated objects by introducing plausible representations of 3D objects based on real measurements. The rich interaction behavior exhibited by real-world objects depends not only on how objects look, but also on their properties: how stiff they are, how their surface feels and sounds when touched, or how they deform on contact. The development of interactive models for the complex behavior of objects based on real measurements has not followed the rapid progress of visual and audio realism achieved in the entertainment and computer gaming industry. Rising up to this challenge, the long-term goal of this research program is to develop enhanced sensing and modeling techniques for applications that require higher realism by means of plausible 3D object models based on real multisensory measurements, along with their integration, representation and interpretation.**From the multitude of aspects involved in this long-term objective, this original research program focuses on three major aspects: (1) the acquisition and integration of visual, audio and tactile measurements into realistic, coherent 3D models; (2) the rendering of the complex behavior of such objects; and (3) the integration and evaluation of multisensory object models in the context of specific applications. More specifically, human visual and tactile sensing approaches, tactile and visual sampling strategies using various sensors, vision algorithms and computational intelligence techniques will be adapted and refined for use on real measurements. They will be advantageously combined for the formulation of novel object models based on multisensory visual, tactile and audio data. Original computational intelligence methods will be proposed to allow for an efficient use of various sources of information on the object behavior and for the seamless merge of multisensory models in interactive environments. In an initial stage, a dedicated interface will be implemented for the visualization and evaluation of the developed models. Next, the models will be validated in a Cave Automatic Virtual Environment (CAVE) where small imperfections become immediately perceptible. Models will finally be integrated into two practical applications, one for phobia treatment and the other one for safe model-based object manipulation of deformable objects. **The proposed program will contribute the theoretical and practical knowledge required to build the next generation of interactive applications that will rely on plausible 3D contents acquired from real measurements, instead of hard-coded, approximate computer-generated graphics models. Advancements in this area of research will have a significant impact on interactive applications that require higher accuracy and improved realism. These include, beyond the two applications above, interactive environments for training, interactive virtual prototyping for precision-sensitive industrial CAD-based processes, smart manufacturing as well as dexterous manipulation applications such as robotic assembly, tele-robotics, medical and assistive robotics. This research work will support the training of highly qualified specialists with cutting edge approaches and technologies, who are in demand for the strong multimedia, gaming, and virtual reality industry in Canada.