Artificial Intelligent naturally communicating virtual characters

点击翻译按钮获取中文摘要

Virtro is an independent Virtual & Augments Reality (VR, AR) gaming studio based in Vancouver, makinggaming content and systems that bring a new level of interactive applications to audiences. Their mandate is toprovide a different, more inclusive style of immersive experience for VR/AR. For this more inclusive,innovate style, one need is to incorporate very natural 3D VR characters that can reason, emote and conversewith users. This is especially true for VR systems for experiential learning, like a VR/AR foreign languagelearning game that simulates the quick experiential learning that happens when you are immersed in anotherculture/country that speaks a new language that you must progressively communicate in. More natural andimmersive systems like this could greatly improve the effectiveness of new forms of immersive edutainment.Our current SFU research system uses a realistic game like 3D avatar with full facial and bodily gesturesunder parameterized animated AI control. The consistency of these movements allows the system togenerate a sense of personality in affective cognitive agents. Our system makes use of these bodilymovements and full facial expression (and lipsync) to contribute to a more natural interaction with the user.Combined with our AI NLP language system [1] that allows our 3d AI avatar to reason and speak with users,the work is uniquely positioned as a solution to natural interactions in smart VR games. However, moving ourresearch systems to more consumer VR/AR/mobile systems is needed for it to benefit Canadians for educationand learning. This project with SFU and Virtro is to redesign/update the AI based 3D VR character system tobe as natural as possible but work on a much more standard and limited real-time computer graphics VRpipeline. This effort is twofold, 1) update the conversational character system with all is natural outputcapabilities (voice, lip sync, emotions, facial expression, body gestures, ...) to VR/AR gaming platforms and 2)improve the topic-based AI-based conversational system that promotes the sense of personality andbelievability to the virtual character.