Artificial Intelligent naturally communicating virtual characters

Virtro是一家独立的虚拟和增强现实（VR，AR）游戏工作室，位于温哥华，制作游戏内容和系统，为观众带来新的互动应用程序。他们的任务是为VR/AR提供一种不同的、更具包容性的沉浸式体验。对于这种更具包容性的创新风格，一个需要是融入非常自然的3D VR角色，这些角色可以推理，情感表达和与用户交谈。对于体验式学习的VR系统来说尤其如此，比如VR/AR外语学习游戏，它模拟了当你沉浸在另一种文化/国家时发生的快速体验式学习，这种文化/国家说一种你必须逐步交流的新语言。像这样更自然和更逼真的系统可以大大提高新形式的沉浸式教育娱乐的有效性。我们目前的SFU研究系统使用逼真的游戏，如3D化身，在参数化的动画AI控制下，具有完整的面部和身体姿势。这些动作的一致性使系统能够在情感认知代理中产生个性感。我们的系统利用这些身体动作和完整的面部表情（以及对口型）来促进与用户的更自然的交互。结合我们的AI NLP语言系统[1]，允许我们的3D AI化身与用户进行推理和对话，这项工作被独特地定位为智能VR游戏中自然交互的解决方案。然而，将我们的研究系统转移到更多的消费者VR/AR/移动的系统是必要的，因为它有利于加拿大人的教育和学习。这个项目与SFU和Virtro是重新设计/更新基于AI的3D VR角色系统，使其尽可能自然，但工作在一个更标准和有限的实时计算机图形VR管道上。这项工作是双重的，1）更新对话字符系统与所有的是自然的输出能力（语音，嘴唇同步，情绪，面部表情，身体姿势，.）2）改进基于主题的AI会话系统，提升虚拟角色的个性感和可信度。

Virtro is an independent Virtual & Augments Reality (VR, AR) gaming studio based in Vancouver, makinggaming content and systems that bring a new level of interactive applications to audiences. Their mandate is toprovide a different, more inclusive style of immersive experience for VR/AR. For this more inclusive,innovate style, one need is to incorporate very natural 3D VR characters that can reason, emote and conversewith users. This is especially true for VR systems for experiential learning, like a VR/AR foreign languagelearning game that simulates the quick experiential learning that happens when you are immersed in anotherculture/country that speaks a new language that you must progressively communicate in. More natural andimmersive systems like this could greatly improve the effectiveness of new forms of immersive edutainment.Our current SFU research system uses a realistic game like 3D avatar with full facial and bodily gesturesunder parameterized animated AI control. The consistency of these movements allows the system togenerate a sense of personality in affective cognitive agents. Our system makes use of these bodilymovements and full facial expression (and lipsync) to contribute to a more natural interaction with the user.Combined with our AI NLP language system [1] that allows our 3d AI avatar to reason and speak with users,the work is uniquely positioned as a solution to natural interactions in smart VR games. However, moving ourresearch systems to more consumer VR/AR/mobile systems is needed for it to benefit Canadians for educationand learning. This project with SFU and Virtro is to redesign/update the AI based 3D VR character system tobe as natural as possible but work on a much more standard and limited real-time computer graphics VRpipeline. This effort is twofold, 1) update the conversational character system with all is natural outputcapabilities (voice, lip sync, emotions, facial expression, body gestures, ...) to VR/AR gaming platforms and 2)improve the topic-based AI-based conversational system that promotes the sense of personality andbelievability to the virtual character.