CISE CRI: Community Resources for Inventing the Next Generation of Human Computer Interfaces

该项目旨在加快支持与虚拟人自然对话交互的计算机界面的研究和开发，开发用于设计和试验支持与虚拟人实时对话交互的多媒体程序的虚拟人工具包，以及用于基于语料库的研究的资源，以根据文本和/或语音自动生成计算机角色的动画。虚拟人工具包将包括一套集成的创作工具和研究工具，用于开发将虚拟人纳入智能教学系统、语言治疗系统和其他应用程序的多媒体应用程序。该工具包提供了创作工具和教程，用于呈现媒体(文本、图像和动画)并与其交互，设计和控制逼真的3D计算机角色，以及设计自然的有声对话。研究进展和实时系统应该通过提供高效和有效的计算机接口来影响个人和社会。这些研究工具可以用来通过设计操纵虚拟人的媒体和行为呈现的系统来获得对学习和治疗过程的新见解。拟议的工作应该会降低障碍，并提供新的工具来促进以下领域的知识：智能教学系统、口语对话系统、语音识别、计算机角色动画、机器感知和学习以及人类社会动态。广泛影响：虚拟人的发展可能会影响许多文盲、有困难或残疾的人的生活质量。虚拟人的行为像敏感而有效的阅读教师、语言治疗师、医疗助手和一般助理。基于计算机的学习和辅导系统为受益于可获得和廉价的辅导和治疗系统提供了前所未有的机会。

This project, aiming at accelerating research and development of computer interfaces that support natural conversational interaction with virtual humans, develops a virtual human toolkit for designing and experimenting multimedia programs that support real time conversational interaction with virtual humans, and resources for corpus-based research to generate animation of computer characters automatically from text and/or speech. The virtual human toolkit will consist of an integrated set of authoring tools and research tools for developing multimedia applications that incorporate virtual humans in intelligent tutoring systems, speech therapy systems, and other applications. The toolkit provides authoring tools and tutorials for presenting and interacting with media (text, images, and animations), designing and controlling lifelike 3D computer characters, and designing natural spoken dialogs. Research advances and real time systems should impact individuals and society by providing efficient and effective computer interfaces. The research tools can be used to gain new insights into the learning and therapeutic processes by designing systems that manipulate presentation of media and behaviors of virtual humans. The proposed work should lower the barriers and provide new tools to advance knowledge in some of the following areas: intelligent tutoring systems, spoken dialog systems, speech recognition, computer character animation, machine perception and learning, and human social dynamics.Broader Impact: The development of virtual humans that behave like sensitive and effective reading teachers, speech therapists, medical helpers, and general assistants might impact quality of life for many illiterate, challenged, or disabled individuals. Computer-based learning and tutoring systems provide unprecedented opportunities for benefiting from accessible and inexpensive tutoring and therapy systems.