CHS: Medium: Collaborative Research: Augmented Reality Agents with Pervasive Awareness, Appearance, and Abilities

基于语音的助手对人们的命令做出反应，可以被认为是虚拟的伴侣，总是在旁边播放音乐、告诉我们天气、关灯等。虽然它们做出反应和代表我们采取行动的能力正在增强，但与真正的伴侣不同，它们基本上没有意识到我们的存在，几乎没有主动性，看起来像电器而不是互动伙伴，回答问题以及感知和控制我们周围的真实物体的能力有限。该项目将开发增强现实代理(ARA)来体现这些基于语音的助手，使它们更多地感知我们的外表、情感和行为，给出动态的视觉表示，使我们了解它们的状态和行为；并利用物联网(IoT)基础设施和设备的增长来增加它们感知的广度和深度。总而言之，这些进步将导致更有效和更被接受的基于语音的助手，无论是在家里还是在其他地方。这类ARA有许多潜在的应用，包括医疗保健(通过增加临床模拟和培训的现实感，或通过远程通信和虚拟陪伴为护理提供支持)和教育(通过更有吸引力的导师或代表历史上重要的个人)。为了开发具有普遍的情景感知、外观和能力的嵌入式增强现实代理(ARA)，研究人员将开展一项研究计划，旨在将与增强现实(AR)、智能虚拟代理(IVA)和物联网(IoT)相关的概念和技术联系起来。为了最大化预期的知识成果，研究人员将他们的计划分为三类。首先，他们将以一种既不需要也不依赖于自动化行为的特定技术实现的方式，发展对ARA意识、外观和能力的新理解和优先事项。其次，他们将使用现成的和定制的组件来实现普及的ARA功能，以促进与基本ARA行为相关的形成性实验，并开发特定领域的应用程序和实验。第三，研究人员将使用特定于应用的实现来评估与陪伴和医疗培训的两个领域相关的潜在有用性：儿科患者模拟器和基于团队的广域医疗培训。这项以医疗保健为重点的工作将利用UF和UCF的相关课程，以及UCF的NSF REU物联网中心，让核心团队以外的学生参与有意义的研究。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Voice-based assistants that respond to commands of people can be thought of as virtual companions that are always standing by to play music, tell us the weather, turn the lights off, etc. While their powers to respond and act on our behalf are increasing, unlike real companions they are largely unaware of our presence, take little initiative, look like appliances rather than interaction partners, and have limited abilities to both respond to queries and to sense and control real objects around us. This project will develop Augmented Reality Agents (ARAs) to embody these voice-based assistants, making them more are aware of our appearance, emotions, and behaviors, giving dynamic visual representations that make us aware of their state and behaviors; and leveraging the growth in "Internet of Things" (IoT) infrastructure and devices to increase the breadth and depth of their awareness. Together, these advances will lead to more effective and accepted voice-based assistants, both in the home and beyond. Such ARAs have a number of potential applications, including healthcare (by increasing the realism of clinical simulation and training, or providing support for caregiving through remote communication and virtual companionship) and education (by being more engaging tutors or representing historically important individuals). To develop embodied Augmented Reality Agents (ARAs) with pervasive contextual awareness, appearance, and abilities the researchers will undertake a program of research aimed at the nexus of concepts and technologies associated with Augmented Reality (AR), Intelligent Virtual Agents (IVA), and the Internet of Things (IoT). To maximize the expected knowledge outcomes the researchers have organized their plans into three categories. First, they will develop new understanding of and priorities for ARA awareness, appearance, and abilities in a manner that does not require, nor depend on, a specific technological realization of automated behaviors. Second, they will use off-the-shelf and custom components to realize pervasive ARA functionality, to facilitate formative experiments related to basic ARA behaviors, and develop domain-specific applications and experiments. Third, the researchers will use application-specific realizations to assess the potential usefulness related to companionship and two areas of healthcare training: pediatric patient simulators and wide-area team-based medical training. The healthcare-focused work will leverage relevant courses at UF and UCF, and UCF's NSF REU center on the Internet of Things to engage students beyond their core team, in meaningful research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.