Synthesizing Event-Centric Knowledge Graphs of Daily Activities Using Virtual Space
Synthesizing Event-Centric Knowledge Graphs of Daily Activities Using Virtual Space
复制标题
DOI：
10.1109/access.2023.3253807
复制
发表时间：
2023-07
期刊：
影响因子：
3.9
通讯作者：
S. Egami;Takanori Ugai;Mikiko Oono;K. Kitamura;Ken Fukuda
中科院分区：
文献类型：
--
作者：
S. Egami;Takanori Ugai;Mikiko Oono;K. Kitamura;Ken Fukuda
人工智能（AI）预计将体现在软件代理，机器人和网络物理系统中，这些系统可以理解家庭环境中日常生活的各种上下文信息，以支持各种情况下的人类行为和决策。场景图和知识图构建技术是基于知识的嵌入式问答系统的研究热点。然而，在物理空间中收集和管理各种实验条件下日常活动的真实的数据是相当昂贵的，并且开发理解意图和上下文的AI是困难的。未来，可以将条件容易变更的虚拟空间和条件难以变更的物理空间的数据结合起来，分析日常生活活动。然而，研究幼儿园建设的日常活动，利用虚拟空间和他们的应用还没有取得进展。潜力和挑战仍然必须澄清，以促进人类日常生活中的人工智能发展。因此，本研究提出VirtualHome2KG架构，以产生虚拟空间中日常生活活动的合成KG。该框架基于所提出的以事件为中心的模式和虚拟空间仿真结果，增强了日常活动的合成视频数据和与视频内容相对应的上下文语义数据。因此，可以分析上下文感知数据，并且可以开发由于相关数据和语义信息的可用性不足而传统上难以开发的各种应用。我们还通过几个用例展示了所提出的VirtualHome2KG框架的实用性和潜力，包括通过查询，嵌入和聚类分析日常活动，以及基于专家知识的老年人跌倒风险检测。因此，我们能够开发一种支持工具，以1.0的精度，0.6的召回率和0.75的F1分数检测跌倒风险，并将其可视化并解释其原理。使用在这项工作中探索的情况下，我们还澄清和分类的挑战，未来的研究合成KG生成系统应解决的模拟，模式和人类活动。最后，我们讨论了实现高级应用程序以支持我们日常生活的潜在解决方案。
Artificial intelligence (AI) is expected to be embodied in software agents, robots, and cyber-physical systems that can understand the various contextual information of daily life in the home environment to support human behavior and decision making in various situations. Scene graph and knowledge graph (KG) construction technologies have attracted much attention for knowledge-based embodied question answering meeting this expectation. However, collecting and managing real data on daily activities under various experimental conditions in a physical space are quite costly, and developing AI that understands the intentions and contexts is difficult. In the future, data from both virtual spaces, where conditions can be easily modified, and physical spaces, where conditions are difficult to change, are expected to be combined to analyze daily living activities. However, studies on the KG construction of daily activities using virtual space and their application have yet to progress. The potential and challenges must still be clarified to facilitate AI development for human daily life. Thus, this study proposes the VirtualHome2KG framework to generate synthetic KGs of daily life activities in virtual space. This framework augments both the synthetic video data of daily activities and the contextual semantic data corresponding to the video contents based on the proposed event-centric schema and virtual space simulation results. Therefore, context-aware data can be analyzed, and various applications that have conventionally been difficult to develop due to the insufficient availability of relevant data and semantic information can be developed. We also demonstrate herein the utility and potential of the proposed VirtualHome2KG framework through several use cases, including the analysis of daily activities by querying, embedding, and clustering, and fall risk detection among older adults based on expert knowledge. As a result, we are able to develop a support tool that detects the fall risk with 1.0 precision, 0.6 recall, and 0.75 F1-score and visualize it with an explanation of its rationale. Using the cases explored in this work, we also clarify and classify the challenges that future research on synthetic KG generation systems should resolve in terms of simulation, schema, and human activity. Finally, we discuss the potential solutions for implementing advanced applications to support our daily life.