Sharing Cognition Between Human and Intelligence Machine

在本研究中，三个主题进行了讨论，以解释“看到我所看到的”系统作为共享人类认知的接口的细节。首先，可穿戴凝视跟踪（GT）估计视线位置在二维坐标空间。商业燃气轮机是现成的，但他们通常是在相同的尺寸制造。为了实现低成本、性能合理的可穿戴GT，设计了一种可三维打印的框架和开源架构。这款GT的输出在75 cm或更高时保持非常稳定，精度为2.58°。三维（3D）目标检测的发展，使系统工作在一个真实的环境。该模型在使用RGBD信息形成的四个通道上使用卷积神经网络（CNN）（在单独的通道中分成R、G、B、深度）。评价结果表明，RGB和深度的结合提高了目标识别的准确性。视线跟踪是“看我所见”系统在视觉世界坐标系中识别人的注意力所关注的三维物体的重要工具。可穿戴GT需要场景几何和相机的校准，使其适用于视觉世界坐标。该方法的结果报告注视估计误差为5度。

In this research, three topics are discussed to explain the detail of “See what I See” system as an interface for sharing human’s cognition. At first, a wearable Gaze Tracking (GT) was made to estimate gaze position in 2D coordinate space. Commercial GT is readily available, but they are usually fabricated at the same size. A three-dimensional (3-D)-printable frame and an open-source architecture was made to fabricate a wearable GT with low-cost configuration and reasonable performance. The output of this GT stays very steady at 75 cm or more with an accuracy of 2.58°. Three dimensions (3D) object detection was developed to make the system work in a real environment. The model uses Convolution Neural Network (CNN) on four channels formed using RGBD information (splitting into R, G, B, Depth in separate channels). The evaluation results show that the combination of RGB and depth improve the accuracy of object recognition. Gaze tracking is the important tool for “see what I see” system to identify the 3D object of the person’s attention in the visual world coordinate. The wearable GT requires calibration of scene geometry and camera to make it applicable in visual world coordinates. The result of this approach reports that the gaze estimation error is 5 degrees.