Sharing Cognition Between Human and Intelligence Machine

在本研究中，讨论了三个主题来解释“See what I See”系统作为共享人类认知的接口的细节。首先，设计了一种可穿戴式凝视跟踪（GT），用于估计二维坐标空间中的凝视位置。商用GT很容易获得，但它们通常以相同的尺寸制造。采用三维可打印框架和开源架构，制造了一种配置成本低、性能合理的可穿戴式GT。这个GT的输出保持非常稳定在75厘米或更高的精度为2.58°。开发了三维（3D）目标检测，使系统能够在真实环境中工作。该模型使用卷积神经网络（CNN）对RGBD信息形成的四个通道（在不同的通道中分成R、G、B、Depth）进行处理。评价结果表明，RGB和深度的结合提高了目标识别的精度。注视跟踪是“我所见”系统识别人在视觉世界坐标中所注意的三维物体的重要工具。可穿戴式GT需要对场景几何和摄像机进行标定，才能适用于视觉世界坐标。结果表明，该方法的注视估计误差为5度。

In this research, three topics are discussed to explain the detail of “See what I See” system as an interface for sharing human’s cognition. At first, a wearable Gaze Tracking (GT) was made to estimate gaze position in 2D coordinate space. Commercial GT is readily available, but they are usually fabricated at the same size. A three-dimensional (3-D)-printable frame and an open-source architecture was made to fabricate a wearable GT with low-cost configuration and reasonable performance. The output of this GT stays very steady at 75 cm or more with an accuracy of 2.58°. Three dimensions (3D) object detection was developed to make the system work in a real environment. The model uses Convolution Neural Network (CNN) on four channels formed using RGBD information (splitting into R, G, B, Depth in separate channels). The evaluation results show that the combination of RGB and depth improve the accuracy of object recognition. Gaze tracking is the important tool for “see what I see” system to identify the 3D object of the person’s attention in the visual world coordinate. The wearable GT requires calibration of scene geometry and camera to make it applicable in visual world coordinates. The result of this approach reports that the gaze estimation error is 5 degrees.