Touch detection in augmented Omni-surface for human-robot teaming
Touch detection in augmented Omni-surface for human-robot teaming
复制标题
DOI：
10.62704/10057/24828
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Fujian Yan;Edgar Chavez;Yimesker Yihun;Hongsheng He
中科院分区：
文献类型：
--
作者：
Fujian Yan;Edgar Chavez;Yimesker Yihun;Hongsheng He
本文提出了一种将任意表面增强为交互式触摸界面的架构。所提出的架构可以通过使用卷积神经网络（CNN）检测和识别指尖来检测人类操作者触摸指尖的次数。 CNN 模型的输入是 RGB-D 传感器获取的图像。 RGB-D 传感器获取的对齐深度信息生成平面模型，该模型确定指尖是否接触表面。所提出的系统也可以在曲面上工作，而不是只能用于平面的传统平面建模方法。根据检测到的表面上的触摸手指来定义相应的手势。机器人的反馈通过交互式投影仪相应地投射到工作表面上。与传统的编程界面相比，直接触摸对于人类来说更加自然。根据实验，所提出的系统可以减少操作员的大量培训时间。
This paper proposes an architecture that augments arbitrary surfaces into an interactive touching interface. The proposed architecture can detect the number of touching fingertips of human operators by detecting and recognizing the fingertips with a convolutional neural network (CNN). The inputs of the CNN model are images that are acquired by an RGB-D sensor. The aligned depth information acquired by the RGB-D sensor generates the plane model, which determines whether fingertips are touching the surface or not. Instead of the traditional plane modeling method that can only be used for flat surfaces, the proposed system can also work on curved surfaces. Corresponding gestures are defined based on the detected touching fingers on the surface. The feedback of the robots is projected on the working surface accordingly with an interactive projector. Compared to conventional programming interfaces, directly touching is much more natural for human beings. Based on the experiments, the proposed system could reduce the massive training time of operators.