Functional advance and various realization of step-on interface (SOI)

本项目旨在使操作机器人和机电一体化系统的步进接口（SOI）具有更高的功能性（更高的可靠性和精度）和更多样化（多种形式的实现）。在高功能方面，实现了相机图像处理（使用OpenCV识别投影屏幕和光点）和深度数据处理（使用三维深度传感器）。在多样化方面，实现了激光笔开/关手势对屏幕的操作（点击/拖动操作），通过平移变焦相机控制对投影屏幕的实时跟踪，基于深度数据对指尖背景的接触/非接触操作的识别，以及在虚拟键盘和虚拟木琴上的应用。

This project aimed to make the step-on interface (SOI) to operate robotic and mechatronic systems both higher functional (higher reliability and higher accuracy) and more diversified (realization in various forms). For high functionality, the camera image processing (recognition of a projection screen and a light spot using OpenCV) and the depth date processing (by use of a three-dimensional depth sensor) have been achieved. For diversification, the operation on screen by on/off gesture of laser pointer (click/drag operation), the real-time follow-up to a projection screen by a pan-tilt-zoom camera control, the recognition of the contact/non-contact operation to the background of a finger-tip based on depth data, and the application to a virtual keyboard and a virtual xylophone have been realized.