Erkennung von Mehrfingergesten mit einem Fingerring: Ubiquitäre Interaktion für das Internet of Things

带指环的多指手势识别：物联网无处不在的交互

• 批准号: 320971279
• 负责人: Professor Dr.-Ing. Sahin Albayrak
• 金额: --
• 依托单位: Institut für Wirtschaftsinformatik und Quantitative Methoden
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2016
• 资助国家: 德国
• 起止时间: 2015-12-31 至 2019-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/320971279?language=en
• 关键词: Erkennung; von; Mehrfingergesten; mit; einem

The `Internet of Things' (IoT) refers to a wide variety of connected devices with build-in sensors, which collect and exchange information. Most of these objects are not equipped with dedicated user interface technology. Interaction with the IoT thus requires new interaction paradigms with ubiquitous interaction devices, devices that are seamlessly present throughout the users' daily live and not bound to any specific things.Gestures are naturally used in inter-human communication. They also represent a promising modality for interaction with the IoT as they can communicate spatial relations referring to the physical side of the IoT. Finger rings equipped with sensors are a promising basis for ubiquitous gesture detection. However, most existing sensor rings can only track the motion of one finger or are severely limited in the gestures they can recognize. Additionally, gesture recognition algorithms, if applied continuously, can easily exceed the power capacity of a battery that is small enough to be integrated in a finger ring. These disadvantages limit the applicability and utility of current sensor rings. The objective of this project is to explore and evaluate the capabilities and limits of capacitive proximity sensing for enabling multiple finger gesture recognition with a single finger ring over the whole day. For this purpose we follow a 3-fold research agenda by answering the following questions: (1) Which configuration of capacitive proximity sensing and other sensors is best suited to detect gestures and what are the limits of this technology? (2) How can the produced data streams be mapped to a skeleton hand model to enable finger tracking? (3) How far can the power usage be optimized by a prefiltering algorithm? To answer these questions we develop multiple generations of ring prototypes and optimize them in combination with the finger tracking and prefiltering algorithms based on recorded gestures.

物联网(IoT)指的是各种带有内置传感器的互联设备，这些设备可以收集和交换信息。这些对象中的大多数都没有配备专用的用户界面技术。因此，与物联网的互动需要新的互动范式，使用无处不在的互动设备，这些设备无缝地存在于用户的日常生活中，不受任何特定事物的约束。手势自然用于人与人之间的交流。它们还代表了与物联网交互的一种很有前途的方式，因为它们可以传达涉及物联网物理方面的空间关系。配备传感器的指环是检测无处不在的手势的很有前途的基础。然而，现有的大多数传感器环只能跟踪一个手指的运动，或者严重限制了它们可以识别的手势。此外，手势识别算法，如果持续应用，很容易超过足够小到可以集成到指环中的电池的电量。这些缺点限制了电流传感器环的适用性和实用性。本项目的目标是探索和评估电容式接近感测的能力和局限性，以实现全天使用单个指环进行多个手指手势识别。为此，我们遵循了三重研究议程，通过回答以下问题：(1)哪种电容式接近感应器和其他传感器最适合检测手势，这项技术的局限性是什么？(2)如何将产生的数据流映射到手部骨架模型以实现手指跟踪？(3)通过预滤波算法可以在多大程度上优化功耗？为了回答这些问题，我们开发了多代戒指原型，并结合基于记录手势的手指跟踪和预过滤算法对其进行优化。

项目成果

Ring-based finger tracking using capacitive sensors and long short-term memory

使用电容式传感器和长短期记忆进行基于环的手指跟踪

• DOI: 10.1145/3377325.3377535
• 发表时间: 2020
• 期刊: Proceedings of the 25th International Conference on Intelligent User Interfaces
• 作者: M. Wilhelm;J.-P. Lechler;D. Krakowczyk;S. Albayrak
• 通讯作者: S. Albayrak

Demonstration of Finger Tracking Using Capacitive Sensing with a Ring

使用指环电容式感应进行手指跟踪的演示

• DOI: 10.1145/3379336.3381475
• 发表时间: 2020
• 期刊: Proceedings of the 25th International Conference on Intelligent User Interfaces Companion
• 作者: M. Wilhelm;J.-P. Lechler;D. Krakowczyk;S. Albayrak
• 通讯作者: S. Albayrak

A Gesture Control System for Drones Used with Special Operations Forces

特种作战部队使用的无人机手势控制系统

• DOI: 10.1145/3371382.3378206
• 发表时间: 2020
• 期刊: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction
• 作者: M. Montebaur;M. Wilhelm;A. Hessler;S. Albayrak
• 通讯作者: S. Albayrak

PeriSense: Ring-Based Multi-Finger Gesture Interaction Utilizing Capacitive Proximity Sensing

PeriSense：利用电容式接近感应的基于环的多指手势交互

• DOI: 10.3390/s20143990
• 发表时间: 2020
• 期刊: Sensors (Basel, Switzerland)
• 作者: M. Wilhelm;D. Krakowczyk;S. Albayrak
• 通讯作者: S. Albayrak