CRII: CSR: Pervasive Gesture Recognition Using Ambient Light

随着计算设备变得越来越小、越来越智能、越来越普遍，计算已经开始以各种形式嵌入到我们的环境中，比如智能恒温器、智能电器、远程控制的家用设备和基于天气的自动草坪灌溉系统。因此，我们需要新的方式来无缝、有效地与这些无处不在、随时可用的计算设备进行沟通和交互。这种交流和互动的自然选择是人类的手势，因为手势是人类在日常生活中相互交流和互动的一个组成部分。该项目旨在利用环境光和廉价的商用现成光传感器来开发手势识别系统。这种方法背后的直觉是，当用户在一个有光的房间里做一个手势时，他/她反射和阻挡的光量会发生变化，从而导致房间所有部分的光强度发生变化。光强度的变化模式对于不同的手势是不同的，这是可以学习和用来识别手势的。在开发基于环境光的手势识别系统时，本项目有两个主要目标：(1)环境独立性，即使系统与环境特征无关，例如不同的照明条件；(2)用户独立性，即使系统与房间内的用户数量和他们的日常活动无关。在开发基于环境光的手势识别系统的过程中出现了一些挑战，例如自动检测手势的开始和结束，从传感器值的时间序列中去除噪声，处理同一手势不同出现的不同持续时间，同时识别多人的手势，以及识别非静止用户的手势。该项目不仅将解决这些和其他类似的挑战，而且还将通过产生利用光强度变化建模人类手势和日常活动的理论基础，推进对环境光在新系统中使用的知识和理解。这项工程的成功完成将极大地造福于我们的社会。首先，本项目将公开项目期间收集的数据集，供研究使用。其次，本文提出的基于环境光的手势识别系统将为用户提供一种新的、方便的方式来与嵌入在其环境中的计算进行交互。第三，拟议的项目将连接几个不同的社区，如系统、信号处理、机器学习、移动计算和人机交互；促进他们之间的互动和交流。第四，该项目的教育方面将把研究成果整合到北卡罗莱纳州立大学的本科和研究生课程中。

As computing devices are becoming smaller, smarter, and more ubiquitous, computing has started to embed into our environment in various forms such as intelligent thermostats, smart appliances, remotely controllable household equipment, and weather based automated lawn irrigation systems. Consequently, we need new ways to seamlessly and effectively communicate and interact with such ubiquitous and always-available computing devices. A natural choice for such communication and interaction is human gestures because gestures are an integral part of the way humans communicate and interact with each other in their daily lives. This project aims at using ambient light and cheap commercial off-the-shelf light sensors to develop a gesture recognition system. The intuition behind this approach is that as a user performs a gesture in a room that is lit with light, the amount of light that he/she reflects and blocks changes, resulting in changes in the intensity of light in all parts of the room. The patterns of change in the intensity of light are different for different gestures, which can be learnt and used to recognize the gestures.In developing the ambient light based gesture recognition system, this project has two primary objectives: (1) environment independence, i.e., making the system agnostic to the characteristics of the environment, such as different lighting conditions, and (2) user independence, i.e., making the system agnostic to the number of users in a room and their routine activities. Several challenges arise in developing the ambient light based gesture recognition system, such as automatically detecting the start and end of a gesture, removing noise from the time-series of sensor values, handling the varying time durations of the different occurrences of the same gesture, simultaneously recognizing the gestures of multiple people, and recognizing the gestures of non-stationary users. This project will not only address these and other similar challenges, but will also advance the knowledge and understanding of the use of ambient light for novel systems by yielding a theoretical foundation for modeling human gestures and routine activities using changes in the intensity of light.The successful completion of this project will greatly benefit our society. First, this project will make the data set collected during this project publicly available for research. Second, the proposed ambient light based gesture recognition system will introduce a new and convenient way for users to interact with the computing embedded in their environments. Third, the proposed project will bridge several different communities such as systems, signal processing, machine learning, mobile computing, and human computer interaction; and foster interaction and communication among them. Fourth, the educational side of the project will integrate the research findings into the undergraduate and graduate curricula at North Carolina State University.