CRII: CSR: Pervasive Gesture Recognition Using Ambient Light

随着计算设备变得更小、更智能和更无处不在，计算已经开始以各种形式嵌入到我们的环境中，例如智能恒温器、智能电器、远程可控家用设备和基于天气的自动草坪灌溉系统。因此，我们需要新的方法来无缝和有效地与这种无处不在和始终可用的计算设备进行通信和交互。这种通信和交互的自然选择是人类手势，因为手势是人类在日常生活中彼此通信和交互的方式的组成部分。该项目旨在利用环境光和廉价的商业现成的光传感器来开发手势识别系统。这种方法背后的直觉是，当用户在用光照亮的房间中执行手势时，他/她反射和阻挡的光量改变，导致房间所有部分中的光强度改变。在开发基于环境光的手势识别系统时，本项目有两个主要目标：（1）环境独立性，即，使系统不知道环境的特性，例如不同的照明条件，以及（2）用户独立性，即，使得系统对于房间中的用户数量和他们的日常活动是不可知的。在开发基于环境光的手势识别系统时出现了几个挑战，例如自动检测手势的开始和结束、从传感器值的时间序列中去除噪声、处理同一手势不同出现的不同持续时间、同时识别手势多个人的手势，以及识别非静止用户的手势。该项目不仅将解决这些和其他类似的挑战，而且还将通过为利用光强度变化建模人类手势和日常活动提供理论基础，促进对环境光用于新型系统的知识和理解。该项目的成功完成将极大地造福于我们的社会。首先，该项目将公开项目期间收集的数据集，供研究使用。其次，所提出的基于环境光的手势识别系统将为用户引入一种新的和方便的方式来与嵌入在其环境中的计算进行交互。第三，拟议的项目将连接多个不同的社区，例如系统、信号处理、机器学习、移动的计算和人机交互;并促进它们之间的交互和沟通。第四，该项目的教育方面将把研究成果纳入北卡罗来纳州州立大学的本科生和研究生课程。

As computing devices are becoming smaller, smarter, and more ubiquitous, computing has started to embed into our environment in various forms such as intelligent thermostats, smart appliances, remotely controllable household equipment, and weather based automated lawn irrigation systems. Consequently, we need new ways to seamlessly and effectively communicate and interact with such ubiquitous and always-available computing devices. A natural choice for such communication and interaction is human gestures because gestures are an integral part of the way humans communicate and interact with each other in their daily lives. This project aims at using ambient light and cheap commercial off-the-shelf light sensors to develop a gesture recognition system. The intuition behind this approach is that as a user performs a gesture in a room that is lit with light, the amount of light that he/she reflects and blocks changes, resulting in changes in the intensity of light in all parts of the room. The patterns of change in the intensity of light are different for different gestures, which can be learnt and used to recognize the gestures.In developing the ambient light based gesture recognition system, this project has two primary objectives: (1) environment independence, i.e., making the system agnostic to the characteristics of the environment, such as different lighting conditions, and (2) user independence, i.e., making the system agnostic to the number of users in a room and their routine activities. Several challenges arise in developing the ambient light based gesture recognition system, such as automatically detecting the start and end of a gesture, removing noise from the time-series of sensor values, handling the varying time durations of the different occurrences of the same gesture, simultaneously recognizing the gestures of multiple people, and recognizing the gestures of non-stationary users. This project will not only address these and other similar challenges, but will also advance the knowledge and understanding of the use of ambient light for novel systems by yielding a theoretical foundation for modeling human gestures and routine activities using changes in the intensity of light.The successful completion of this project will greatly benefit our society. First, this project will make the data set collected during this project publicly available for research. Second, the proposed ambient light based gesture recognition system will introduce a new and convenient way for users to interact with the computing embedded in their environments. Third, the proposed project will bridge several different communities such as systems, signal processing, machine learning, mobile computing, and human computer interaction; and foster interaction and communication among them. Fourth, the educational side of the project will integrate the research findings into the undergraduate and graduate curricula at North Carolina State University.