Tensorial modeling of dynamical systems for gait and activity recognition

随着监控和安全的自动识别系统开始广泛普及，人脸、虹膜或指纹识别等生物识别技术在过去十年中受到了越来越多的关注。然而，它们受到两个主要限制：它们不能远距离使用，需要用户合作，这些假设在现实世界中是不切实际的。有趣的是，心理学研究表明，人们能够通过他们走路的方式来识别他们的朋友，即使他们的步态在点光源显示中不能很好地表现出来。与其他生物特征识别方法相比，步态具有几个优点，因为它可以远距离测量，难以伪装或遮挡，即使在低分辨率图像中也可以识别，并且本质上是非合作的。此外，步态和人脸生物特征可以很容易地结合起来进行身份识别。尽管步态识别具有诱人的特点，但步态识别还远未准备好在实践中部署。限制其在真实世界场景中采用的原因是影响步态外观和动态的大量滋扰因素的影响。这些因素包括，例如：行走表面、照明、摄像机设置(视点)，但也包括鞋和衣服、携带的物体、执行时间、行走速度。运动分类的其他应用程序也存在类似的问题，例如动作和活动识别。多线性或张量模型，其中一些(滋扰)因素线性混合，产生我们观察到的东西(在我们的情况下，步行步态)，最近被证明能够描述这些因素的影响，例如在人脸识别的背景下。然而，视频序列是比单个图像更复杂的对象。我们首先需要以一种紧凑的方式表示视频片段，通过某种动态模型来编码视频的动态已经被证明在动作识别和步态识别中都是有效的，在动态具有严重区分性的情况下。此外，必须从视频序列中在时间上分割感兴趣的动作，而有时可能必须比较长度非常不同的动作。动态表示在处理时间检测和压缩方面非常有效，事实上，一些研究人员已经探索了通过线性、非线性、随机或混沌动态系统来编码运动的思想。因此，在本项目中，我们提出了一种新颖的、通用的视频序列分类框架(重点关注步行步态)，该框架基于张量分解技术对表示为合适的动态模型的图像序列进行表示。该框架将允许我们以原则性的方式处理对步态识别和活动识别有很大影响的干扰因素的问题。主要目标是推动更广泛地传播步态ID，作为在当前不确定情况下提高该国安全水平的具体贡献。由于其对预防犯罪和安全的影响，生物识别和监控是快速增长的商业领域，这一事实从大多数发达经济体政府支持的该领域越来越多的倡议中可见一斑。此外，本提案中设计的技术可扩展到具有巨大商业开发潜力的动作和身份识别，范围从从YouTube等存储库基于内容的视频检索到HMI，再到互动视频游戏等。

Biometrics such as face, iris, or fingerprint recognition have received growing attention in the last decade, as automatic identification systems for surveillance and security have started to enjoy widespread diffusion. They suffer, however, from two major limitations: they cannot be used at a distance, and require user cooperation, assumptions impractical in real-world scenarios. Interestingly, psychological studies show that people are capable of recognizing their friends just from the way they walk, even when their gait is poorly represented by point light display. Gait has several advantages over other biometrics, as it can be measured at a distance, is difficult to disguise or occlude, can be identified even in low-resolution images, and is non-cooperative in nature. Furthermore, gait and face biometrics can be easily integrated for human identity recognition.Despite its attractive features, though, gait identification is still far from being ready to be deployed in practice. What limits its adoption in real-world scenarios is the influence of a large number of nuisance factors which affect appearance and dynamics of the gait. These include, for instance: walking surface, lighting, camera setup (viewpoint), but also footwear and clothing, objects carried, time of execution, walking speed. Similar issues are shared by other applications of motion classification, such as action and activity recognition. Multilinear or tensorial models, in which a number of (nuisance) factors linearly mix to generate what we observe (in our case the walking gait), have been proven in the recent past to be able to describe the influence of such factors, for instance in the context of face recognition. However, video sequences are more complex objects than single images. We first need to represent video footages in a compact way.Encoding the dynamics of videos by means of some sort of dynamical model has been proven effective in both action recognition and gait identification, in situations in which the dynamics is critically discriminative. Besides, the actions of interest have to be temporally segmented from a video sequence, while actions of sometimes very different lengths might have to be compared. Dynamical representations are very effective in coping with temporal detection and compression, and indeed several researchers have explored the idea of encoding motions via linear, nonlinear, stochastic or chaotic dynamical systems.In this project, therefore, we propose to develop a novel, general framework for the classification of video sequences (with a focus on the walking gait), based on the application of tensorial decomposition techniques to image sequences represented as realizations of suitable dynamical models.The proposed framework will allow us to deal with the issue of the nuisance factors which greatly affect identification from gait and activity recognition in a principled way. The main goal is to push towards a more widespread diffusion of gait ID, as a concrete contribution to enhancing the security levels in the country in the current, uncertain scenarios. With their implications for crime prevention and security, biometrics and surveillance are fast growing business areas, a fact reflected by the increasing number of government-sponsored initiatives in the area in most advanced economies. In addition, the techniques devised in this proposal are extendable to action and identity recognition with immense commercial exploitation potential, ranging from content-based video retrieval from repositories such as YouTube, to HMI, to interactive video games, etcetera.