A Motion-Analysis VLSI Image Sensor System Extracting the Meaning of Action From Moving Images

人类的感知是基于大脑对过去经验的自动检索，这些经验与眼前发生的当前事件最为相关。基于这一假设，本课题组开发了一种模拟人脑处理的类人智能超大规模集成电路系统。本研究的目的是进一步将系统的能力从对静止图像的理解扩展到对运动图像中的运动和动作的识别。在理解运动的过程中，从运动图像中生成运动场并用特征向量表示是至关重要的。然而，它们对计算的要求非常高，因此，专门的VLSI硬件开发是必须的，以实现系统的实时性能。采用一种新的时域硬件计算算法，研制了一种能够以500fps的速度检测正常光流的模拟CMOS图像传感器。提出了一种基于块匹配算法的更精确的运动场生成方法。利用方向边缘信息及其直方图匹配，构建了一种数字运动场生成处理器，并通过测量对其进行了评价。在20MHz这样的低时钟频率下，它比使用2.8GHzCPU的软件处理速度快1000倍以上。基于生成的运动场，形成时间序列的PPMD（投影主运动分布）向量，并使用隐马尔可夫模型识别动作。通过对PPMD矢量序列进行时空整合，提出了另一种运动表示形式“运动历史矢量”，并通过简单的模板匹配成功地应用于原始手势识别。演示了该系统在自我运动感知和目标跟踪方面的应用。(274字)

Human perception is based on the automatic retrieval of past experience in the brain that is most relevant to the current event happening in front. Based on this postulate, a human-like intelligent VLSI system has been developed in our group mimicking the processing in the mind. The objective of this research is to further extend the capability for the system from the understanding of still images to the recognition of motions and actions in moving images. In understanding the motion, generating motion fields from moving images and representing them by feature vectors are of primary importance. However, they are computationally very demanding, and therefore, specialized VLSI hardware development is mandatory to achieve a real time performance of the system. An analog CMOS image sensor capable of detecting normal optical flow at 500fps has been developed employing a new time-domain hardware computation algorithm. A more accurate motion field generation based on the block matching algorithm has also been developed. By using the directional edge information and their histogram matching, a digital motion-field-generation processor has been built and evaluated by measurements. It shows more than 1000 times faster a performance than the software processing using 2.8GHzCPU at such a low clock frequency as 20MHz. Based on the motion fields thus generated, a time sequence of PPMD (Projected Principal Motion Distribution) vectors are formed and the action is recognized using Hidden Markov Models. Another motion representation called "Motion History Vector" has been developed by spatio-temporally integrating the PPMD vector sequence, which has been successfully applied to primitive gesture recognition by simple template matching. Application of the system to ego motion perception and object tracking are also demonstrated. (274 words)