A 127mW 1.63TOPS sparse spatio-temporal cognitive SoC for action classification and motion tracking in videos
A 127mW 1.63TOPS sparse spatio-temporal cognitive SoC for action classification and motion tracking in videos
复制标题
127mW 1.63TOPS 稀疏时空认知 SoC，用于视频中的动作分类和运动跟踪
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Zhengya Zhang
中科院分区：
文献类型：
--
作者：
Ching;Thomas Chen;Zhengya Zhang
设计了一种稀疏时空（ST）认知SoC，用于从视频中提取ST特征，以进行动作分类和运动跟踪。SoC内核是一个稀疏ST卷积自动编码器，使用3层网络实现递归。在每一层处理中强制执行高稀疏性，将ST卷积的复杂性降低两个数量级，并允许所有乘法累加（MAC）被选择加法（SA）取代。该设计在3.98mm2 40nm CMOS SoC中进行了演示，该SoC采用OpenRISC处理器提供软件定义的控制和分类。应用ST内核压缩以减少43%的内存。在0.9V和240MHz下，SoC实现了1.63TOPS，以满足60fps 1920×1080 HD视频数据速率，功耗为127mW。
A sparse spatio-temporal (ST) cognitive SoC is designed to extract ST features from videos for action classification and motion tracking. The SoC core is a sparse ST convolutional auto-encoder that implements recurrence using a 3-layer network. High sparsity is enforced in each layer of processing, reducing the complexity of ST convolution by two orders of magnitude and allowing all multiply-accumulates (MAC) to be replaced by select-adds (SA). The design is demonstrated in a 3.98mm2 40nm CMOS SoC with an OpenRISC processor providing software-defined control and classification. ST kernel compression is applied to reduce memory by 43%. At 0.9V and 240MHz, the SoC achieves 1.63TOPS to meet the 60fps 1920×1080 HD video data rate, dissipating 127mW.