Transfer Learning for Frame-based Activity Recognition

活动识别是家庭监控的一项重要任务，可以监测儿童、宠物和老人的健康状况，也可以用于安全目的。尽管家庭中使用的视频监视器越来越多，但提供智能监控的却很少。因此，这项研究将致力于视觉活动识别，以检测和分类可用于确定个人健康状况的行为。大多数关于活动识别的研究都集中在识别视频中的单个高级动作（例如踢足球或做三明治），然而在家庭监控的背景下，基于帧的活动识别提供了更有意义的信息，提供了每帧动作发生时的低级动作（例如放下盘子或拿起杯子）。最先进的活动识别方法包括卷积（CNN）和/或循环神经网络（RNN）。在这些方法中，结合视频中的时间信息大大提高了识别的准确性。常用的活动识别方法包括使用cnn从RGB和光流帧中提取特征用于分类，或者在cnn提取的RGB特征上训练长短期记忆单元（LSTM）。这些技术在数据集上取得了不同程度的成功，特别是基于帧的动作识别，与手工制作的特征相比，它只提供了很小的好处。这种缺乏成功的部分原因是由于缺乏可用的训练数据，这是由于收集和注释行动数据集的困难。在训练数据较少的特定环境下，迁移学习可以提高目标识别的准确性。通过在更大的对象数据集上预训练神经网络，模型可以学习与测试环境相关的特征，然后在测试环境上微调模型，使用很少的训练数据。时间域的迁移学习对动作识别很重要，但研究很少。这项工作将侧重于设计模型，以提高基于框架的低级动作识别的准确性，这些低级动作可以很好地转移到缺乏大量训练数据的不同环境中。

Activity recognition is an important task for home surveillance to monitor the wellbeing of children, pets and elderly as well as for security purposes. Despite an increasing number of video monitors used in households little provide smart monitoring. Therefore, this research will be working towards visual activity recognition to detect and classify actions that can be used to determine the health of an individual.Most research towards activity recognition has focused on recognising a single high-level action in a video (e.g. playing football or making a sandwich) however in the context of home surveillance frame-based activity recognition provides more meaningful information that provide low-level actions (e.g. put down plate or pick up mug) for each frame as soon as the action occurs.State of the art methods for activity recognition incorporate Convolutional (CNN) and/or Recurrent Neural Networks (RNN). With these methods, incorporating temporal information across a video greatly improves the accuracy of recognition. Popular approaches to activity recognition include extracting features from both RGB and Optical Flow frames using CNNs used for classification, or to train Long Short Term Memory units (LSTM) on the RGB features extracted by CNNs. These techniques have shown varying success across datasets, particularly for frame based action recognition that provide only a small benefit compared to hand crafted features. This lack of success is partly due to the lack of available training data due to the difficulties in collecting and annotating datasets for actions.Transfer learning has shown to improve the accuracy of object recognition where the specific environment to test the models on has little training data. By pre-training Neural Networks on larger datasets of objects the models can learn features that are also relevant to the test environment before fine-tuning the model on test environment with little training data available. Transfer learning in the temporal domain, shown to be important for action recognition, has had little research.This work will focus on designing models to improve the accuracy of frame-based activity recognition of low level actions that transfer well to different environments that lack large amounts of training data.