Transfer Learning for Frame-based Activity Recognition

活动识别是家庭监控的一项重要任务，可以监控儿童，宠物和老人的健康状况以及安全目的。尽管越来越多的家庭使用的视频监视器很少提供智能监控。因此，我们认为，这项研究将致力于视觉活动识别，以检测和分类可用于确定个人健康的动作。大多数活动识别研究都集中在识别视频中的单个高级动作（例如，踢足球或做三明治），然而，在家庭监视的上下文中，基于帧的活动识别提供了更有意义的信息，其提供了低用于活动识别的现有技术方法结合卷积（CNN）和/或递归神经网络（RNN）。通过这些方法，将视频中的时间信息结合起来，大大提高了识别的准确性。流行的活动识别方法包括使用用于分类的CNN从RGB和光流帧中提取特征，或者在CNN提取的RGB特征上训练长短期记忆单元（LSTM）。这些技术在数据集上取得了不同的成功，特别是对于基于帧的动作识别，与手工制作的特征相比，这些动作识别只提供了很小的好处。这种缺乏成功的部分原因是由于缺乏可用的训练数据，这是由于收集和注释动作数据集的困难。迁移学习已被证明可以提高对象识别的准确性，其中测试模型的特定环境几乎没有训练数据。通过在较大的对象数据集上预训练神经网络，模型可以在测试环境中微调模型之前学习与测试环境相关的特征，而几乎没有可用的训练数据。时间域中的迁移学习对动作识别很重要，但研究很少。这项工作将集中在设计模型，以提高低级别动作的基于帧的活动识别的准确性，这些动作可以很好地迁移到缺乏大量训练数据的不同环境中。

Activity recognition is an important task for home surveillance to monitor the wellbeing of children, pets and elderly as well as for security purposes. Despite an increasing number of video monitors used in households little provide smart monitoring. Therefore, this research will be working towards visual activity recognition to detect and classify actions that can be used to determine the health of an individual.Most research towards activity recognition has focused on recognising a single high-level action in a video (e.g. playing football or making a sandwich) however in the context of home surveillance frame-based activity recognition provides more meaningful information that provide low-level actions (e.g. put down plate or pick up mug) for each frame as soon as the action occurs.State of the art methods for activity recognition incorporate Convolutional (CNN) and/or Recurrent Neural Networks (RNN). With these methods, incorporating temporal information across a video greatly improves the accuracy of recognition. Popular approaches to activity recognition include extracting features from both RGB and Optical Flow frames using CNNs used for classification, or to train Long Short Term Memory units (LSTM) on the RGB features extracted by CNNs. These techniques have shown varying success across datasets, particularly for frame based action recognition that provide only a small benefit compared to hand crafted features. This lack of success is partly due to the lack of available training data due to the difficulties in collecting and annotating datasets for actions.Transfer learning has shown to improve the accuracy of object recognition where the specific environment to test the models on has little training data. By pre-training Neural Networks on larger datasets of objects the models can learn features that are also relevant to the test environment before fine-tuning the model on test environment with little training data available. Transfer learning in the temporal domain, shown to be important for action recognition, has had little research.This work will focus on designing models to improve the accuracy of frame-based activity recognition of low level actions that transfer well to different environments that lack large amounts of training data.