Facial Re-identification Using Deep Learning on Combined Real-Virtual Environments

人脸识别、身份鉴别和身份验证是生物特征识别领域的常见问题。多年来，人们开发了许多方法，产生了强大而准确的方法，特别是随着深度神经网络的到来。然而，尽管正在取得进展，但当扩展到数百/数千个用户时，算法仍然受到环境和用户上下文引入的多个可变性来源的影响，例如头部取向，面部配件，不同的背景和使用不同的记录设备。这项工作的目的是解决这些限制，并开发一个新的统一框架的特征提取和度量学习使用深度学习架构。通过将人脸识别与重新识别相结合，我们的目标是将我们的方法扩展到庞大的数据集，例如当前的社交网络或国家警察数据库，在大多数情况下，每个主题只有几张图像（例如他们的个人资料照片或警察的面部照片）。该算法应该能够使用这几张照片来识别低分辨率和不同视点和姿势的图像质量的CCTV镜头中的主题。在真实的世界监控摄像机上测试这种虚拟增强的人脸重新识别范例将是该项目的基本目标。该项目旨在开发一种方法，可以使用一个或两个来自警察的面部照片的图像来识别CCTV流中的个人。这项工作的目标包括：-实现深度卷积神经网络来解决自动特征提取。- 将联合收割机自动特征提取与暹罗网络架构中的度量学习相结合，以专门解决验证和重新识别的问题。- 研究在零拍和一拍场景下人脸识别的新配置。- 扩展重新识别框架，允许图像到视频识别，能够解决质量差的CCTV镜头。- 探索强大的策略，如数据增强和丢失，以解决面部遮挡和其他可能影响识别的变化来源，如不同的姿势，服装，化妆品，眼镜，围巾，帽子等-开发新的深度学习架构，使用上下文或主题配置文件中的语义信息增强视觉面部重新识别。- 为了评估验证与重新识别的性能和局限性，将真实的生活快照与社交网络配置文件和/或CCTV捕获相关联。这项研究对预防犯罪具有潜在影响-其中面部照片可以与CCTV摄像头匹配，人口贩运-其中非法网站中的图片可以与入境口岸拍摄的图像进行比较-以及提高机场安全性，- 将护照照片或嫌疑人观察名单上的图像与机场摄像头进行比较，从而实施更安全和透明的电子门。这项研究符合人工智能技术和图像与视觉计算的EPSRC研究领域。

Face recognition, identification and verification are common problems in the field of biometrics. Many approaches have been developed over the years resulting in robust and accurate methodologies, specially with the arrival of deep neural networks. However, in spite of the ongoing progress, algorithms still suffer when scaling to hundreds/thousands of users with the multiple sources of variability introduced by the environment and the user context, such as head orientation, facial accessories, different backgrounds and the use of different recording devices.The aim of this work is to address these limitations and to develop a novel unified framework for feature extraction and metric learning using deep learning architectures. By assimilating face recognition to re-identification, we aim to extend the application of our methodology to vast datasets such as current social networks or national police databases, where in most cases only a few images per subject are available (for example their profile picture or police mugshots). The algorithm should be able to use these few photos to identify subjects in CCTV footage at low resolution and image quality in different view-points and poses. The testing of such a virtually enhanced face re-identification paradigm on real world surveillance cameras will be the underlying objective of this project. The project will aim to develop an approach which can use one or two images from police mugshots to identify individuals in CCTV stream.The objectives of this work include: - To implement deep convolutional neural networks to tackle automatic feature extraction. - To combine automatic feature extraction with metric learning in Siamese network architectures, to specifically address the problem of verification and re-identification. - To investigate new configurations for face recognition in zero-shot and one-shot scenarios. - To extend the re-identification framework to allow image-to-video recognition that's able to tackle poor quality CCTV footage. - To explore robust strategies such as data augmentation and dropout to address facial occlusions and other sources of variation which may affect identification, such as different poses, clothing, cosmetics, glasses, scarfs, hats, etc. - To develop new deep learning architectures that enhance visual facial re-identification using semantic information from the context, or the subject profile. - To evaluate the performance and limitation of verification versus re-identification, to link real life snapshots with social network profiles and/or CCTV captures.This research has potential impact for crime prevention -where mugshots can be matched against CCTV cameras, human trafficking -where pictures in illegal websites can be compared against images taken at the port of entry- and increasing security in airports, -where passport pictures or images in a suspect watch list are compared against the airport cameras, leading to the implementation of more secure and transparent e-gates. This research fits the EPSRC Research areas of Artificial Intelligence Technologies and Image and Vision Computing.