Cognitive Systems Foresight: Human Attention and Machine Learning

人类观察者移动他们的眼睛是为了将他们的注意力引向视觉场景的重要方面。有一种模型叫做显著性地图；它们能预测在观看场景时眼睛的移动方向。目前，这些模型不处理视频输入，也不能预测观察者的任务将如何影响他们的视线。换句话说，对于现实生活中的观看情况，没有模型，在这种情况下，观察者有特定的任务。我们正在提出解决这个问题的新方法。我们可以从城市监控中使用的摄像机获取视频信息，也可以接触到操作员，他们的工作就是在这些视频输入中发现异常行为。我们将获得（以前未见过的）英国城市街道事件的视频记录，并在模拟控制室中将其展示给熟悉该城镇的操作员。我们将监视他们在视频屏幕上的位置，以及当他们决定某一事件不正常和/或需要某种形式的干预时，例如报警。我们将使用眼睛注视的记录来教计算机系统区分正常和异常事件。通过这种方式，我们将能够通过观察人类的眼睛注视行为来了解对人类进行这种监视的重要意义，这是一个现实的（和困难的）任务和一组现实生活中的视频序列。这个项目之所以重要，有四个原因。首先，这将是开发人类注意力/眼球运动模型的第一次尝试，该模型将牢固地基于真实的视频输入和真实的任务。其次，这将是计算机系统第一次能够以这种方式从人类行为中学习。第三，随着电视监视器数量的增加，我们将更多地了解训练有素的观察员应付一项要求很高的任务的能力。最后，我们将开发一个自动化系统，该系统将能够分析来自任何城市闭路电视摄像机的输入，以便提醒操作员查看视频流——目前，大多数闭路电视视频流都没有被任何人观察到，因为摄像机太多，无法容纳观察者的数量。因此，一个自动报警系统是非常必要的，这个项目是迄今为止最好的尝试。

Human observers move their eyes in order to direct their attention to important aspects of a visual scene. There are models called salience maps; they predict where the eyes will move to when looking at a scene. At present, these models do not deal with video input, nor do they predict how an observer's task will affect where they look. In other words, there are no models for real-life viewing situations, where an observer has a specific task.We are proposing a new approach to this problem. We have access to video information from cameras used in urban surveillance, and to the operators whose job it is to spot abnormal behaviour in such video inputs. We shall obtain (previously unseen) video recordings of events in UK urban streets, and display them in a simulated control room to operators familiar with the town in question. We shall monitor where they look on the bank of video screens, and also when they decide that an event is abnormal and/or requires some form of intervention, e.g. calling the police. We shall use the record of eye fixations to teach a computer system to distinguish between normal and abnormal events. In this way, we shall be able to learn what is important for humans to do such surveillance by observing their eye fixation behaviour, for a realistic (and difficult) task and set of real-life video sequences. The project is important for four reasons. First, this will be the first attempt to develop a model of human attention/eye movements which will be firmly based on realistic video input and a real task. Second, this will be the first time that a computer system is able to learn from human behaviour in this way. Third, we will learn much about the ability of trained observers to cope with a demanding task as the number of TV monitors increases. Finally, we will develop an automated system which will be able to analyse the input from any urban CCTV camera in order to alert operators to look at that video stream - at present, most CCTV video streams are not observed by anyone since there are too many cameras for the number of human observers. Therefore, an automated alerting system is greatly neeeded and this project constitutes the best attempt to date to produce one.