Machine Networks of Attention from Human Networks of Attention

机器注意力网络与人类注意力网络

• 批准号: 2889016
• 金额: --
• 依托单位: Swansea University
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2889016
• 关键词: Machine; Networks; Attention; Human

The addition of 'attention' to machine learning has recently improved many algorithms with the potential to transform a wide range of machine learning approaches (transformers, perceivers). Attention, simply, allows an algorithm to allocate more weight to input that is relevant for certain tasks, and less weight to the irrelevant. We propose that attention mechanisms in machine learning will become increasingly important as the volume of input data increases, and even efficient algorithms will have to make informed choices about which input should receive priority or actively inhibited. We will apply current theories of human attention to improve machine learning algorithms that adjust to the goals of the agent. Biological attention has been studied for more than 100 years and comprises multiple overlapping networks that help an organism allocate neural processing to sensory input that is important to a given task. For example, the orienting network uses eye movements and shifts of spatial attention to inspect important areas of our environment. The executive control network adjusts sensory priority for our evolving goals. We will use high quality eye tracking data in various tasks as a proxy for human attention and use these data to inform novel attentional mechanisms for machine learning. The key objectives of the research will be to improve existing attention mechanisms in machine learning algorithms such that they are better able to prioritize input data for a given task. The resulting improved efficiency will reduce resources needed for existing tasks and and improve the scope of these algorithms for more computationally expensive tasks. By explicitly testing mechanisms that we know from human visual processing, we will gain a better understanding of how machine and human attention might work together in joint tasks.

对机器学习的关注最近改进了许多算法，有可能改变广泛的机器学习方法(转换器、感知器)。简单地说，注意力允许算法将更多的权重分配给与某些任务相关的输入，而将更少的权重分配给不相关的输入。我们认为，随着输入数据量的增加，机器学习中的注意机制将变得越来越重要，即使是高效的算法也必须做出明智的选择，决定哪些输入应该优先或主动禁止。我们将应用当前的人类注意力理论来改进机器学习算法，以适应代理的目标。生物注意力的研究已经有100多年的历史了，它由多个重叠的网络组成，这些网络帮助有机体将神经处理分配给对给定任务至关重要的感觉输入。例如，定位网络使用眼球运动和空间注意力的转移来检查我们环境的重要领域。执行控制网络根据我们不断进化的目标调整感觉优先级。我们将在各种任务中使用高质量的眼睛跟踪数据作为人类注意力的代理，并使用这些数据来为机器学习提供新的注意机制。这项研究的主要目标将是改进机器学习算法中现有的注意力机制，以便它们能够更好地为给定任务的输入数据排序优先级。由此提高的效率将减少现有任务所需的资源，并扩大这些算法用于计算成本更高的任务的范围。通过明确测试我们从人类视觉处理中了解到的机制，我们将更好地理解机器和人类的注意力如何在联合任务中协同工作。