Towards Machine Recognition of Facial Expressions of Pain in Horses.
Towards Machine Recognition of Facial Expressions of Pain in Horses.
复制标题
马匹疼痛面部表情的机器识别。
DOI：
10.3390/ani11061643
复制
发表时间：
2021-06-01
期刊：
影响因子：
--
通讯作者：
Kjellström H
中科院分区：
文献类型：
--
作者：
Andersen PH;Broomé S;Rashid M;Lundblad J;Ask K;Li Z;Hernlund E;Rhodin M;Kjellström H
面部活动可以传达有关马疼痛经历的有效信息。然而，基于面部活动对马的疼痛进行评分仍处于起步阶段，准确的评分只能由训练有素的评估人员进行。现在可以利用计算机视觉和机器学习，从面部视频片段中可靠地识别人类的疼痛。我们研究了将这些技术应用于马匹的障碍，并提出了两种自动马匹疼痛识别的通用方法。第一种方法涉及自动检测客观定义的面部表情方面，不涉及任何人类对表情“含义”的判断。然后可以根据基于规则的系统来完成疼痛表情的自动分类，因为面部表情方面是根据该信息来定义的。另一个涉及使用已知真实疼痛状态的马的原始视频来训练非常灵活的机器学习方法。这种方法的优点是系统可以访问视频中的所有信息，而无需设计过滤掉大部分变化的中间方法。然而，一个巨大的挑战是需要具有可靠疼痛注释的大型数据集。我们从这两种方法中都获得了有希望的结果。使用基于计算机视觉和机器学习的方法，自动识别人类痛苦和情绪的面部表情在某种程度上是一个已解决的问题。然而，事实证明将这种方法应用于马是很困难的。主要障碍是缺乏足够大的带注释的马数据库，以及由于马是非语言的而难以获得正确的疼痛分类。这篇评论描述了我们使用两种不同方法克服这些障碍的工作。其中一种涉及使用手动但相对客观的面部活动分类系统（面部动作编码系统），在使用机器学习原理编码后对数据进行疼痛表情分析。我们设计了一些工具，可以通过识别马的面部和面部关键点来帮助手动标记。这种方法在从图像中自动识别面部动作单元方面提供了有希望的结果。第二种方法是循环神经网络端到端学习，需要从视频中提取较少的特征和表示，而是依赖于大量具有基本事实的视频数据。我们的初步结果清楚地表明，动力学对于疼痛识别很重要，并且表明循环神经网络的组合可以比人类评估者更好地对少数马匹的实验疼痛进行分类。
Facial activity can convey valid information about the experience of pain in a horse. However, scoring of pain in horses based on facial activity is still in its infancy and accurate scoring can only be performed by trained assessors. Pain in humans can now be recognized reliably from video footage of faces, using computer vision and machine learning. We examine the hurdles in applying these technologies to horses and suggest two general approaches to automatic horse pain recognition. The first approach involves automatically detecting objectively defined facial expression aspects that do not involve any human judgment of what the expression “means”. Automated classification of pain expressions can then be done according to a rule-based system since the facial expression aspects are defined with this information in mind. The other involves training very flexible machine learning methods with raw videos of horses with known true pain status. The upside of this approach is that the system has access to all the information in the video without engineered intermediate methods that have filtered out most of the variation. However, a large challenge is that large datasets with reliable pain annotation are required. We have obtained promising results from both approaches. Automated recognition of human facial expressions of pain and emotions is to a certain degree a solved problem, using approaches based on computer vision and machine learning. However, the application of such methods to horses has proven difficult. Major barriers are the lack of sufficiently large, annotated databases for horses and difficulties in obtaining correct classifications of pain because horses are non-verbal. This review describes our work to overcome these barriers, using two different approaches. One involves the use of a manual, but relatively objective, classification system for facial activity (Facial Action Coding System), where data are analyzed for pain expressions after coding using machine learning principles. We have devised tools that can aid manual labeling by identifying the faces and facial keypoints of horses. This approach provides promising results in the automated recognition of facial action units from images. The second approach, recurrent neural network end-to-end learning, requires less extraction of features and representations from the video but instead depends on large volumes of video data with ground truth. Our preliminary results suggest clearly that dynamics are important for pain recognition and show that combinations of recurrent neural networks can classify experimental pain in a small number of horses better than human raters.
登录
查看更多内容
影响因子：
3
作者：
Dalla Costa, Emanuela;Stucke, Diana;Lebelt, Dirk
通讯作者：
Lebelt, Dirk
DOI：
10.3390/ani10112155
发表时间：
2020-11-19
期刊：
Animals : an open access journal from MDPI
影响因子：
--
作者：
Ask K;Rhodin M;Tamminen LM;Hernlund E;Haubro Andersen P
通讯作者：
Haubro Andersen P
影响因子：
2.1
作者：
Browne, M. Erin;Hadjistavropoulos, Thomas;Taati, Babak
通讯作者：
Taati, Babak
影响因子：
2.4
作者：
Bussieres, G.;Jacques, C.;Troncy, E.
通讯作者：
Troncy, E.
DOI：
10.1016/j.jveb.2017.03.005
发表时间：
2017-05-01
影响因子：
1.8
作者：
Dyson, Sue;Berger, Jeannine M.;Mullard, Jessica
通讯作者：
Mullard, Jessica