RI: Medium: Assessing Speaker and Teacher Effectiveness through Gestural Analysis, EEG Recordings, and Eye Tracking

RI：中：通过手势分析、脑电图记录和眼动追踪评估演讲者和教师的有效性

• 批准号: 1513853
• 负责人: John Kender
• 金额: $ 89.98万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1513853&HistoricalAwards=false
• 关键词: RI; Medium; Assessing; Speaker; Teacher

This project helps speakers and teachers to measure and improve their impact on their audiences. It uses visual observations of body, head, and hand gestures of the communicator, plus recordings of brain activity and eye movements of the audience. Together, these determine which sections of a presentation elicit the most audience engagement. The project is developing new methods to capture and calibrate electroencephalogram and eye-tracking data from listeners and from students. It is determining new ways to relate this subject information to what a speaker or teacher can be seen to be doing while developing an argument or reviewing a concept. The project produces analyses of when and how the communicator is most effective. This system is being ported to the Columbia Video Network distance education facility, for their use in improving the online delivery of Columbia University Master's level technical courses. This project continues a research effort that has involved women, minorities, disabled students, and undergrads.This research investigates the degree to which certain speaker gestures can convey significant information that are correlated to audience engagement, in speeches and in classroom lectures. The project develops and validates a catalog of gestural attributes derived from pose and movements of body, head, and hand, and automatically extracts these attributes from videos. It demonstrates correlations between gesture attributes and an objective method of measuring audience engagement: electroencephalography (EEG). The project leverages a multi-disciplinary approach, with neural engineers and computer/media scientists collaborating to build a system that identifies and tracks physiological measures of engagement, and relates these to features in the video as well as information content. It records subjects' high-density EEG, and tracks their eyes and pupillary responses while they are watching video lectures. It uses machine learning, specifically novel methods which expand upon canonical correlation analysis, to relate inter- and intra-subject correlations, between the physiological changes and the gestural features derived from the video by using enhanced computer vision techniques. These measures are further integrated with pupillary measures, which have been shown to correlate with arousal, as well as with gaze measures, which are indicative of attention. The project is producing an analysis of body, head, and hand gestures useful in persuasion and in education, and a catalog of their influence on engagement and speaker effectiveness.

该项目帮助演讲者和教师衡量和改善他们对听众的影响。 它使用视觉观察的身体，头部，和手势的沟通，加上记录的大脑活动和眼球运动的观众。 这些因素共同决定了演示文稿的哪些部分最能吸引观众的参与。该项目正在开发新的方法来捕获和校准来自听众和学生的脑电图和眼动跟踪数据。它正在确定新的方法，将这些主题信息与演讲者或教师在发展论点或回顾概念时所做的事情联系起来。 该项目分析了沟通者何时以及如何最有效。该系统正在移植到哥伦比亚视频网络远程教育设施，用于改进哥伦比亚大学硕士技术课程的在线提供。本项目延续了一项涉及妇女、少数民族、残疾学生和本科生的研究工作，调查了演讲者和课堂讲座中某些演讲者手势传达与听众参与相关的重要信息的程度。 该项目开发并验证了一个从身体，头部和手部的姿势和运动中获得的手势属性目录，并自动从视频中提取这些属性。 它展示了手势属性和测量观众参与度的客观方法之间的相关性：脑电图（EEG）。 该项目利用多学科方法，神经工程师和计算机/媒体科学家合作建立一个系统，识别和跟踪参与的生理指标，并将这些指标与视频中的特征以及信息内容联系起来。 它记录受试者的高密度脑电图，并跟踪他们在观看视频讲座时的眼睛和瞳孔反应。 它使用机器学习，特别是扩展典型相关分析的新方法，通过使用增强的计算机视觉技术，将生理变化和从视频中导出的手势特征之间的主体间和主体内相关性联系起来。 这些测量进一步与瞳孔测量相结合，瞳孔测量已被证明与唤醒相关，以及与凝视测量相结合，凝视测量指示注意力。 该项目正在对身体、头部和手势进行分析，这些手势在说服和教育中很有用，并列出了它们对参与度和演讲者有效性的影响。