RI: Medium: Assessing Speaker and Teacher Effectiveness through Gestural Analysis, EEG Recordings, and Eye Tracking

该项目帮助演讲者和教师衡量和改善他们对听众的影响。它使用对传播者的身体、头部和手势的视觉观察，以及对观众的大脑活动和眼睛运动的记录。这些因素共同决定了演示文稿的哪些部分吸引了最多的观众。该项目正在开发新的方法来捕获和校准来自听众和学生的脑电和眼球跟踪数据。它正在确定新的方式，将这一主题信息与演讲者或教师在发展论点或审查概念时所做的事情联系起来。该项目对沟通者何时以及如何最有效进行分析。该系统正在被移植到哥伦比亚视频网络远程教育设施，用于改进哥伦比亚大学硕士级技术课程的在线教学。这一项目继续了一项涉及女性、少数族裔、残疾学生和本科生的研究工作。这项研究调查了某些演讲者的手势在多大程度上可以传达与听众参与相关的重要信息，在演讲和课堂演讲中。该项目开发并验证了从身体、头部和手的姿势和运动派生的手势属性目录，并自动从视频中提取这些属性。它展示了手势属性与衡量观众参与度的客观方法：脑电(EEG)之间的相关性。该项目利用多学科方法，神经工程师和计算机/媒体科学家合作建立一个系统，识别和跟踪参与的生理测量，并将这些与视频和信息内容中的特征联系起来。它记录受试者的高密度脑电，并在他们观看视频讲座时跟踪他们的眼睛和瞳孔反应。它使用机器学习，特别是在典型相关分析的基础上扩展的新方法，通过使用增强的计算机视觉技术，将来自视频的生理变化和手势特征之间的对象间和对象内的相关性联系起来。这些测量进一步与瞳孔测量以及凝视测量相结合，前者已被证明与唤醒相关，后者表明注意力。该项目正在对在说服和教育中有用的身体、头部和手势进行分析，并编制了它们对参与度和演讲者有效性的影响目录。

This project helps speakers and teachers to measure and improve their impact on their audiences. It uses visual observations of body, head, and hand gestures of the communicator, plus recordings of brain activity and eye movements of the audience. Together, these determine which sections of a presentation elicit the most audience engagement. The project is developing new methods to capture and calibrate electroencephalogram and eye-tracking data from listeners and from students. It is determining new ways to relate this subject information to what a speaker or teacher can be seen to be doing while developing an argument or reviewing a concept. The project produces analyses of when and how the communicator is most effective. This system is being ported to the Columbia Video Network distance education facility, for their use in improving the online delivery of Columbia University Master's level technical courses. This project continues a research effort that has involved women, minorities, disabled students, and undergrads.This research investigates the degree to which certain speaker gestures can convey significant information that are correlated to audience engagement, in speeches and in classroom lectures. The project develops and validates a catalog of gestural attributes derived from pose and movements of body, head, and hand, and automatically extracts these attributes from videos. It demonstrates correlations between gesture attributes and an objective method of measuring audience engagement: electroencephalography (EEG). The project leverages a multi-disciplinary approach, with neural engineers and computer/media scientists collaborating to build a system that identifies and tracks physiological measures of engagement, and relates these to features in the video as well as information content. It records subjects' high-density EEG, and tracks their eyes and pupillary responses while they are watching video lectures. It uses machine learning, specifically novel methods which expand upon canonical correlation analysis, to relate inter- and intra-subject correlations, between the physiological changes and the gestural features derived from the video by using enhanced computer vision techniques. These measures are further integrated with pupillary measures, which have been shown to correlate with arousal, as well as with gaze measures, which are indicative of attention. The project is producing an analysis of body, head, and hand gestures useful in persuasion and in education, and a catalog of their influence on engagement and speaker effectiveness.