Collaborative Research: CSL-MultiAD: Assessing Collaborative STEM Learning through Rich Information Flow based on Multi-Sensor Audio Diarization
协作研究:CSL-MultiAD:通过基于多传感器音频二值化的丰富信息流评估协作 STEM 学习
基本信息
- 批准号:1918032
- 负责人:
- 金额:$ 45万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2019
- 资助国家:美国
- 起止时间:2019-08-01 至 2024-07-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
The ability to learn concepts, especially for science and math (STEM) based disciplines, is impacted by educators who inspire, motivate, and create supportive environments and teaching methodologies which lower the entry barrier for students learning STEM subjects. Teaching resources nationwide have historically been constrained as STEM based science content for education expands with increasing student diversity based on prior science exposure in the classroom. A key aspect of student learning is to assess the quality of human communications between student-and-student as well as teacher-and-student. In STEM learning, students who are able to ask the right questions, know what they understand as well as what they need help with, allows educators to structure their teaching methods to help students overcome learning challenges. However, to date, it has been virtually impossible to collect and measure student-to-student or student-to-teacher voice communications in the classroom. Also, current speech technology is not sufficiently effective to overcome multi-speaker and naturalistic communications in classrooms. This project will develop classroom audio collection and measurement tools for students working together to solve problems, as well as teacher involvement with individual/groups of students. The audio collection solution includes both individual recorders on a sub-set of classroom students, as well as central smart speaker microphone collection units within each student group. Computer programs will be developed to analyze who is speaking and when, as well as spot keywords of interest for STEM topics and learning assessment. Privacy is maintained, since audio analysis is focused on high level measures such as individual student word counts, anonymous tagging of each speaker, and connecting conversational turns between students and teachers. A teacher driven keyword set will be used to help measure which students are having problems understanding concepts. These individual communication measured terms will be integrated into a dashboard display, to empower teachers with easy to use feedback on student engagement for STEM learning. The project has the potential to improve the ability to assess learning through classroom communications, and potentially help teachers better direct their time/expertise more efficiently to improve STEM learning for students. This project will develop ways to assess learning in classrooms by measuring the quality of human communication engagement between students-and-peers as well as teachers-and-students. Research has shown that learning is improved if there is dynamic interaction between student-to-student and student-to-teacher in voice communications. The project introduces personal recorders in the classroom to capture voice interactions during the entire day. Next, these multi-microphone recording streams are pooled together, where speech and language processing algorithms will be formulated to perform "audio diarization" - the process of determining "who spoke, what, and when", with potential keywords of interest based on classroom topics identified. The diarization output will drive the formulation of metrics to assess communication engagement. Communication based features derived from individual audio streams (word count, talk time, turn-taking, keyword profile) will be extracted on a per student basis through audio diarization. Next, this information flow will be used to develop class based group dynamics. This solution represents an approach for teachers to monitor student engagement over time in science activity areas, helping teachers identify students who are not verbally engaged in science discourse and quickly assess the impact of changes in classroom practices to improve learning. A number of technology challenges will be addressed for automatic audio stream based voice processing of naturalistic audio data using speech activity detection, speaker diarization based on machine learning models, and keyword spotting for science topic identification and tracking. These research aims will be assessed in classroom settings with teacher feedback on the effectiveness of the resulting solutions. The resulting speech technology advancements would offer new opportunities for future smart classrooms for voice assessment for teachers to better assess student involvement in science vs. infrequent traditional standardized testing. Ultimately, this effort will equip teachers with tools to identify and frequently monitor early indicators of disengagement in science learning, and potentially increase science interest by under-represented student populations and further diversify the STEM workforce.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
学习概念的能力,特别是科学和数学(STEM)为基础的学科,受到教育工作者的影响,他们激励,激励和创造支持性的环境和教学方法,降低学生学习STEM科目的入学门槛。全国范围内的教学资源在历史上一直受到限制,因为基于STEM的科学教育内容随着学生在课堂上先前科学接触的增加而增加。学生学习的一个关键方面是评估学生与学生以及教师与学生之间的人际沟通质量。在STEM学习中,能够提出正确问题的学生,知道他们理解什么以及他们需要什么帮助,允许教育工作者构建他们的教学方法,以帮助学生克服学习挑战。然而,迄今为止,在教室中收集和测量学生与学生或学生与教师的语音通信几乎是不可能的。此外,目前的语音技术还不足以有效地克服教室中的多扬声器和自然主义通信。 该项目将开发课堂音频收集和测量工具,供学生一起解决问题,以及教师与学生个人/团体的参与。音频收集解决方案包括教室学生子集上的单独录音机,以及每个学生组内的中央智能扬声器麦克风收集单元。将开发计算机程序来分析谁在说话,什么时候说话,以及发现STEM主题和学习评估的关键词。隐私得到维护,因为音频分析专注于高水平的措施,如个别学生的字数,每个发言者的匿名标记,并连接学生和教师之间的会话回合。教师驱动的关键字集将用于帮助衡量哪些学生在理解概念方面存在问题。这些单独的沟通测量术语将被集成到仪表板显示中,以使教师能够轻松使用关于学生参与STEM学习的反馈。该项目有可能提高通过课堂交流评估学习的能力,并有可能帮助教师更有效地指导他们的时间/专业知识,以改善学生的STEM学习。该项目将通过测量学生与同伴以及教师与学生之间的人际交往参与的质量来开发评估课堂学习的方法。研究表明,如果在语音通信中学生与学生和学生与教师之间存在动态交互,则学习会得到改善。该项目在教室中引入了个人录音机,以捕捉全天的语音交互。接下来,这些多麦克风记录流被汇集在一起,其中语音和语言处理算法将被制定为执行“音频日记”-确定“谁说话,什么,什么时候”的过程,以及基于课堂主题确定的潜在关键词。日志化产出将推动制定衡量标准,以评估沟通参与情况。从个人音频流(字数,通话时间,话轮转换,关键字配置文件)派生的通信为基础的功能将通过音频日记每个学生的基础上提取。接下来,这个信息流将被用来开发基于类的组动态。这个解决方案代表了一种方法,让教师监测学生参与随着时间的推移在科学活动领域,帮助教师确定谁是不口头参与科学话语的学生,并迅速评估课堂实践的变化,以改善学习的影响。许多技术挑战将被解决的自动音频流为基础的语音处理的自然音频数据使用语音活动检测,说话人日记的基础上机器学习模型,和关键字定位科学主题识别和跟踪。这些研究目标将在课堂环境中进行评估,教师对所产生的解决方案的有效性进行反馈。由此产生的语音技术进步将为未来的智能教室提供新的机会,以便教师更好地评估学生对科学的参与,而不是不频繁的传统标准化测试。 最终,这一努力将为教师提供工具,以识别和经常监测科学学习脱离的早期指标,并有可能提高代表性不足的学生群体的科学兴趣,并进一步使STEM劳动力多样化。该奖项反映了NSF的法定使命,并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。
项目成果
期刊论文数量(22)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Child vs Adult Speaker Diarization of naturalistic audio recordings in preschool environment using Deep Neural Networks
使用深度神经网络对学前环境中的自然录音进行儿童与成人说话者的分类
- DOI:
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Kothalkar, P.;Hansen, J.H.L.;Buzhardt, J.;Irvin, D.;Rous, B.
- 通讯作者:Rous, B.
Block-Based High Performance CNN Architectures for Frame-Level Overlapping Speech Detection
- DOI:10.1109/taslp.2020.3036237
- 发表时间:2021-01-01
- 期刊:
- 影响因子:5.4
- 作者:Yousefi, Midia;Hansen, John H. L.
- 通讯作者:Hansen, John H. L.
Domain Expansion for End-to-End Speech Recognition: Applications for Accent/Dialect Speech
- DOI:10.1109/taslp.2022.3233238
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Shahram Ghorbani;J. Hansen
- 通讯作者:Shahram Ghorbani;J. Hansen
Can Smartphones be a cost-effective alternative to LENA for Early Childhood Language Intervention?
- DOI:10.21437/s4sg.2022-3
- 发表时间:2022-09
- 期刊:
- 影响因子:0
- 作者:Satwik Dutta;Jacob C. Reyna;J. Buzhardt;Dwight W. Irvin;John H. L. Hansen
- 通讯作者:Satwik Dutta;Jacob C. Reyna;J. Buzhardt;Dwight W. Irvin;John H. L. Hansen
Quantifying Engagement in Preschool Classrooms - Conversational Turn-Taking & Topic Initiations
量化学前班课堂的参与度 - 对话轮流
- DOI:10.18260/1-2--39202
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Sarah A. Tao;Satwik Dutta;Yagmur Seven;Dwight Irvin;J. Buzhardt;John H. L. Hansen
- 通讯作者:John H. L. Hansen
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
John Hansen其他文献
Springer Publisher
施普林格出版社
- DOI:
- 发表时间:
2004 - 期刊:
- 影响因子:0
- 作者:
Huseyin Abut;John Hansen;Kazuya Takeda (Eds.) - 通讯作者:
Kazuya Takeda (Eds.)
An energy and power-aware approach to high-level synthesis of asynchronous systems
用于异步系统高级综合的能量和功率感知方法
- DOI:
10.1109/iccad.2010.5654169 - 发表时间:
2010 - 期刊:
- 影响因子:0
- 作者:
John Hansen;Montek Singh - 通讯作者:
Montek Singh
Pedometer Use as Motivation for Physical Activity in Cardiac Tele-Rehabilitation
在心脏远程康复中使用计步器作为身体活动的动力
- DOI:
10.5334/ijic.2288 - 发表时间:
2015 - 期刊:
- 影响因子:2.4
- 作者:
C. Thorup;Mette Grønkjær;H. Spindler;J. Andreasen;John Hansen;B. Dinesen;Gitte Nielsen;E. E. Sørensen - 通讯作者:
E. E. Sørensen
Characterization of Glutamate Dehydrogenase from the Ammonia-oxidizing Chemoautotroph <em>Nitrosomonas europaea</em>
- DOI:
10.1016/s0021-9258(19)81462-x - 发表时间:
1967-01-25 - 期刊:
- 影响因子:
- 作者:
Alan B. Hooper;John Hansen;Roger Bell - 通讯作者:
Roger Bell
Concurrency-Enhancing Transformations for Asynchronous Behavioral Specifications: A Data-Driven Approach
异步行为规范的并发增强转换:数据驱动的方法
- DOI:
- 发表时间:
2008 - 期刊:
- 影响因子:0
- 作者:
John Hansen;Montek Singh - 通讯作者:
Montek Singh
John Hansen的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('John Hansen', 18)}}的其他基金
COLLABORATIVE RESEARCH: Social-Emotional Analysis of the Language Environment (SEAL): Key Word & Phrase Spotting in Early Childhood Care Settings
合作研究:语言环境的社会情感分析 (SEAL):关键词
- 批准号:
2234916 - 财政年份:2023
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
EAGER: Collaborative Research: Second Language Speech Production: Formulation of Objective Speech Intelligibility Measures and Learner-Specific Feedback
EAGER:协作研究:第二语言语音生成:客观语音清晰度测量和学习者特定反馈的制定
- 批准号:
2140415 - 财政年份:2021
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
CCRI: Medium: Developing a Multi-Channel Naturalistic Audio Corpora for the Natural Language Processing Research Community
CCRI:Medium:为自然语言处理研究界开发多通道自然音频语料库
- 批准号:
2016725 - 财政年份:2020
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
Workshops on NASA Apollo Mission Audio as a Community Research Resource
将 NASA 阿波罗任务音频作为社区研究资源的研讨会
- 批准号:
1943365 - 财政年份:2019
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
Doctoral Dissertation Research in DRMS: The Consumer Logic of Anti-Government Antagonism
DRMS博士论文研究:反政府对抗的消费者逻辑
- 批准号:
1357620 - 财政年份:2014
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
RI: Small: Collaborative Research: 'Houston We Have A Solution': Novel Speech Processing Advancements for Analysis of Large Asynchronous Multi-Channel Audio Corpora
RI:小型:协作研究:“休斯顿,我们有一个解决方案”:用于分析大型异步多通道音频语料库的新颖语音处理进步
- 批准号:
1219130 - 财政年份:2012
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
Collaborative Research: SBE Alliance: Great Lakes Alliance for the Social and Behavioral Sciences (GLASS)
合作研究:SBE 联盟:五大湖社会和行为科学联盟 (GLASS)
- 批准号:
0750599 - 财政年份:2007
- 资助金额:
$ 45万 - 项目类别:
Continuing Grant
Collaborative Research: Primary Elections for U.S. State and Federal Offices: A Comprehensive Database and Analysis
合作研究:美国州和联邦办公室初选:综合数据库和分析
- 批准号:
0617555 - 财政年份:2006
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
SBE Collaborative Research: Northwestern AGEP for SBE
SBE 合作研究:西北大学 AGEP for SBE
- 批准号:
0549069 - 财政年份:2005
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
Interactions of the TCR co-receptors and p56LCK in an Ectothermic Model
变温模型中 TCR 共受体和 p56LCK 的相互作用
- 批准号:
0453924 - 财政年份:2004
- 资助金额:
$ 45万 - 项目类别:
Continuing Grant
相似国自然基金
Research on Quantum Field Theory without a Lagrangian Description
- 批准号:24ZR1403900
- 批准年份:2024
- 资助金额:0.0 万元
- 项目类别:省市级项目
Cell Research
- 批准号:31224802
- 批准年份:2012
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research
- 批准号:31024804
- 批准年份:2010
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research (细胞研究)
- 批准号:30824808
- 批准年份:2008
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
- 批准号:10774081
- 批准年份:2007
- 资助金额:45.0 万元
- 项目类别:面上项目
相似海外基金
Collaborative Research: REU Site: Earth and Planetary Science and Astrophysics REU at the American Museum of Natural History in Collaboration with the City University of New York
合作研究:REU 地点:地球与行星科学和天体物理学 REU 与纽约市立大学合作,位于美国自然历史博物馆
- 批准号:
2348998 - 财政年份:2025
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
Collaborative Research: REU Site: Earth and Planetary Science and Astrophysics REU at the American Museum of Natural History in Collaboration with the City University of New York
合作研究:REU 地点:地球与行星科学和天体物理学 REU 与纽约市立大学合作,位于美国自然历史博物馆
- 批准号:
2348999 - 财政年份:2025
- 资助金额:
$ 45万 - 项目类别:
Standard Grant
"Small performances": investigating the typographic punches of John Baskerville (1707-75) through heritage science and practice-based research
“小型表演”:通过遗产科学和基于实践的研究调查约翰·巴斯克维尔(1707-75)的印刷拳头
- 批准号:
AH/X011747/1 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Research Grant
Democratizing HIV science beyond community-based research
将艾滋病毒科学民主化,超越社区研究
- 批准号:
502555 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Translational Design: Product Development for Research Commercialisation
转化设计:研究商业化的产品开发
- 批准号:
DE240100161 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Discovery Early Career Researcher Award
Understanding the experiences of UK-based peer/community-based researchers navigating co-production within academically-led health research.
了解英国同行/社区研究人员在学术主导的健康研究中进行联合生产的经验。
- 批准号:
2902365 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Studentship
XMaS: The National Material Science Beamline Research Facility at the ESRF
XMaS:ESRF 的国家材料科学光束线研究设施
- 批准号:
EP/Y031962/1 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Research Grant
FCEO-UKRI Senior Research Fellowship - conflict
FCEO-UKRI 高级研究奖学金 - 冲突
- 批准号:
EP/Y033124/1 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Research Grant
UKRI FCDO Senior Research Fellowships (Non-ODA): Critical minerals and supply chains
UKRI FCDO 高级研究奖学金(非官方发展援助):关键矿产和供应链
- 批准号:
EP/Y033183/1 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Research Grant
TARGET Mineral Resources - Training And Research Group for Energy Transition Mineral Resources
TARGET 矿产资源 - 能源转型矿产资源培训与研究小组
- 批准号:
NE/Y005457/1 - 财政年份:2024
- 资助金额:
$ 45万 - 项目类别:
Training Grant