CAREER: Enabling Trustworthy Speech Technologies for Mental Health Care: From Speech Anonymization to Fair Human-centered Machine Intelligence

职业：为心理健康护理提供值得信赖的语音技术：从语音匿名化到公平的以人为本的机器智能

• 批准号: 2046118
• 负责人: Theodora Chaspari
• 金额: $ 60万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2046118&HistoricalAwards=false
• 关键词: CAREER; Enabling; Trustworthy; Speech; Technologies

Speech-based technologies have been heralded as promising solutions to overcome the limitations of existing clinical modalities related to limited healthcare access, non-naturalistic in-clinic interactions, and social stigma. Speech measures combined with artificial intelligence can serve as valuable biomarkers for mental health conditions, such as depression and post-traumatic stress disorder. Yet, in order for artificial intelligence to truly succeed in a future-of-work landscape in which clinicians will be expected to work side-by-side with artificial intelligence systems, both clinicians and patients need to calibrate their trust in the algorithms that power this decision-making process. The goal of this project is to design reliable machine learning, notably for speech-based diagnosis and monitoring of mental health, for addressing three pillars of trustworthiness: explainability, privacy preservation, and fair decision making. Trustworthiness is critical for both patients and clinicians: patients must be treated fairly and without the risk of reidentification, while clinical decision-making needs to rely on explainable and unbiased machine learning. This research program further provides a fertile ground for training high school and college students providing them with the knowledge about (and inclination toward) ethically applying computing research in sensitive populations. The tangible applications developed as part of this research serve as a vehicle to encourage students to pursue careers in Science, Technology, Engineering, and Mathematics, and prepare them to work in transdisciplinary settings for solving real-world problems.This project seeks to design explainable, anonymized, and fair speech biomarkers for mental health, integrating aspects of speech acquisition, transparent modeling, and unbiased decision making. The work is divided into three technical objectives. The first objective designs novel speaker anonymization algorithms that retain mental health information and suppress information related to the identity of the speaker. The anonymization algorithms learn a mapping between the original speech and a latent space, which embeds information about speaker identity, mental health, and phonological sequence through deterministic and probabilistic operations. The second objective improves explainability of speech-based models for tracking mental health through novel convolutional architectures that learn explainable spectrotemporal transformations relevant to speech production fundamentals. The third objective examines how bias in data and model design may perpetuate social disparities in mental health, and designs new machine learning to mitigate unwanted bias in speech-based mental health diagnosis. Through a series of experiments this work further contributes to understanding ways in which human-machine partnerships are formed in mental healthcare settings along dimensions of trust formation, maintenance, and repair.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

基于语音的技术已被誉为有希望的解决方案，以克服现有临床模式的局限性，这些限制与有限的医疗保健机会、非自然主义的临床互动和社会耻辱有关。语音测量与人工智能相结合，可以作为精神健康状况的有价值的生物标记物，如抑郁症和创伤后应激障碍。然而，为了让人工智能在未来的工作环境中真正取得成功，临床医生将被期望与人工智能系统并肩工作，临床医生和患者都需要调整他们对推动这一决策过程的算法的信任。该项目的目标是设计可靠的机器学习，特别是基于语音的诊断和心理健康监测，以解决可信度的三大支柱：可解释性、隐私保护和公平决策。可信度对患者和临床医生都至关重要：患者必须得到公平对待，没有重新识别的风险，而临床决策需要依赖于可解释和不偏不倚的机器学习。这项研究计划还为培训高中生和大学生提供了肥沃的土壤，为他们提供了在敏感人群中应用计算研究的伦理知识(和倾向)。作为这项研究的一部分，开发的有形应用程序作为一种工具，鼓励学生在科学、技术、工程和数学领域追求职业生涯，并为他们在跨学科环境中解决现实世界的问题做好准备。该项目旨在设计可解释的、匿名的、公平的心理健康语音生物标记物，整合语音获取、透明建模和公正决策的各个方面。这项工作分为三个技术目标。第一个目标是设计新颖的说话人匿名算法，该算法保留了说话人的心理健康信息，并抑制了与说话人身份相关的信息。匿名化算法学习原始语音和潜在空间之间的映射，潜在空间通过确定性和概率操作嵌入关于说话人身份、心理健康和语音序列的信息。第二个目标是通过学习与语音产生基础相关的可解释的谱时间变换的新颖卷积结构来提高基于语音的跟踪心理健康的模型的可解释性。第三个目标是研究数据和模型设计中的偏差如何延续心理健康方面的社会差异，并设计新的机器学习来减轻基于语音的心理健康诊断中不必要的偏差。通过一系列实验，这项工作进一步有助于了解在精神卫生保健环境中，如何沿着信任形成、维护和修复的维度形成人机合作伙伴关系。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

A Knowledge-Driven Vowel-Based Approach of Depression Classification from Speech Using Data Augmentation

使用数据增强从语音进行抑郁症分类的知识驱动的基于元音的方法

• DOI: 10.1109/icassp49357.2023.10096105
• 发表时间: 2023
• 期刊: Speech and Signal Processing (ICASSP
• 作者: Feng, Kexin;Chaspari, Theodora
• 通讯作者: Chaspari, Theodora

Preserving Mental Health Information in Speech Anonymization

在语音匿名化中保留心理健康信息

• DOI: 10.1109/aciiw57231.2022.10086012
• 发表时间: 2022
• 期刊: 2022 10th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW
• 作者: Ravuri, Vinesh;Gutierrez-Osuna, Ricardo;Chaspari, Theodora
• 通讯作者: Chaspari, Theodora

Investigating Trust in Human-Machine Learning Collaboration: A Pilot Study on Estimating Public Anxiety from Speech

调查人机学习协作中的信任：从语音估计公众焦虑的试点研究

• DOI: 10.1145/3462244.3479926
• 发表时间: 2021
• 期刊: 23rd ACM International Conference on Multimodal Interaction (ICMI 2021
• 作者: Tutul, Abdullah Aman;Nirjhar, Ehsanul Haque;Chaspari, Theodora
• 通讯作者: Chaspari, Theodora

Toward Knowledge-Driven Speech-Based Models of Depression: Leveraging Spectrotemporal Variations in Speech Vowels

• DOI: 10.1109/bhi56158.2022.9926939
• 发表时间: 2022-09
• 期刊: 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI)
• 作者: Kexin Feng;Theodora Chaspari

An Engineering View on Emotions and Speech: From Analysis and Predictive Models to Responsible Human-Centered Applications

• DOI: 10.1109/jproc.2023.3276209
• 发表时间: 2023-10
• 期刊: Proceedings of the IEEE
• 影响因子: 20.6
• 作者: Shrikanth S. Narayanan