SenSE: Smart Electropalatography for Linguistic and Medical Applications (SELMA)

语音作为一种非侵入性的生物标志物，可以为研究人员和临床医生提供新的手段，以捕捉与正常人群的语言现象或疾病患者的发音功能变化相关的语音发音模式的细微变化。该项目的总体目标是建立一种基于证据的、量化的、数据驱动的、非侵入性的方法，将语音作为生物标记物，方便地检测语言学和生物医学领域内各种神经运动障碍的跨语言变异和发音变化模式。这个跨学科的项目有三个互动部门：开发智能、无线、腭电(EPG)系统；收集行为语音数据，与现有的EPG系统进行比较；以及机器学习，用于识别表示异常发音模式的舌腭部接触模式。这项工作将导致建立新的技术语言学和技术言语病理学学科和相关课程。塞尔玛的表演将通过新课程、课堂讲座、会议报告、出版物和大众/社交媒体进行传播。PIS是少数民族节目的志愿者。语音作为生物标志物的使用提供了非接触检测和远程监控等有利特征。然而，仅凭声学特征不足以为语言和临床应用提供可靠的感知。在这个项目中，设计并实现了一个独特的实时、多模式、无线的语言和医疗应用智能腭电(EPG)系统(SELMA)，以提供更多的口腔传感模式，包括说话时舌头与腭部接触的准确位置、大小、压力和持续时间。作为该系统的主要设备，伪腭部的厚度是典型伪腭部的1/20，最大限度地减少了语音产生过程中气流的阻塞，并与腭部保持一致，减少了佩戴者的不适感，并在语音清晰度期间提供了接近正常的口腔状况。使用无线充电和通信，提高了可靠性和安全性。通过一项行为语言学实验来评估Selma设备的优势，以评估对一类常见的跨语言语音过程的检测，这种过程被称为螺旋化，这一过程涉及语音清晰度的降低。机器学习模型，结合发音(即EPG)和声学数据，用于预测晶状体的程度和主观晶状体评级。通过使用帕金森病患者、醉酒患者和带有第二语言口音的人的非典型语音大数据来微调潜在语音模式的模型，SELMA系统将为医疗和语言目的提供一种独特的诊断形式。语音过程中舌头接触的高分辨率时空信息的使用可以扩展到对诸如帕金森病、肌萎缩侧索硬化症(ALS)、创伤性脑损伤和其他神经疾病等神经疾病的便捷监测。此外，SELMA将便宜且足够小，可用于监测影响清晰度的儿科发音障碍的治疗进展，如腭裂、言语失用和唐氏综合症。SELMA的未来变种可能会将其用途扩展到其他疾病监测，并将更多传感器集成到设备中，潜在地允许使用唾液生物标记物传感器监测癌症、艾滋病毒、吸烟和性传播疾病。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Speech as a non-invasive biomarker could provide researchers and clinicians with new means to capture fine changes in speech articulation patterns associated with linguistic phenomena in the normal population or functional changes in articulation in individuals with disorders. The overarching goal of this project is to establish an evidence-based, quantified, data driven, non-invasive method for using speech as a biomarker for facile detection of cross-linguistic variation and patterns of articulatory change in various neuromotor disorders within the linguistics and biomedical realms. This interdisciplinary project has three interactive arms: the development of a smart, wireless, electropalatography (EPG) system, behavioral speech data collection for comparison with existing EPG systems, and machine learning for identifying patterns of tongue-palate contact that signify abnormal patterns of articulation. This work would lead to establishing a new techno-linguistic and techno-speech pathology disciplines and related curricula. SELMA performance will be disseminated through new courses, class lectures, conference presentations, publications, and mass/social media. PIs volunteer for minority programs.The usage of speech sounds as a biomarker offers advantageous features such as non-contact detection and remote monitoring. However, the acoustic signature alone is not sufficient to provide reliable sensing for linguistic and clinical use. In this project, a unique real-time, multi-modal, wireless smart electropalatography (EPG) system for linguistic and medical applications (SELMA) is devised and implemented to provide additional oral sensing modalities, including precise position, size, pressure, and duration of tongue contacts with the palate in speaking. A pseudo-palate, the main apparatus of the system, is 1/20 the thickness of a typical pseudo-palate, minimizing the blockage of airflow during speech production and conform to the palate, reducing the discomfort of the wearer and offering a close-to-normal mouth condition during speech articulation. Wireless charging and communication are used increasing reliability and safety. The advantage of the SELMA device is evaluated via a behavioral linguistic experiment to assess detection of a common class of cross-linguistic, phonological processes known as spirantization, a process involving decreased precision of speech articulation. Machine learning models, integrating articulatory (i.e., EPG) and acoustic data, are used to predict degrees of lenition and subjective lenition ratings. By fine-tuning the models on latent speech patterns with big data of atypical speech from PD patients, intoxicated speakers and second-language accented speakers, the SELMA system will provide a unique form of diagnostics for medical and linguistic purposes. The usage of high resolution spatio-temporal information of tongue contacts during speech can be extended for facile monitoring of neurological diseases such as PD, amyotrophic lateral sclerosis (ALS), traumatic brain injuries, and other neurological disorders. Further, SELMA will be inexpensive and small enough that can be used for monitoring treatment progress in pediatric articulation disorders affecting intelligibility, such as cleft palate, apraxia of speech, and Down syndrome. Future variations of SELMA may extend its utility to other disease monitoring with additional sensors integrated into the device, potentially allowing monitoring of cancer, HIV, smoking and sexually transmitted diseases using salivary biomarker sensors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.