SenSE: Smart Electropalatography for Linguistic and Medical Applications (SELMA)

语音作为一种非侵入性的生物标志物，可以为研究人员和临床医生提供新的手段，以捕捉与正常人群中的语言现象或个体的语音功能变化相关的语音清晰度模式的细微变化。该项目的总体目标是建立一种基于证据的，量化的，数据驱动的，非侵入性的方法，使用语音作为生物标志物，用于在语言学和生物医学领域内的各种神经运动障碍中轻松检测跨语言变化和发音变化模式。这个跨学科的项目有三个互动的手臂：智能，无线，electropalatography（EPG）系统的开发，与现有的EPG系统进行比较的行为语音数据收集，以及用于识别舌腭接触模式的机器学习，这意味着异常的发音模式。这项工作将导致建立一个新的技术语言学和技术语言病理学学科和相关课程。SELMA的表现将通过新课程，课堂讲座，会议演示，出版物和大众/社交媒体传播。私人侦探志愿参与少数民族项目。使用语音作为生物标志物提供了有利的功能，如非接触式检测和远程监控。然而，单独的声学签名不足以为语言和临床使用提供可靠的感测。在这个项目中，一个独特的实时，多模态，无线智能electropalatography（EPG）系统的语言和医疗应用（SELMA）的设计和实施，以提供额外的口腔传感模式，包括精确的位置，大小，压力，和舌头与腭在说话时接触的持续时间。该系统的主要装置--伪腭的厚度是典型伪腭的1/20，从而最大限度地减少了语音产生期间的气流阻塞，并与腭相适应，减少了佩戴者的不适，并在语音清晰度期间提供了接近正常的口腔条件。使用无线充电和通信，提高了可靠性和安全性。通过行为语言学实验评估SELMA设备的优势，以评估检测一种常见的跨语言的语音过程，称为spirantization，这是一种涉及语音清晰度降低的过程。机器学习模型，整合发音（即，EPG）和声学数据被用于预测轻发音程度和主观轻发音等级。通过使用来自PD患者、醉酒说话者和第二语言口音说话者的非典型语音的大数据来微调潜在语音模式的模型，SELMA系统将为医学和语言学目的提供一种独特的诊断形式。在讲话期间舌头接触的高分辨率时空信息的使用可以扩展用于容易地监测神经系统疾病，诸如PD、肌萎缩性侧索硬化（ALS）、创伤性脑损伤和其他神经系统疾病。此外，SELMA将是廉价和足够小，可用于监测治疗进展的儿科构音障碍影响可懂度，如腭裂，言语失用症，唐氏综合征。SELMA的未来变化可能会将其效用扩展到其他疾病监测与额外的传感器集成到设备中，可能允许使用唾液生物标志物传感器监测癌症，艾滋病，吸烟和性传播疾病。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Speech as a non-invasive biomarker could provide researchers and clinicians with new means to capture fine changes in speech articulation patterns associated with linguistic phenomena in the normal population or functional changes in articulation in individuals with disorders. The overarching goal of this project is to establish an evidence-based, quantified, data driven, non-invasive method for using speech as a biomarker for facile detection of cross-linguistic variation and patterns of articulatory change in various neuromotor disorders within the linguistics and biomedical realms. This interdisciplinary project has three interactive arms: the development of a smart, wireless, electropalatography (EPG) system, behavioral speech data collection for comparison with existing EPG systems, and machine learning for identifying patterns of tongue-palate contact that signify abnormal patterns of articulation. This work would lead to establishing a new techno-linguistic and techno-speech pathology disciplines and related curricula. SELMA performance will be disseminated through new courses, class lectures, conference presentations, publications, and mass/social media. PIs volunteer for minority programs.The usage of speech sounds as a biomarker offers advantageous features such as non-contact detection and remote monitoring. However, the acoustic signature alone is not sufficient to provide reliable sensing for linguistic and clinical use. In this project, a unique real-time, multi-modal, wireless smart electropalatography (EPG) system for linguistic and medical applications (SELMA) is devised and implemented to provide additional oral sensing modalities, including precise position, size, pressure, and duration of tongue contacts with the palate in speaking. A pseudo-palate, the main apparatus of the system, is 1/20 the thickness of a typical pseudo-palate, minimizing the blockage of airflow during speech production and conform to the palate, reducing the discomfort of the wearer and offering a close-to-normal mouth condition during speech articulation. Wireless charging and communication are used increasing reliability and safety. The advantage of the SELMA device is evaluated via a behavioral linguistic experiment to assess detection of a common class of cross-linguistic, phonological processes known as spirantization, a process involving decreased precision of speech articulation. Machine learning models, integrating articulatory (i.e., EPG) and acoustic data, are used to predict degrees of lenition and subjective lenition ratings. By fine-tuning the models on latent speech patterns with big data of atypical speech from PD patients, intoxicated speakers and second-language accented speakers, the SELMA system will provide a unique form of diagnostics for medical and linguistic purposes. The usage of high resolution spatio-temporal information of tongue contacts during speech can be extended for facile monitoring of neurological diseases such as PD, amyotrophic lateral sclerosis (ALS), traumatic brain injuries, and other neurological disorders. Further, SELMA will be inexpensive and small enough that can be used for monitoring treatment progress in pediatric articulation disorders affecting intelligibility, such as cleft palate, apraxia of speech, and Down syndrome. Future variations of SELMA may extend its utility to other disease monitoring with additional sensors integrated into the device, potentially allowing monitoring of cancer, HIV, smoking and sexually transmitted diseases using salivary biomarker sensors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.