EAGER: Exploring the Feasibility of Phoneme Sound Origins to Enhance Mobile Authentication

使用移动设备来验证个人的身份，无论是为了访问设备本身，还是作为验证对附近其他设备的访问的平台，都是构建安全和私人计算系统时需要解决的一个重要问题。这项提议旨在通过开发人们声道的物理模型来改进语音识别作为一种身份验证工具，这种模型对个人发出声音的方式产生独特的影响。这些新颖的生物特征将使用许多移动设备上存在的各种传感器来捕获和推断，并研究它们在唯一识别个人和实际应用方面的潜力。这项工作还包括系统地研究设备特征、用户行为和用户身体状态(例如，感冒)的差异如何影响语音产生和建模并将其用作生物识别的能力，以及推断语音特征的方法如何解释这些差异。这项工作将导致对发声科学的科学贡献，以及关于利用物理系统的独特特征、在认证方面的潜在实际应用以及支持本科生和研究生教育的机会等更一般的问题。这项拟议的研究展示了如何探索人类生理和移动感知来增强移动身份验证。围绕物理声音产生建模的工作将专注于为人类声道中不同位置的不同声音建模音素声音来源。个体生理上的这些差异类似于使用计算设备的物理特征的微小变化来为每个设备生成唯一的基于硬件的签名的类似想法。它们将通过信号处理算法进行感知，这些算法利用从多个麦克风捕获声音的时间差，并通过不同大小的数据集的错误率来单独评估身份验证质量，并与其他生物特征一起进行评估。下一阶段的工作将研究捕获的背景如何影响生物识别质量。这些包括各种设备的麦克风放置和音频芯片组和采样率；人的握力和与设备的交互以及设备与嘴巴的相对位置，以及他们的姿势和动作；以及他们的生理(即疾病)和心理状态的方面(通过通过视频诱导情感的标准技术)。最后，为了解决跨背景调整模型的问题，项目团队将开发技术来感知姿势、距离和情绪状态，并评估利用统计学习方法的潜力，这些方法非常适合相对数据值而不是绝对数据值，如相关分析和高斯混合模型来解决这些背景变化。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Using mobile devices to authenticate a person's identity, both for access to the device itself and as a platform for verifying access to other nearby devices, is an important problem to address in building secure and private computing systems. This proposal seeks to improve voice recognition as an authentication tool by developing physical models of people's vocal tracts that uniquely affect how individual people produce sounds. These novel biometric traits will be captured and inferred using a variety of sensors that are present on many mobile devices, and studied for their potential to both uniquely identify individuals and be practically used in real contexts. The work also includes systematic studies of how differences in device characteristics, user behavior, and users' physical state (for instance, having a cold) affect both voice production and the ability to model it and use it as a biometric identifier, and how methods for inferring voice characteristics can account for these differences. The work will lead to scientific contributions to both the science of voice production and more general questions about leveraging unique characteristics of physical systems, potential practical applications in authentication, and opportunities to support both undergraduate and graduate education. The proposed research demonstrates how human physiology and mobile sensing can be explored to enhance mobile authentication. The work around modeling physical voice production will focus on modeling the phoneme sound origin for different sounds from different places in the human vocal tract. These differences in individual physiology are analogous to similar ideas that use small variations in the physical characteristics of computing devices to generate a unique hardware-based signature for each device. They will be sensed through signal processing algorithms that leverage time differences in sound capture from multiple microphones and be evaluated both individually and in combination with other biometric features on the quality of authentication as measured by error rates across datasets of different sizes. The next phase of the work will examine how the context of capture affects the biometric quality. These include the microphone placement and audio chipset and sampling rates of a variety of devices; aspects of a person's grip and interaction with the device and its relative location to their mouth, as well as their posture and motion; and aspects of their physiological (i.e., sickness) and psychological state (through standard techniques for eliciting emotion through video). Finally, to address the problem of adapting models across contexts, the project team will develop techniques to sense pose, distance, and emotional state, as well as evaluate the potential to leverage statistical learning methods well-suited to relative rather than absolute data values such as correlation analysis and Gaussian Mixture Models to address these contextual variations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.