EAGER: Exploring the Feasibility of Phoneme Sound Origins to Enhance Mobile Authentication

使用移动设备来验证一个人的身份，既可以访问设备本身，也可以作为验证访问附近其他设备的平台，这是构建安全和私有计算系统需要解决的一个重要问题。该提案旨在通过开发人类声道的物理模型来改进声音识别，使其成为一种认证工具，这种模型可以独特地影响个体如何发出声音。这些新的生物特征将被捕获，并推断使用各种传感器存在于许多移动设备，并研究其潜力，以唯一识别个人，并在实际环境中实际使用。这项工作还包括系统地研究设备特征、用户行为和用户身体状态（例如，感冒）的差异如何影响声音产生和建模能力，并将其用作生物识别标识符，以及推断声音特征的方法如何解释这些差异。这项工作将为声音产生科学和更多关于利用物理系统独特特性的一般问题、身份验证中的潜在实际应用以及支持本科和研究生教育的机会做出科学贡献。提出的研究展示了如何探索人体生理学和移动传感来增强移动认证。围绕物理声音产生建模的工作将集中于对人类声道中不同位置的不同声音的音素声音起源建模。个体生理上的这些差异类似于使用计算设备物理特性的微小变化来为每个设备生成唯一的基于硬件的签名的类似想法。它们将通过信号处理算法感知，该算法利用来自多个麦克风的声音捕获的时间差，并通过不同大小的数据集的错误率来单独或与其他生物特征结合评估认证质量。下一阶段的工作将研究捕获环境如何影响生物识别质量。这些包括麦克风放置和音频芯片组以及各种设备的采样率；一个人的握持和与设备的交互方面，以及它与嘴的相对位置，以及他们的姿势和运动；以及他们的生理（即疾病）和心理状态的各个方面（通过视频引发情感的标准技术）。最后，为了解决跨上下文调整模型的问题，项目团队将开发感知姿势、距离和情绪状态的技术，并评估利用统计学习方法的潜力，这些方法非常适合于相对而不是绝对数据值，如相关分析和高斯混合模型，以解决这些上下文变化。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Using mobile devices to authenticate a person's identity, both for access to the device itself and as a platform for verifying access to other nearby devices, is an important problem to address in building secure and private computing systems. This proposal seeks to improve voice recognition as an authentication tool by developing physical models of people's vocal tracts that uniquely affect how individual people produce sounds. These novel biometric traits will be captured and inferred using a variety of sensors that are present on many mobile devices, and studied for their potential to both uniquely identify individuals and be practically used in real contexts. The work also includes systematic studies of how differences in device characteristics, user behavior, and users' physical state (for instance, having a cold) affect both voice production and the ability to model it and use it as a biometric identifier, and how methods for inferring voice characteristics can account for these differences. The work will lead to scientific contributions to both the science of voice production and more general questions about leveraging unique characteristics of physical systems, potential practical applications in authentication, and opportunities to support both undergraduate and graduate education. The proposed research demonstrates how human physiology and mobile sensing can be explored to enhance mobile authentication. The work around modeling physical voice production will focus on modeling the phoneme sound origin for different sounds from different places in the human vocal tract. These differences in individual physiology are analogous to similar ideas that use small variations in the physical characteristics of computing devices to generate a unique hardware-based signature for each device. They will be sensed through signal processing algorithms that leverage time differences in sound capture from multiple microphones and be evaluated both individually and in combination with other biometric features on the quality of authentication as measured by error rates across datasets of different sizes. The next phase of the work will examine how the context of capture affects the biometric quality. These include the microphone placement and audio chipset and sampling rates of a variety of devices; aspects of a person's grip and interaction with the device and its relative location to their mouth, as well as their posture and motion; and aspects of their physiological (i.e., sickness) and psychological state (through standard techniques for eliciting emotion through video). Finally, to address the problem of adapting models across contexts, the project team will develop techniques to sense pose, distance, and emotional state, as well as evaluate the potential to leverage statistical learning methods well-suited to relative rather than absolute data values such as correlation analysis and Gaussian Mixture Models to address these contextual variations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.