RI:Small: Nonlinear signal representations for speech applications

RI:Small：语音应用的非线性信号表示

• 批准号: 1816165
• 负责人: Najim Dehak
• 金额: $ 33.68万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816165&HistoricalAwards=false
• 关键词: RI; Small; Nonlinear; signal; representations

Human speech is a very rich signal. In addition to words, it contains several kinds of important information about the speaker such as identity, gender, age, native language, dialect, and emotion. It also provides information about the transmission channel and environment; for example, whether the speech came from a phone call or a high-fidelity recording, and whether or not there was background noise. This project aims to create a powerful uniform representation that reflects all the information carried by speech. Such representation would enable important speech applications in multiple sectors of society: commercial (security, healthcare, user interfaces), government (security, information filtering), and law enforcement (speaker identification, forensics).In this project, Johns Hopkins University researchers, who invented the original i-vector framework, intend to progress beyond the linear i-vector approach by investigating non-linear models with the expectation to better explain the complex structure of speech. To achieve this goal, two different models are investigated. First, a non-linear i-vector version is explored. In this method, the speech signal distribution is modeled by a Gaussian mixture model (GMM). The super-vector formed by the GMM means is a non-linear function (neural network) of a latent variable (speech representation). The parameters of the neural network and the latent representation can be jointly estimated by stochastic gradient descent iterations. Secondly, the team intends to investigate different types of auto-encoder networks (AE, VAE, RBM, DBM) to obtain representations from their hidden layers. Preliminary research shows that it is feasible to obtain good representations by combining activations from several hidden layers. Visualization tools are used to understand how the speech data have been represented and structured. By understanding the non-linear relationships created via the auto-encoder network modeling and using the visualization tools, there is potential to produce valuable insights into speech modeling. These insights can help cognitive science and neuroscience researchers to understand how the brain represents speech signals. The proposed methods are developed as software that takes a speech segment as input and generates a single vector that may be used to characterize the segment for the important applications mentioned in the previous paragraph.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类语音是一种非常丰富的信号。除了文字之外，它还包含说话者的身份、性别、年龄、母语、方言、情感等多种重要信息。它还提供有关传输通道和环境的信息；例如，语音是否来自电话或高保真录音，以及是否有背景噪音。 该项目旨在创建一个强大的统一表示，反映语音携带的所有信息。这种表示将使社会多个领域的重要语音应用成为可能：商业（安全、医疗保健、用户界面）、政府（安全、信息过滤）和执法（说话人识别、取证）。 在这个项目中，发明了原始 i-向量框架的约翰·霍普金斯大学研究人员打算通过研究非线性模型来超越线性 i-向量方法，以期更好地解释语音的复杂结构。为了实现这一目标，研究了两种不同的模型。首先，探索非线性 i 向量版本。在该方法中，语音信号分布通过高斯混合模型（GMM）进行建模。由 GMM 均值形成的超向量是潜在变量（语音表示）的非线性函数（神经网络）。神经网络的参数和潜在表示可以通过随机梯度下降迭代来联合估计。其次，该团队打算研究不同类型的自动编码器网络（AE、VAE、RBM、DBM），以从其隐藏层获取表示。初步研究表明，通过组合多个隐藏层的激活来获得良好的表示是可行的。可视化工具用于理解语音数据是如何表示和结构化的。通过理解通过自动编码器网络建模创建的非线性关系并使用可视化工具，有可能对语音建模产生有价值的见解。这些见解可以帮助认知科学和神经科学研究人员了解大脑如何表示语音信号。所提出的方法被开发为软件，该软件将语音片段作为输入并生成单个向量，该向量可用于表征前一段中提到的重要应用的片段。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。