Augmented speech communication using multi-modal signals with real-time, low-latency voice conversion

使用具有实时、低延迟语音转换的多模信号的增强语音通信

基本信息

  • 批准号:
    22KJ1519
  • 负责人:
  • 金额:
    $ 1.41万
  • 依托单位:
  • 依托单位国家:
    日本
  • 项目类别:
    Grant-in-Aid for JSPS Fellows
  • 财政年份:
    2023
  • 资助国家:
    日本
  • 起止时间:
    2023-03-08 至 2024-03-31
  • 项目状态:
    已结题

项目摘要

The purpose of this research is to apply voice conversion (VC) to realize an interactive speech production paradigm for real-world applications, with the help of multimodal signals and real-time processing techniques. In the second year, the applicant focused on three aspects.(1) Continued improvement on fundamental VC techniques, specifically self-supervised speech representation (S3R)-based VC, an emerging trend which reduces training data requirements. The applicant kept on updating S3PRL-VC, an open-source toolkit for researchers to evaluate S3R models for VC, and published the latest experimental results in the IEEE Journal of Selected Topics in Signal Processing.(2) Foreign accent conversion, a task that helps reduce foreign accents for efficient communication. A paper that provides an unified evaluation of current approaches and identifies unsolved problems is submitted to an international conference and currently under review.(3) Singing voice conversion, a fundamental technique that has the potential to augment the communication ability of human. The applicant is running a scientific event named the Singing Voice Conversion Challenge 2023, which aims to provide an unified experimental setting including task and dataset, in order to attract researchers world-wide to look into this problem and explore the limitation of the state-of-the-art techniques.
本研究的目的是应用语音转换(VC),以实现一个交互式语音生产的范例,为现实世界的应用程序,与多模态信号和实时处理技术的帮助。第二年,申请人重点关注了三个方面。(1)基本VC技术的持续改进,特别是基于自监督语音表示(S3 R)的VC,这是一种新兴趋势,可降低训练数据要求。申请人不断更新S3 PRL-VC,这是一个供研究人员评估VC的S3 R模型的开源工具包,并在IEEE Journal of Selected Topics in Signal Processing上发表了最新的实验结果。(2)外国口音转换,这是一项有助于减少外国口音以提高沟通效率的任务。向一次国际会议提交了一份文件,对目前的做法进行了统一评价,并查明了尚未解决的问题,目前正在审查该文件。(3)歌唱声转换技术是一项有潜力提高人类交流能力的基础技术。申请人正在举办一项名为“歌唱声音转换挑战2023”的科学活动,旨在提供一个统一的实验设置,包括任务和数据集,以吸引世界各地的研究人员研究这个问题,并探索最先进技术的局限性。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion
  • DOI:
    10.21437/interspeech.2021-208
  • 发表时间:
    2021-06
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Wen-Chin Huang;Kazuhiro Kobayashi;Yu-Huai Peng;Ching-Feng Liu;Yu Tsao;Hsin-Min Wang;T. Toda
  • 通讯作者:
    Wen-Chin Huang;Kazuhiro Kobayashi;Yu-Huai Peng;Ching-Feng Liu;Yu Tsao;Hsin-Min Wang;T. Toda
CRANK: an Open-Source Software for Nonparallel Voice Conversion based on Vetor-Quantized Variational Autoencoder
CRANK:基于矢量量化变分自动编码器的非并行语音转换开源软件
  • DOI:
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Kazuhiro Kobayashi;Wen-Chin Huang;Yi-Chiao Wu;Patrick Tobing;Tomoki Hayashi;and Tomoki Toda
  • 通讯作者:
    and Tomoki Toda
On Prosody Modeling for ASR+TTS Based Voice Conversion
S3PRL-VC: Open-Source Voice Conversion Framework with Self-Supervised Speech Representations
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

HUANG WENCHIN其他文献

HUANG WENCHIN的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

相似海外基金

SCH: Dementia Early Detection for Under-represented Populations via Fair Multimodal Self-Supervised Learning
SCH:通过公平的多模式自我监督学习对代表性不足的人群进行痴呆症早期检测
  • 批准号:
    10816864
  • 财政年份:
    2023
  • 资助金额:
    $ 1.41万
  • 项目类别:
Development of a Sign Language Recognition Engine Using Self-Supervised Learning Methods
使用自我监督学习方法开发手语识别引擎
  • 批准号:
    23K17511
  • 财政年份:
    2023
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Grant-in-Aid for Challenging Research (Exploratory)
Self-Supervised Learning to Improve Transferability of Agricultural Deep Learning Models
自监督学习提高农业深度学习模型的可迁移性
  • 批准号:
    574936-2022
  • 财政年份:
    2022
  • 资助金额:
    $ 1.41万
  • 项目类别:
    University Undergraduate Student Research Awards
RI: Medium: Foundations of Self-Supervised Learning Through the Lens of Probabilistic Generative Models
RI:媒介:通过概率生成模型的视角进行自我监督学习的基础
  • 批准号:
    2211907
  • 财政年份:
    2022
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Standard Grant
Broader Self-Supervised Learning with applications in anomaly detection, tabular data, and visual reinforcement learning
更广泛的自我监督学习在异常检测、表格数据和视觉强化学习中的应用
  • 批准号:
    577169-2022
  • 财政年份:
    2022
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Alliance Grants
One-shot self-supervised learning for high quality 3D shape scanning
用于高质量 3D 形状扫描的一次性自我监督学习
  • 批准号:
    22K17907
  • 财政年份:
    2022
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
A Biologically-plausible Deep Learning framework to model self-supervised learning in the visual cortex
一种生物学上合理的深度学习框架,用于模拟视觉皮层的自我监督学习
  • 批准号:
    566601-2021
  • 财政年份:
    2022
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Vanier Canada Graduate Scholarship Tri-Council - Doctoral 3 years
Measuring Cancer Prognosis with Self-Supervised Learning
通过自我监督学习衡量癌症预后
  • 批准号:
    2766128
  • 财政年份:
    2022
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Studentship
A Biologically-plausible Deep Learning framework to model self-supervised learning in the visual cortex
一种生物学上合理的深度学习框架,用于模拟视觉皮层的自我监督学习
  • 批准号:
    566601-2021
  • 财政年份:
    2021
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Vanier Canada Graduate Scholarship Tri-Council - Doctoral 3 years
Quantum enhanced self supervised learning
量子增强自监督学习
  • 批准号:
    2607531
  • 财政年份:
    2021
  • 资助金额:
    $ 1.41万
  • 项目类别:
    Studentship
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了