SBIR Phase I: Low-latency polyphonic coding for interactive immersive applications

这个小企业创新研究（SBIR）第一阶段项目及其基础技术创新的更广泛的影响/商业潜力在于实现涉及交互式和沉浸式媒体的各种应用程序，这些应用程序是未来几年将大幅增长的几个行业的核心。具体而言，该项目开发的低延迟音频技术是开发未来产品的关键推动因素，这些产品对技术行业的几个领域具有重要价值，其中最重要的是，为增强现实游戏和相关应用提供完全沉浸式交互媒体产品。预计在促进和支持互联网上的音乐合作以及实现远程音乐教育方面会产生额外的影响，这两者都具有明确的文化和教育意义。另一个重要的影响是实现了真正现实的电话会议体验，对商业和社交网络都有相当大的影响，后者进一步提供了一个现实的选择，完全互动的社会聚会的团体和家庭，而无需诉诸昂贵的旅行。这个小企业创新研究（SBIR）第一阶段项目开发了一个新的范式编码和网络的复调音频内容在低，通过有效的预测来减少延迟，这对于交互式沉浸式超现实多媒体的新兴领域中的许多应用至关重要。复调音频或多个周期性分量加上噪声的混合物长期以来一直抵抗有效的预测，从而迫使现有技术的编码器要么采用长变换，其引起相当大的延迟并且与需要低延迟、低复杂度和低比特率的应用不兼容，要么接受显著降低的性能。该项目通过有效利用复调音频信号的所有周期性分量中的时间冗余，开发出接近最佳性能的技术，尽管存在延迟，复杂性和比特率的限制。具体而言，编码范例建立在级联长期预测的新技术上，该技术能够以低延迟联合预测混合物中的所有周期性分量。这种预测方法是通过开发功能强大的低复杂度参数估计技术来补充的，以最大限度地减少资源需求，有效地适应基频变化，边信息优化以最大限度地减少比特率成本，实际重新设计所有编码器模块以充分利用预测能力，以及增强有损分组网络上的流传输的容错能力。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project, and of its underlying technological innovation, is in enabling a diverse variety of applications involving interactive and immersive media, which are central to several sectors that are poised to grow substantially in the coming years. Specifically, the low-latency audio technology developed in the project is a critical enabler for the development of future products that are of significant value to several sectors of the technology industry including, most importantly, the enabling of fully immersive interactive media products for augmented reality games and related applications. Additional impact is expected in the advancement and support of musical collaboration over the internet and enabling remote music education, both with clear cultural and educational implications. Another significant impact is in enabling a truly realistic teleconferencing experience with considerable implications for both business and social networks, with the latter further providing a realistic alternative to fully interactive social gatherings of groups and families without recourse to costly travel.This Small Business Innovation Research (SBIR) Phase I project develops a novel paradigm for coding and networking of polyphonic audio content at low-latency via efficient prediction, which is critical to numerous applications in the emerging field of interactive immersive hyper-realistic multimedia. Polyphonic audio, or the mixture of multiple periodic components plus noise, has long resisted effective prediction, thus forcing state-of-the-art coders to either employ long transformation that incurs substantial delay and is incompatible with applications requiring low latency, low complexity and low bitrate, or accept significantly degraded performance. This project develops technologies that approach optimal performance despite constraints on latency, complexity and bit rate, by effectively exploiting temporal redundancies in all periodic components of polyphonic audio signals. Specifically, the coding paradigm builds on the novel technique of cascaded long term prediction, which enables joint prediction of all periodic components in the mixture, at low delay. This prediction approach is complemented by the development of powerful low-complexity parameter estimation techniques to minimize resource requirements, effective adaptation to fundamental frequency changes, side information optimization to minimize bitrate costs, practical redesign of all coder modules to fully exploit the prediction capabilities, and enhanced error-resilience for streaming over lossy packet networks.