Large-Scale Discriminative Modelling for Data-Intensive Speech and Language Processing

随着无线互联网和智能手机在我们的日常生活中越来越普遍，许多服务提供商使用所谓的云计算框架为客户提供各种类型的语音和语言相关服务。在这种背景下，这些提供商可以非常容易和快速地在其中央服务器中积累大量真实世界的数据。因此，在这个前所未有的大数据时代，如何利用海量的可用数据，进一步将许多现实应用的性能提升到一个新的水平，已经成为一个非常有趣的研究课题。在本研究计划中，我们将在数据密集型语音和语言应用的背景下研究这个问题，如自动语音识别、文本分类和口语处理。我们首先将研究重点放在研究新的判别模型上，这些模型可以有效地对大量训练数据中最重要和最相关的信息进行建模，这些数据可能是嘈杂的和未标记的。同时，我们还将设计高效的学习算法，这些算法足够灵活，可以充分利用当前流行的并行计算设施，如多核cpu、gpu和基于map-reduce的分布式计算平台。更具体地说，在我们之前基于边缘的判别学习研究的基础上，我们将研究三个重要的子主题：1)如何提取语音和语言数据的判别和紧凑特征；Ii)如何在数据密集型语音和语言应用中构建灵活的判别模型，能够处理大量数据；Iii)如何并行化学习算法以有效解决模型估计中的大规模优化问题。我们认为，这些问题对于我们充分受益于许多数据密集型语音和语言应用程序中的大量真实数据至关重要。这一研究项目将有助于推动基于越来越流行的云计算模型的许多加拿大和国际工业IT部门的大规模学习和自动知识发现技术。

As the wireless Internet and smartphones have become more and more prevalent in our daily life, many service providers have used the so-called cloud-computing framework to deliver various types of speech and language related services to customers. Under this background, it is very easy and fast for these providers to accumulate massive amount of real-world data in their central servers. Therefore, it has become a very interesting research topic regarding how to take advantage of the mass data available in this unprecedented big data era to further boost up performance to next level for many real world applications. In this research program, we will study this problem in the context of data-intensive speech and language applications, such as automatic speech recognition, text categorization and spoken language processing. We will first focus our research on investigating new discriminative models that are effective to model the most important and pertinent information available in the massive training data, which may be noisy and unlabelled. Meanwhile, we will also design efficient learning algorithms that are flexible enough to take full advantage of currently popular parallel computing facilities, such as multi-core CPUs, GPUs and map-reduce based distributed computing platform. More specifically, based on our previous research on margin-based discriminative learning, we will study three important sub-topics in this research program: i) how to extract discriminative and compact features for speech and language data; ii) how to build flexible discriminative models capable of dealing with a vast quantity of data in data-intensive speech and language applications; iii) how to parallelize learning algorithms to efficiently solve large-scale optimization in model estimation. We believe these issues are critical for us to fully benefit from massive real-world data in many data-intensive speech and language applications. This research program will help to advance large-scale learning and automatic knowledge discovery technologies for many Canadian and International industrial IT sections that are based on the more and more popular cloud-computing model.