Large-Scale Discriminative Modelling for Data-Intensive Speech and Language Processing

随着无线互联网和智能手机在我们的日常生活中越来越普遍，许多服务提供商已经使用所谓的云计算框架来向客户提供各种类型的语音和语言相关服务。在这种背景下，这些提供商在其中央服务器上积累海量现实世界数据变得非常容易和快捷。因此，如何利用这个史无前例的大数据时代的海量数据，进一步提升许多现实世界应用的性能，成为一个非常有趣的研究课题。在这个研究项目中，我们将在数据密集型语音和语言应用的背景下研究这一问题，例如自动语音识别、文本分类和口语处理。我们将首先将研究重点放在研究新的判别模型上，这些模型可以有效地对海量训练数据中最重要和最相关的信息进行建模，这些数据可能是噪声和未标记的。同时，我们还将设计高效的学习算法，这些算法足够灵活，可以充分利用目前流行的并行计算设施，如多核CPU、GPU和基于MAP-REDUTE的分布式计算平台。更具体地说，在前人对基于边界的判别学习研究的基础上，我们将研究三个重要的子课题：i)如何提取语音和语言数据的区分特征和紧凑特征；ii)如何在数据密集型语音和语言应用中建立灵活的能够处理海量数据的判别模型；iii)如何将学习算法并行化以有效地解决模型估计中的大规模优化问题。我们认为，这些问题对于我们在许多数据密集型语音和语言应用程序中充分受益于海量真实世界数据至关重要。这项研究计划将有助于推动加拿大和国际许多工业IT部门基于越来越流行的云计算模型的大规模学习和自动知识发现技术。

As the wireless Internet and smartphones have become more and more prevalent in our daily life, many service providers have used the so-called cloud-computing framework to deliver various types of speech and language related services to customers. Under this background, it is very easy and fast for these providers to accumulate massive amount of real-world data in their central servers. Therefore, it has become a very interesting research topic regarding how to take advantage of the mass data available in this unprecedented big data era to further boost up performance to next level for many real world applications. In this research program, we will study this problem in the context of data-intensive speech and language applications, such as automatic speech recognition, text categorization and spoken language processing. We will first focus our research on investigating new discriminative models that are effective to model the most important and pertinent information available in the massive training data, which may be noisy and unlabelled. Meanwhile, we will also design efficient learning algorithms that are flexible enough to take full advantage of currently popular parallel computing facilities, such as multi-core CPUs, GPUs and map-reduce based distributed computing platform. More specifically, based on our previous research on margin-based discriminative learning, we will study three important sub-topics in this research program: i) how to extract discriminative and compact features for speech and language data; ii) how to build flexible discriminative models capable of dealing with a vast quantity of data in data-intensive speech and language applications; iii) how to parallelize learning algorithms to efficiently solve large-scale optimization in model estimation. We believe these issues are critical for us to fully benefit from massive real-world data in many data-intensive speech and language applications. This research program will help to advance large-scale learning and automatic knowledge discovery technologies for many Canadian and International industrial IT sections that are based on the more and more popular cloud-computing model.