Large-Scale Discriminative Modelling for Data-Intensive Speech and Language Processing

随着无线互联网和智能手机在我们的日常生活中变得越来越普遍，许多服务提供商已经使用所谓的云计算框架来向客户提供各种类型的语音和语言相关服务。在这种背景下，这些提供商可以非常容易和快速地在他们的中央服务器上积累大量的真实世界数据。因此，如何利用这个前所未有的大数据时代提供的海量数据，进一步将许多真实的应用程序的性能提升到一个新的水平，已经成为一个非常有趣的研究课题。在这个研究项目中，我们将在数据密集型语音和语言应用的背景下研究这个问题，例如自动语音识别，文本分类和口语处理。我们将首先集中研究新的判别模型，这些模型可以有效地对大量训练数据中最重要和最相关的信息进行建模，这些数据可能是嘈杂的和未标记的。同时，我们还将设计高效的学习算法，这些算法足够灵活，可以充分利用当前流行的并行计算设施，如多核CPU，GPU和基于map-reduce的分布式计算平台。更具体地说，基于我们以前对基于边缘的判别学习的研究，我们将在这个研究计划中研究三个重要的子课题：i）如何提取语音和语言数据的判别和紧凑特征; ii）如何建立灵活的判别模型，能够处理数据密集型语音和语言应用中的大量数据; iii）如何并行化学习算法以有效地解决模型估计中的大规模优化。我们相信，这些问题对于我们在许多数据密集型语音和语言应用程序中充分受益于大量真实世界数据至关重要。这项研究计划将有助于推进大规模学习和自动知识发现技术，为许多加拿大和国际工业IT部门的基础上越来越流行的云计算模型。

As the wireless Internet and smartphones have become more and more prevalent in our daily life, many service providers have used the so-called cloud-computing framework to deliver various types of speech and language related services to customers. Under this background, it is very easy and fast for these providers to accumulate massive amount of real-world data in their central servers. Therefore, it has become a very interesting research topic regarding how to take advantage of the mass data available in this unprecedented big data era to further boost up performance to next level for many real world applications. In this research program, we will study this problem in the context of data-intensive speech and language applications, such as automatic speech recognition, text categorization and spoken language processing. We will first focus our research on investigating new discriminative models that are effective to model the most important and pertinent information available in the massive training data, which may be noisy and unlabelled. Meanwhile, we will also design efficient learning algorithms that are flexible enough to take full advantage of currently popular parallel computing facilities, such as multi-core CPUs, GPUs and map-reduce based distributed computing platform. More specifically, based on our previous research on margin-based discriminative learning, we will study three important sub-topics in this research program: i) how to extract discriminative and compact features for speech and language data; ii) how to build flexible discriminative models capable of dealing with a vast quantity of data in data-intensive speech and language applications; iii) how to parallelize learning algorithms to efficiently solve large-scale optimization in model estimation. We believe these issues are critical for us to fully benefit from massive real-world data in many data-intensive speech and language applications. This research program will help to advance large-scale learning and automatic knowledge discovery technologies for many Canadian and International industrial IT sections that are based on the more and more popular cloud-computing model.