Towards more efficient machine learning algorithms: theory and practice

机器学习与智能计算机系统的发展有关，这些系统能够从收集的数据中学习和推广。许多重要的任务都需要这种能力，这些任务太过复杂，无法由人类明确编程，例如识别智能手机上的语音指令，或识别图像中的面孔，或显示查询的最佳结果。完成这些任务的最好的计算机系统都有一个共同点，即它们的能力是通过在大量数据上运行学习算法获得的。其中许多任务现在对我们的经济至关重要，以至于基于机器学习的技术现在已经司空见惯。然而，这项技术需要大大改进。事实上，信用卡被屏蔽的频率太高，我们仍然收到太多不受欢迎的电子邮件，自动语音识别仍然不令人满意。构建更强大的硬件只是解决方案的一部分，因为我们还必须找到最有效的学习算法。因此，提出的研究计划旨在解决如何改进和修改现有学习算法的重要问题，以便它们在使用最小（或至少是可接受的）资源量的情况下产生更好的预测器。为了实现这一目标，我们计划从理论和实践的角度来解决具有挑战性的机器学习问题。理论分析是必要的，因为我们想要找到具有可证明保证的高效学习算法。更具体地说，我们计划在短期内改进现有的结构化输出预测和领域适应的学习算法。这是目前两个具有挑战性的机器学习问题，它们对实践中经常遇到的情况进行建模，但我们还没有令人满意的学习算法。我们还计划找到最有效的增强型算法，用于从大规模数据集中学习。最后，从长远来看，我们计划扩展集合覆盖机和决策列表机，用于从基因组数据预测表型。这些是学习算法（我们在过去提出过），它们产生非常稀疏的预测器，很容易解释。因此，这些预测因子原则上可以帮助我们揭示表型的基因组原因。我们最近从细菌基因组预测抗生素耐药性方面获得的令人印象深刻的结果激励我们考虑，在短期内，从人类序列预测癌症类型的更雄心勃勃的问题，可能是DNA， RNA或蛋白质序列。因此，该研究项目有望为不同的基于机器学习的应用程序提供可证明的有效学习方法，从而提高我们的生活质量。此外，预计该研究项目将产生相当于2名硕士毕业生和4名博士毕业生。

Machine learning is concerned with the development of intelligent computer systems that are able to learn and generalize from collected data. This ability is required for many important tasks that are just too complex to be explicitly programmed by humans such as recognizing voice directives on smart phones, or recognizing faces in images, or displaying the best results from a query. The best computer systems that achieve these tasks have in common the fact that their ability has been acquired by running learning algorithms on vast amounts of data. Many of these tasks are now so vital to our economy that machine learning-based technology is now commonplace. However, that technology needs to be greatly improved. Indeed, credit cards are too frequently being blocked, we are still receiving too many undesirable emails, and automatic speech recognition is still not satisfactory. Building more powerful hardware is just part of the solution as we must also find the most efficient learning algorithms. Consequently, the proposed research program aims at addressing the important problem of how to improve and modify existing learning algorithms such that they yield better predictors while using a minimal (or, at least, an acceptable) amount of resources. To meet this objective we plan to approach challenging machine-learning problems from both a theoretical and a practical perspective. Theoretical analysis is needed because we want to find efficient learning algorithms with provable guarantees. More specifically, we plan, in the short term, to improve the existing learning algorithms for structured output prediction and domain adaptation. These are currently two challenging machine-learning problems that model situations often encountered in practice but for which we do not have yet satisfactory learning algorithms. We also plan to find the most efficient boosting-type algorithms for learning from large-scale data sets. Finally, in the long term, we plan to expand the set covering machine and the decision list machine for predicting phenotypes from genomic data. These are learning algorithms (that we have proposed in the past) that produce uncharacteristically sparse predictors that are easy to interpret. Hence, these predictors can, in principle, help us to uncover the genomic cause of a phenotype. The impressive results we have obtained recently on the prediction of antibiotic resistance from bacterial genomes motivate us to consider, in the short term, the more ambitious problem of predicting cancer types from human sequences, which could be DNA, RNA, or protein sequences. It is thus expected that this research program will yield provably efficient learning methods for different machine learning-based applications that, in turn, will improve our quality of life. Moreover, it is expected that this research program will deliver the equivalent of two M.Sc graduates and four Ph.D graduates.