Learning with Discrete Structure

深度学习已经彻底改变了人工智能的许多子领域。深度学习模型最初受到人类大脑的启发，通常被称为神经网络，可以被训练来模拟非常复杂的输入-输出关系。输入-输出关系的例子包括与给定语音记录或给定图像中存在的对象类型相对应的书面文本。深度学习模型擅长于从输入中提取细微的统计模式，并可用于对输出产生非常准确的预测。多年来，我们训练深度学习模型的能力有了显著提高，这给许多需要准确预测的应用带来了革命性的变化。想想OpenAI的GPT3系统。GPT3是一种自然语言模型，可以学习预测句子中的下一个单词。经过训练后，对模型进行修改，以解决各种任务，包括语义搜索、客户服务和内容理解。通过简单的web界面提供对训练模型的访问。GPT3的预测质量如此之高，以至于人类很难将其与人类的预测区分开来，该系统现在每天处理数百万个查询。部分由于这样的成功，有一种感觉，革命最终将波及每一个领域。工业和医疗应用通常被认为是深度学习的下一个前沿领域。在许多这些应用中，已知数据具有离散结构。例如，在药物发现方法中，人们通常对预测候选分子是否具有某种性质感兴趣。分子可以用图形表示，其中节点对应原子，边缘对应化学键。这种离散结构可用于改进机器学习系统的预测。再举一个例子，考虑整数约束下的线性优化问题。这些都是具有挑战性的组合优化问题，一些研究人员提出了直接预测最优解的深度学习模型。在这种设置中，应该满足问题规范中的约束的输出是一个结构化的离散对象。尽管深度学习前景光明，但在将其应用于离散结构化数据方面仍然存在挑战。例如，试图学习组合解算器的启发式算法，通常会受到神经网络计算速度缓慢的阻碍。在分子药物设计中，深度学习邮件无法产生可解释的模型，这使得科学家难以充分利用深度学习预测。我的研究项目的长期目标是解决这些挑战，并实现对离散结构化数据的深度学习的承诺。

Deep learning has revolutionized many subfields of artificial intelligence. Deep learning models, originally inspired by the human brain and often called neural nets, can be trained to mimic very complex input-output relationships. Examples of input-output relationships include the written text that corresponds to a given recording of speech or the type of object present in a given image. Deep learning models excel at extracting subtle statistical patterns from their inputs and can be used to produce very accurate predictions of the outputs. Our ability to train deep learning models has improved dramatically over the years, which has revolutionized many applications where accurate predictions are important. Consider OpenAI's GPT3 system. GPT3 is a natural language model that learns to predict the next word in a sentence. Once trained, the model is modified to solve a wide range of tasks, including semantic search, customer service, and content comprehension. Access to the trained model is offered via a simple web interface. The quality of GPT3's predictions is so high that humans have trouble distinguishing them from human predictions, and the system now serves millions of queries a day. Due in part to successes like this, there is a sense that the revolution will eventually come to every domain. Industrial and medical applications are often cited as the next frontier for deep learning. In many of these applications the data is known to have discrete structure. For example, in drug discovery methods, one is often interested in predicting whether a candidate molecule has a certain property. Molecules can be represented as graphs in which nodes correspond to atoms and edges correspond to chemical bonds. This discrete structure can be used to improve the predictions of machine learning systems. For another example, consider the problem of linear optimization with integer constraints. These are challenging combinatorial optimization problems, and some researchers have proposed deep learning models that directly predict optimal solutions. In this setting, the output, which should satisfy the constraints in the problem specification, is a structured discrete object. Despite the promise of deep learning, challenges remain in its application to discrete structured data. Attempts to learn the heuristics of combinatorial solvers, for instance, are typically held back by the slowness of neural net computations. In molecular drug design, deep learning mail fail to produce interpretable models, making it difficult for scientists to take full advantage of deep learning predictions. The long-term goal of my research program is to address these challenges and deliver on the promise of deep learning for discrete structured data.