Learning with Discrete Structure

离散结构学习

• 批准号: RGPIN-2021-03445
• 负责人: Maddison, Christopher
• 金额: $ 2.91万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750047
• 关键词: Learning; Discrete; Structure

Deep learning has revolutionized many subfields of artificial intelligence. Deep learning models, originally inspired by the human brain and often called neural nets, can be trained to mimic very complex input-output relationships. Examples of input-output relationships include the written text that corresponds to a given recording of speech or the type of object present in a given image. Deep learning models excel at extracting subtle statistical patterns from their inputs and can be used to produce very accurate predictions of the outputs. Our ability to train deep learning models has improved dramatically over the years, which has revolutionized many applications where accurate predictions are important. Consider OpenAI's GPT3 system. GPT3 is a natural language model that learns to predict the next word in a sentence. Once trained, the model is modified to solve a wide range of tasks, including semantic search, customer service, and content comprehension. Access to the trained model is offered via a simple web interface. The quality of GPT3's predictions is so high that humans have trouble distinguishing them from human predictions, and the system now serves millions of queries a day. Due in part to successes like this, there is a sense that the revolution will eventually come to every domain. Industrial and medical applications are often cited as the next frontier for deep learning. In many of these applications the data is known to have discrete structure. For example, in drug discovery methods, one is often interested in predicting whether a candidate molecule has a certain property. Molecules can be represented as graphs in which nodes correspond to atoms and edges correspond to chemical bonds. This discrete structure can be used to improve the predictions of machine learning systems. For another example, consider the problem of linear optimization with integer constraints. These are challenging combinatorial optimization problems, and some researchers have proposed deep learning models that directly predict optimal solutions. In this setting, the output, which should satisfy the constraints in the problem specification, is a structured discrete object. Despite the promise of deep learning, challenges remain in its application to discrete structured data. Attempts to learn the heuristics of combinatorial solvers, for instance, are typically held back by the slowness of neural net computations. In molecular drug design, deep learning mail fail to produce interpretable models, making it difficult for scientists to take full advantage of deep learning predictions. The long-term goal of my research program is to address these challenges and deliver on the promise of deep learning for discrete structured data.

深度学习给人工智能的许多子领域带来了革命性的变化。深度学习模型最初是由人脑启发的，通常被称为神经网络，可以被训练成模拟非常复杂的输入输出关系。输入-输出关系的例子包括对应于给定的语音记录或给定图像中存在的对象的类型的书面文本。深度学习模型擅长从其输入中提取微妙的统计模式，并可用于产生非常准确的输出预测。多年来，我们训练深度学习模型的能力有了显著提高，这给许多需要准确预测的应用程序带来了革命性的变化。考虑一下OpenAI的GPT3系统。GPT3是一个自然语言模型，它学习预测句子中的下一个单词。一旦经过训练，该模型将被修改以解决广泛的任务，包括语义搜索、客户服务和内容理解。通过简单的Web界面即可访问经过训练的模型。GPT3的S预测的质量如此之高，以至于人类很难将它们与人类的预测区分开来，该系统现在每天处理数以百万计的查询。部分由于这样的成功，人们有一种感觉，这场革命最终将进入每一个领域。工业和医疗应用经常被认为是深度学习的下一个前沿。在许多这样的应用中，众所周知，数据具有离散的结构。例如，在药物发现方法中，人们经常对预测候选分子是否具有某种性质感兴趣。分子可以表示为图，其中节点对应于原子，边对应于化学键。这种离散结构可用于提高机器学习系统的预测能力。再举一个例子，考虑具有整数约束的线性优化问题。这些都是具有挑战性的组合优化问题，一些研究人员提出了直接预测最优解的深度学习模型。在这种情况下，应该满足问题说明中的约束的输出是结构化的离散对象。尽管深度学习带来了希望，但在将其应用于离散结构化数据方面仍然存在挑战。例如，学习组合求解器的启发式算法的尝试通常会受到神经网络计算速度缓慢的阻碍。在分子药物设计中，深度学习邮件无法产生可解释的模型，这使得科学家很难充分利用深度学习预测。我的研究计划的长期目标是应对这些挑战，并实现对离散结构化数据进行深度学习的承诺。