Learning with Discrete Structure

点击翻译按钮获取中文摘要

Deep learning has revolutionized many subfields of artificial intelligence. Deep learning models, originally inspired by the human brain and often called neural nets, can be trained to mimic very complex input-output relationships. Examples of input-output relationships include the written text that corresponds to a given recording of speech or the type of object present in a given image. Deep learning models excel at extracting subtle statistical patterns from their inputs and can be used to produce very accurate predictions of the outputs. Our ability to train deep learning models has improved dramatically over the years, which has revolutionized many applications where accurate predictions are important. Consider OpenAI's GPT3 system. GPT3 is a natural language model that learns to predict the next word in a sentence. Once trained, the model is modified to solve a wide range of tasks, including semantic search, customer service, and content comprehension. Access to the trained model is offered via a simple web interface. The quality of GPT3's predictions is so high that humans have trouble distinguishing them from human predictions, and the system now serves millions of queries a day. Due in part to successes like this, there is a sense that the revolution will eventually come to every domain. Industrial and medical applications are often cited as the next frontier for deep learning. In many of these applications the data is known to have discrete structure. For example, in drug discovery methods, one is often interested in predicting whether a candidate molecule has a certain property. Molecules can be represented as graphs in which nodes correspond to atoms and edges correspond to chemical bonds. This discrete structure can be used to improve the predictions of machine learning systems. For another example, consider the problem of linear optimization with integer constraints. These are challenging combinatorial optimization problems, and some researchers have proposed deep learning models that directly predict optimal solutions. In this setting, the output, which should satisfy the constraints in the problem specification, is a structured discrete object. Despite the promise of deep learning, challenges remain in its application to discrete structured data. Attempts to learn the heuristics of combinatorial solvers, for instance, are typically held back by the slowness of neural net computations. In molecular drug design, deep learning mail fail to produce interpretable models, making it difficult for scientists to take full advantage of deep learning predictions. The long-term goal of my research program is to address these challenges and deliver on the promise of deep learning for discrete structured data.