Integration of Heterogeneous Data in Artificial Neural Networks for Image Classification

点击翻译按钮获取中文摘要

The goal of our research program is to develop new methods in machine learning, which is a type of artificial intelligence, to perform decision-making with combinations of different types of data, which we call multi-modal data. More specifically, we focus on large and complex images and will develop methods to integrate other types of information to enable categorization or classification of the images into informative outcomes. For example, a person's medical scans, such as magnetic resonance images (MRIs), and his or her medical history can be used in combination to predict an outcome (e.g., new symptoms or not). Machine learning is very helpful for such tasks, because computers can look through very large databases of information to identify patterns much faster and find more subtle links than humans can, allowing the user to focus more on defining and using the outcomes. However, one of the open questions in machine learning is how to combine multiple types of data that are very different from each other (so called heterogeneous multi-modal data) so that we can make the most accurate predictions. Answering this question is important because we do not always know ahead of time which combination of data types is the most useful for a given prediction task. In recent years, an approach of machine learning called deep learning has been shown to be very effective in extracting patterns in large image databases, and we have developed a number of new deep learning methods for learning patterns from brain MRIs, which are large complex images. The next step in our research is to investigate how to combine automatically extracted image patterns with other types of data, so that the computer can make the most accurate predictions. Our data will consist of large sets of MRIs of persons with neurological disorders and their non-imaging data such as biological measurements, demographic information, and clinical histories. The prediction outcomes will be some form of important clinical category, such as worsening, stable, or improving over a period of time. We will build on some recently proposed ideas in the machine learning literature, but we expect that much new work will have to be done, because of the newness of the ideas, and because most of the existing work is applied to much smaller images and in domains where relationships between data types are much clearer. Over the next several years, we expect a continued rapid proliferation of multi-modal data, and the machine learning methods that incorporate good integration strategies will begin to show their advantage. We are confident that the methods and knowledge produced by this research program will positively impact this fundamental need in data science.