Collaborative Research: RI: Medium: Submodular Information Functions with Applications to Machine Learning

越来越多的机器学习应用涉及选择数据子集。例子包括从大得多的数据集中选择较小的子集来标记（以节省标记成本）和训练（以降低计算成本），或者选择视频或照片集的摘要以便于人们查看。子模块化是解决这些问题的自然方法，因为它自然地对多样性，表示和覆盖等许多方面进行建模。在这个项目中，PI将研究丰富的子模块信息度量，不仅建模多样性，代表性，覆盖范围，而且还构建相关性和不相关性，以某些目标概念。其中一个应用是选择具有特定用户规范的数据摘要--例如，与给定查询相关或在隐私约束下的摘要（与特定人相关的照片摘要或避免某些个人信息的照片摘要）。另一个应用是交互式地选择数据样本以在存在稀有类的情况下或在避免离群值的同时进行标记（例如，癌症图像作为医学成像任务的罕见类别）。这一领域的进展可能会在许多领域产生影响，包括数据汇总，减少标签工作（在医学成像等任务中），以及减少在大规模数据集上训练深度学习模型的碳足迹。该项目中提出的底层数学模型是一类称为“子模块信息度量”的丰富函数，其中包括子模块互信息，子模块条件增益，子模多集互信息、有向子模互信息和组合独立性。具体而言，PI将研究和开发：（1）这些子模块信息度量的丰富理论属性和实例;（2）优化算法，近似边界和相关优化问题的硬度结果;（3）子模块信息度量在数据摘要，数据子集选择，主动学习，聚类和多样化划分中的应用。在开展这些活动的同时，PI将让本科生和代表性不足的高中生参与这项研究，以激励他们在AI/ML和其他STEM相关领域从事职业。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

A growing number of machine learning applications involve selecting subsets of data. Examples include selecting smaller subsets from a much larger dataset to label (to save labeling costs) and to train (to reduce computational costs), or selecting a summary of a video or a photo collection to ease viewing by a person. Submodularity is a natural way to address these problems because it naturally models many aspects like diversity, representation, and coverage. In this project, the PIs will study a rich class of submodular information measures that model not only diversity, representation, coverage but also constructs such as relevance and irrelevance to certain target concepts. One application of this is selecting a data summary with certain user specifications -- e.g., a summary relevant to a given query or under a privacy constraint (a photo summary relevant to a specific person or one which avoids certain personal information). Another application is to interactively select data samples to label in the presence of rare classes or while avoiding outliers (e.g., cancerous images as rare classes for medical imaging tasks). Advances in this field can have implications in many areas including data summarization, reducing labeling efforts (in tasks like medical imaging), and reducing the carbon footprint for training deep learning models on massive datasets.The underlying mathematical model proposed in this project is a rich class of functions called ``submodular information measures``, which includes submodular mutual information, submodular conditional gain, submodular multi-set mutual information, directed submodular mutual information, and combinatorial independence. Specifically, the PIs will investigate and develop: (1) rich theoretical properties and instantiations of these submodular information measures; (2) optimization algorithms, approximation bounds, and hardness results of the associated optimization problems; (3) applications of the submodular information measures in data summarization, data subset selection, active learning, clustering, and diversified partitioning. While pursuing these activities, the PIs will involve undergraduate and under-represented high-school students in this research to inspire them to pursue careers in AI/ML and other STEM-related fields.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.