Collaborative Research: RI: Medium: Submodular Information Functions with Applications to Machine Learning

越来越多的机器学习应用涉及选择数据子集。例子包括从一个大得多的数据集中选择较小的子集来标记（以节省标记成本）和训练（以减少计算成本），或者选择视频或照片集合的摘要以方便人们查看。子模块化是解决这些问题的自然方法，因为它自然地对多样性、表示和覆盖等许多方面进行建模。在这个项目中，pi将研究一类丰富的子模块信息度量，这些信息度量不仅对多样性、代表性、覆盖范围进行建模，而且还对某些目标概念的相关性和不相关性等结构进行建模。其中一个应用是选择具有特定用户规范的数据摘要——例如，与给定查询相关的摘要或在隐私约束下的摘要（与特定人员相关的照片摘要或避免某些个人信息的摘要）。另一个应用是交互式地选择数据样本，以便在存在罕见类别或避免异常值的情况下进行标记（例如，将癌症图像作为医学成像任务的罕见类别）。这一领域的进步可以在许多领域产生影响，包括数据总结、减少标记工作（在医学成像等任务中），以及减少在大规模数据集上训练深度学习模型的碳足迹。本项目提出的基础数学模型是一类称为“子模信息测度”的丰富函数，其中包括子模互信息、子模条件增益、子模多集互信息、有向子模互信息和组合独立性。具体而言，pi将调查和开发：(1)这些子模块信息度量的丰富理论属性和实例；(2)相关优化问题的优化算法、近似界和硬度结果；(3)子模块信息测度在数据汇总、数据子集选择、主动学习、聚类和多样化划分等方面的应用。在开展这些活动的同时，pi将让本科生和代表性不足的高中生参与这项研究，以激励他们在人工智能/机器学习和其他stem相关领域从事职业。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A growing number of machine learning applications involve selecting subsets of data. Examples include selecting smaller subsets from a much larger dataset to label (to save labeling costs) and to train (to reduce computational costs), or selecting a summary of a video or a photo collection to ease viewing by a person. Submodularity is a natural way to address these problems because it naturally models many aspects like diversity, representation, and coverage. In this project, the PIs will study a rich class of submodular information measures that model not only diversity, representation, coverage but also constructs such as relevance and irrelevance to certain target concepts. One application of this is selecting a data summary with certain user specifications -- e.g., a summary relevant to a given query or under a privacy constraint (a photo summary relevant to a specific person or one which avoids certain personal information). Another application is to interactively select data samples to label in the presence of rare classes or while avoiding outliers (e.g., cancerous images as rare classes for medical imaging tasks). Advances in this field can have implications in many areas including data summarization, reducing labeling efforts (in tasks like medical imaging), and reducing the carbon footprint for training deep learning models on massive datasets.The underlying mathematical model proposed in this project is a rich class of functions called ``submodular information measures``, which includes submodular mutual information, submodular conditional gain, submodular multi-set mutual information, directed submodular mutual information, and combinatorial independence. Specifically, the PIs will investigate and develop: (1) rich theoretical properties and instantiations of these submodular information measures; (2) optimization algorithms, approximation bounds, and hardness results of the associated optimization problems; (3) applications of the submodular information measures in data summarization, data subset selection, active learning, clustering, and diversified partitioning. While pursuing these activities, the PIs will involve undergraduate and under-represented high-school students in this research to inspire them to pursue careers in AI/ML and other STEM-related fields.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.