Collaborative Research: CIF: Small: Sequential Decision Making Under Uncertainty With Submodular Rewards

合作研究：CIF：小：不确定性下的顺序决策与子模奖励

• 批准号: 2149588
• 负责人: Vaneet Aggarwal
• 金额: $ 25万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-03-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2149588&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Sequential

Many companies, government agencies, and individuals make sequences of challenging decisions over time, for which they must choose from among many possible options, may have limited knowledge about the outcomes of their decisions, and will receive limited feedback. For example, search engines and content providers make decisions for what sets of websites, products, or media to recommend each time a user logs on to their system or submits a query, in some cases having limited knowledge of the users’ underlying preferences. If users' privacy is protected, then only users' past actions, such as which links or media were selected by earlier users, will be available as feedback to inform the search engine or content provider on what to recommend next. This project aims to develop provably good strategies that decision makers can use in such settings, aiding their decision making under uncertainty and with limited feedback. This project will also develop strategies for the more challenging setting where multiple decision makers must coordinate with each other on such problems, but have limited communication available to do so. Furthermore, this project will support undergraduate and graduate research training, as well as graduate-level course development, in machine learning and artificial intelligence, preparing students for careers in advanced technical fields.The goal of this project is to develop novel, provably good strategies for solving sequential decision problems (multi-armed bandit problems) when the actions available have a combinatorial structure (such as choosing subsets of products to recommend), the rewards have a diminishing returns property (submodularity), and there is no side-information available -- the only feedback comes from the reward itself. The proposed work builds on the rich literature of multi-armed bandits and of submodular optimization. The technical aims of the project are divided into two thrusts. The first thrust focuses on developing algorithms and identifying their regret bounds for combinatorial multi-armed bandit problems with submodular rewards and no additional feedback. The second thrust extends those strategies and regret analyses to a decentralized setting, where multiple agents coordinate to solve combinatorial multi-armed bandit problems, despite limited resources for communication.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着时间的推移，许多公司，政府机构和个人都会做出挑战决策的序列，在许多可能的选择中，他们必须选择这些决定，可能对他们的决策结果有限，并且会得到有限的反馈。例如，搜索引擎和内容提供商对用户每次登录其系统或提交查询的网站，产品或媒体集做出决策，在某些情况下，对用户的基本偏好知识有限。如果用户的隐私受到保护，则只有用户的过去操作（例如较早用户选择的链接或媒体）作为反馈可用，以告知搜索引擎或内容提供商下一步推荐的内容。该项目旨在制定正确的良好策略，决策者可以在这种情况下使用这些策略，从而在不确定性和反馈有限的情况下帮助他们的决策做出。该项目还将为更挑战的设置制定策略，其中多个决策者必须在此类问题上彼此协调，但可以进行有限的沟通。此外，该项目将支持本科和研究生研究培训，以及研究生级课程的发展，机器学习和人工智能，为学生在高级技术领域的职业中做好准备。该项目的目标是制定新颖的策略，可能是解决顺序决策问题的良好策略（多臂bastit bastit问题）（多臂bastit bastit问题），当可用的行动可用的范围内有一个组合（例如，都可以选择组合的产品，可以选择一个组合的产品，可以选择一个组合的产品，可以选择一个组合，以选择组合的产品，该策略是一定选择的，该策略是一定的选择，既可以选择了，又可以选择一个组合的产品。 （suppodulonity），并且没有可用的侧面信息 - 唯一的反馈来自奖励本身。拟议的作品建立在多军匪徒的丰富文献基础上，并进行了下义优化。该项目的技术目标分为两个推力。第一个推力重点是开发算法，并确定其对组合多臂匪徒的遗憾界限，并没有额外的反馈。第二个将这些策略扩展到了分散的环境，在该环境中，多个代理协调了解决组合多臂匪徒问题，期望有限的通信资源。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的审查标准来通过评估来通过评估来获得支持的。

项目成果

Combinatorial Stochastic-Greedy Bandit

组合随机贪婪老虎机

• 发表时间: 2024
• 期刊: AAAI
• 作者: Fourati, Fares and

Unified Projection-Free Algorithms for Adversarial DR-Submodular Optimization

用于对抗性 DR 子模优化的统一无投影算法

• 发表时间: 2024
• 期刊: ICLR
• 作者: Pedramfar, Mohammad and

Randomized Greedy Learning for Non-monotone Stochastic Submodular Maximization Under Full-bandit Feedback

全老虎机反馈下非单调随机子模最大化的随机贪婪学习

• 发表时间: 2023
• 期刊: Proceedings of the International Workshop on Artificial Intelligence and Statistics
• 作者: Fourati, Fares;Aggarwal, Vaneet;Quinn, Christopher John;Alouini, Mohamed-Slim
• 通讯作者: Alouini, Mohamed-Slim

Multi-Agent Multi-Armed Bandits with Limited Communication

通信受限的多代理多臂强盗

• 发表时间: 2022
• 期刊: Journal of machine learning research
• 影响因子: 6
• 作者: Mridul Agarwal, Vaneet Aggarwal

A Framework for Adapting Offline Algorithms to Solve Combinatorial Multi-Armed Bandit Problems with Bandit Feedback

• DOI: 10.48550/arxiv.2301.13326
• 发表时间: 2023-01
• 期刊: ArXiv
• 作者: G. Nie;Yididiya Y. Nadew;Yanhui Zhu;V. Aggarwal;Christopher J. Quinn