Innovation Adoption by Committee: Evaluating Decision-Making in the FDA

美国食品和药物管理局（FDA）负责批准美国的新药、生物制品和医疗器械。作为这一过程的一部分，FDA依靠咨询委员会（ACs）就新产品的采用提供建议。在这个项目中，研究人员将量化咨询委员会系统的有效性，并评估可能的制度创新是否会提高其决策质量。这个项目由三篇论文组成。在第一篇论文中，研究小组研究了咨询委员会是否有效地利用了有关新产品的可用信息。为了做到这一点，作者将ACs决策的博弈论模型与ACs会议记录中的审议和投票数据结合起来。这种方法允许研究人员进行政策实验，检查FDA对新产品批准过程的制度变化如何影响获得正确建议的可能性。在第二篇论文中，研究人员利用行政代表组成的外生变化来评估多样化代表授权对审议结果的影响。在第三个项目中，作者利用批准产品的事后表现信息，研究FDA咨询委员会长期决策的质量。总之，该项目将提供关键的新证据，以告知FDA关于采用新产品的决定，以及其制度设计。此外，本项目发展的经济模型和统计方法可用于研究如何在其他决策机构，包括管理机构和中央银行以及公司董事会中使用信息。最先进的统计方法的实施、数据收集工作和新经济模型的开发将为学生提供重要的指导机会，他们将参与相关文献，探索不同假设下简化模型的结果，并协助编码和数据分析。拟议的项目以各种方式促进知识的发展。首先，该提案收集了一个对研究和政策都有用的新的微观数据集（来自公开但分散和未经处理的信息），跟踪了过去15年里每次咨询委员会会议的审议和投票的整个过程。在第一篇论文中，研究人员从结构上估计了一个动态模型，在这个模型中，具有异质偏好的委员会成员集体决定何时停止收集信息，并投票推荐批准或拒绝新产品。该模型仔细考虑了这些成员在收集信息时如何影响其同伴的行为。在数据方面，实证模型为研究该领域的集体审议和信息获取提供了一种新的方法，可以推广到其他集体决策机构。在第二篇论文中，研究人员利用咨询委员会组成的外生变化和网络结构的不完全重叠来评估多样化代表的授权对审议结果的影响。该分析提供了在审议机构范围内这种政策的有效性的新见解，这些政策需要集体学习。第三篇论文有助于了解委员会在信息不完全的情况下做出的决策的长期结果。要做到这一点，它明确地纳入了批准产品的事后性能。由于不同的设备可能有不同的基线错误率（例如，使用复杂的设备可能更难做出正确的决策），研究人员将比较委员会决策下的长期结果与由单个决策者制定政策的反事实政策实验。这将进一步说明基于委员会的决策相对于个人决策的长期好处，使两种环境中的可用信息保持不变。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The Food and Drug Administration (FDA) is responsible for the approval of new drugs, biological products and medical devices in the United States. As part of this process, the FDA relies on advisory committees (ACs) to provide recommendations on the adoption of new products. In this project, the researchers will quantify the effectiveness of the advisory committee system and evaluate whether possible institutional innovations would improve the quality of its decision-making. The project is composed of three papers. In the first paper, the research team studies whether advisory committees use available information about new products effectively. To do this, the authors combine a game-theoretic model of decision-making in ACs with deliberation and voting data from ACs’ meetings’ transcripts. The approach allows the researchers to conduct policy experiments examining how institutional changes to the approval process for new products by the FDA affect the likelihood of reaching correct recommendations. In the second paper, the researchers exploit exogenous variation in the composition of ACs to evaluate the effect of mandates for diverse representation on deliberative outcomes. In the third project, the authors study the quality of FDA advisory committee decision-making in the long run, using information on the ex-post performance of approved products. Altogether, the project will provide crucial new evidence to inform FDA’s decisions about the adoption of new products, as well as its institutional design. Moreover, the economic models and statistical methods developed in this project could be applied to study how information is used in other policy-making bodies, including regulatory bodies and central banks, as well as in boards of directors in corporations. The implementation of state-of-the-art statistical methods, data collection efforts, and development of new economic models will allow significant mentoring opportunities for students, who will engage related literature, explore results of simplified models under alternative assumptions, and aid in coding and data analysis. The proposed project advances knowledge in various ways. At the outset, the proposal assembles a new microlevel dataset (from publicly available but dispersed and unprocessed information) that is useful for both research and policy, tracking the entire process of deliberation and voting for each advisory committee meeting in the past fifteen years. In the first paper, the researchers structurally estimate a dynamic model in which committee members with heterogeneous preferences collectively decide when to stop gathering information, and vote to recommend the approval or rejection of new products. This model carefully considers how such members may influence their peers’ behavior when gathering information. When taken to data, the empirical model provides a new approach to study collective deliberation and information acquisition in the field, which can be extended to other collective policy-making bodies. In the second paper, the researchers leverage exogenous variation in the composition of advisory committees and imperfect overlap in the network structure to evaluate the effect of mandates for diverse representation on deliberative outcomes. The analysis provides new insights of the effectiveness of such policies within the context of deliberative bodies, which are subject to collective learning. The third paper contributes to knowledge on long run outcomes from committee-made decisions with imperfect information. To do so, it explicitly incorporates the ex-post performance of approved products. As different devices may have different baseline error rates (e.g., it may be more difficult to make correct decisions as often with complex devices), the researchers will compare the long run outcomes under committee decision-making with a counterfactual policy experiment where the policy is set by a single decision-maker. This will further inform the long-run benefits of committee-based decisions relative to individual ones, keeping constant the information available in both environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.