Innovation Adoption by Committee: Evaluating Decision-Making in the FDA

美国食品和药物管理局（FDA）负责批准美国的新药、生物制品和医疗器械。作为这一过程的一部分，FDA依靠咨询委员会（AC）提供关于采用新产品的建议。在这个项目中，研究人员将量化咨询委员会制度的有效性，并评估可能的制度创新是否会提高其决策质量。该项目由三篇论文组成。在第一篇论文中，研究小组研究了咨询委员会是否有效地使用了有关新产品的可用信息。为了做到这一点，作者结合联合收割机的AC的决策与审议和表决数据，从AC的会议记录的博弈论模型。该方法允许研究人员进行政策实验，研究FDA对新产品批准程序的制度变化如何影响获得正确建议的可能性。在第二篇论文中，研究者利用AC组成的外生变化来评估不同代表性对审议结果的影响。在第三个项目中，作者研究了FDA咨询委员会长期决策的质量，使用有关批准产品的事后性能的信息。总而言之，该项目将提供关键的新证据，为FDA关于采用新产品及其机构设计的决定提供信息。此外，本项目开发的经济模型和统计方法可用于研究其他决策机构，包括监管机构和中央银行以及公司董事会如何使用信息。最先进的统计方法，数据收集工作和新经济模型的开发的实施将为学生提供重要的指导机会，他们将参与相关文献，在替代假设下探索简化模型的结果，并帮助编码和数据分析。拟议的项目以各种方式推进知识。首先，该提案收集了一个新的微观数据集（来自公开但分散和未处理的信息），对研究和政策都有用，跟踪过去15年来每次咨询委员会会议的审议和投票的整个过程。在第一篇论文中，研究人员从结构上估计了一个动态模型，在这个模型中，具有不同偏好的委员会成员共同决定何时停止收集信息，并投票建议批准或拒绝新产品。该模型仔细考虑了此类成员在收集信息时如何影响同龄人的行为。当采取的数据，实证模型提供了一个新的方法来研究集体审议和信息获取的领域，这可以扩展到其他集体决策机构。在第二篇论文中，研究人员利用咨询委员会组成中的外生变化和网络结构中的不完美重叠来评估不同代表性的授权对审议结果的影响。分析提供了新的见解，这些政策的有效性范围内的审议机构，这是受集体学习。第三篇论文有助于了解委员会在信息不完全的情况下做出的决策的长期结果。为此，它明确纳入了批准产品的事后性能。由于不同的设备可能具有不同的基线错误率（例如，可能更难做出正确的决定，因为往往与复杂的设备），研究人员将比较委员会决策下的长期结果与反事实的政策实验，其中政策是由一个单一的决策者。这将进一步告知基于委员会的决策相对于个人决策的长期利益，保持两种环境中可用的信息不变。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The Food and Drug Administration (FDA) is responsible for the approval of new drugs, biological products and medical devices in the United States. As part of this process, the FDA relies on advisory committees (ACs) to provide recommendations on the adoption of new products. In this project, the researchers will quantify the effectiveness of the advisory committee system and evaluate whether possible institutional innovations would improve the quality of its decision-making. The project is composed of three papers. In the first paper, the research team studies whether advisory committees use available information about new products effectively. To do this, the authors combine a game-theoretic model of decision-making in ACs with deliberation and voting data from ACs’ meetings’ transcripts. The approach allows the researchers to conduct policy experiments examining how institutional changes to the approval process for new products by the FDA affect the likelihood of reaching correct recommendations. In the second paper, the researchers exploit exogenous variation in the composition of ACs to evaluate the effect of mandates for diverse representation on deliberative outcomes. In the third project, the authors study the quality of FDA advisory committee decision-making in the long run, using information on the ex-post performance of approved products. Altogether, the project will provide crucial new evidence to inform FDA’s decisions about the adoption of new products, as well as its institutional design. Moreover, the economic models and statistical methods developed in this project could be applied to study how information is used in other policy-making bodies, including regulatory bodies and central banks, as well as in boards of directors in corporations. The implementation of state-of-the-art statistical methods, data collection efforts, and development of new economic models will allow significant mentoring opportunities for students, who will engage related literature, explore results of simplified models under alternative assumptions, and aid in coding and data analysis. The proposed project advances knowledge in various ways. At the outset, the proposal assembles a new microlevel dataset (from publicly available but dispersed and unprocessed information) that is useful for both research and policy, tracking the entire process of deliberation and voting for each advisory committee meeting in the past fifteen years. In the first paper, the researchers structurally estimate a dynamic model in which committee members with heterogeneous preferences collectively decide when to stop gathering information, and vote to recommend the approval or rejection of new products. This model carefully considers how such members may influence their peers’ behavior when gathering information. When taken to data, the empirical model provides a new approach to study collective deliberation and information acquisition in the field, which can be extended to other collective policy-making bodies. In the second paper, the researchers leverage exogenous variation in the composition of advisory committees and imperfect overlap in the network structure to evaluate the effect of mandates for diverse representation on deliberative outcomes. The analysis provides new insights of the effectiveness of such policies within the context of deliberative bodies, which are subject to collective learning. The third paper contributes to knowledge on long run outcomes from committee-made decisions with imperfect information. To do so, it explicitly incorporates the ex-post performance of approved products. As different devices may have different baseline error rates (e.g., it may be more difficult to make correct decisions as often with complex devices), the researchers will compare the long run outcomes under committee decision-making with a counterfactual policy experiment where the policy is set by a single decision-maker. This will further inform the long-run benefits of committee-based decisions relative to individual ones, keeping constant the information available in both environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.