Network Experimentation at Scale
Network Experimentation at Scale
复制标题
大规模网络实验
DOI：
10.1145/3447548.3467091
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Feng Sun
中科院分区：
文献类型：
--
作者：
B. Karrer;Liang Shi;Monica Bhole;Matt Goldman;Tyrone Palmer;Charlie Gelman;Mikael Konutgan;Feng Sun
我们描述了我们的网络实验框架，部署在Facebook，占实验单位之间的干扰。我们记录了这个系统，包括设计和估计程序，并详细了解我们已经从许多实验中获得的大规模使用这个系统。在我们的估计过程中，我们引入了一个基于聚类的回归调整，大大提高了估计全球治疗效果的精度，以及测试干扰的程序。通过我们的回归调整，我们发现不平衡的集群可以更好地解释干扰，而不会牺牲准确性。此外，我们表明，记录暴露于治疗可能会导致额外的方差减少。干扰是在线现场实验中一个被广泛认可的问题，但现实世界的实验中很少有证据表明在线设置中的干扰。我们通过描述两个案例研究来填补这一空白，这些案例研究捕捉了显著的网络效应，并突出了这个实验框架的价值。
We describe our network experimentation framework, deployed at Facebook, which accounts for interference between experimental units. We document this system, including the design and estimation procedures, and detail insights we have gained from the many experiments that have used this system at scale. In our estimation procedure, we introduce a cluster-based regression adjustment that substantially improves precision for estimating global treatment effects, as well as a procedure to test for interference. With our regression adjustment, we find that imbalanced clusters can better account for interference than balanced clusters without sacrificing accuracy. In addition, we show that logging exposure to a treatment can result in additional variance reduction. Interference is a widely acknowledged issue in online field experiments, yet there is less evidence from real-world experiments demonstrating interference in online settings. We fill this gap by describing two case studies that capture significant network effects and highlight the value of this experimentation framework.
通过地理聚类的随机实验设计
DOI：
10.1145/3292500.3330778
发表时间：
2019
期刊：
ACM
影响因子：
--
作者：
Rolnick, David;Aydin, Kevin;Pouget-Abadie, Jean;Kamali, Shahab;Mirrokni, Vahab;Najmi, Amir
通讯作者：
Najmi, Amir
影响因子：
1.4
作者：
Chin, Alex
通讯作者：
Chin, Alex