Partial recovery of missing responses - a toolbox for efficient design and analysis when data may be missing not at random

数据丢失是许多应用领域普遍存在的问题。缺失值的存在使分析变得复杂，如果处理不当，可能会导致从数据中得出错误的结论。假设有一个生成缺失值的过程，通常称为缺失数据机制，这通常是有帮助的。一个特别有问题的场景是，这种机制在一定程度上是由其他一些未知变量决定的，例如缺失值本身。这被称为非随机缺失(Mnar)机制。如果由于Mnar机制而导致缺失值，则从数据中得出的结论通常是有偏差的。此外，重要的是，不可能知道数据中是否发生了此问题。这是本提案试图解决的具有挑战性的问题领域，即制定能够最好地测试数据中是否存在或Mnar的程序。该提案将考虑可能通过后续样本估计某些缺失值的情况。这样做的主要目的是了解丢失数据的机制，并具体测试Mnar假设是否有效。此外，恢复的数据还将有助于纠正缺失数据对结论的影响。该提案利用最优设计技术来决定哪些缺失的值需要跟进。从本质上讲，某些缺失的值可能比其他值产生更多有关缺失数据机制类型的信息；此外，某些值可能比其他值更有可能被恢复。通过这种方式，我们将确保从恢复的数据中获得最大信息。这将使数据分析人员能够确定是否可能出现Mnar并采取适当的行动。我们将与我们的项目合作伙伴，国家统计局和NHS血液和移植中心合作开发这些方法。我们的项目合作伙伴将为我们提供相关数据，供我们考虑现实情况，我们将与他们讨论中期结果，以确保我们的方法对从业者最有用。我们还将把这项工作作为非洲数学科学研究所(AIMS)缺失数据课程的一部分，以使这项工作的全球效益最大化。本提案中提出的方法将通过论文和演示文稿进行传播。此外，我们将创建一个免费使用的R包，它将实现这些方法，以便于用户使用。作为为期两天的研讨会的一部分，我们将提供使用这个R包的培训，在研讨会上我们将向用户描述我们的方法。将在整个项目期间更新一个专门的网站，以说明事态发展并促进与感兴趣的各方的接触。

Missing data are a common problem in many application areas. The presence of missing values complicates analyses, and if not dealt with properly can result in incorrect conclusions being drawn from the data. It is often helpful to assume there is a process that produces the missing values, typically called a missing data mechanism. A particularly problematic scenario is when this mechanism is in part determined by some other unknown variables, such as the missing values themselves. This is known as a missing not at random (MNAR) mechanism.If missing values arise due to a MNAR mechanism then conclusions drawn from the data will typically be biased. Also, importantly, it is not possible to know whether this problem occurs or not in the data. This is the challenging problem area that this proposal seeks to address, namely developing procedures that can best test whether or MNAR occurs in the data.The proposal will consider scenarios where it is possible to estimate some of the missing values through a follow up sample. The main purpose of this is to learn about the missing data mechanism and specifically test whether the MNAR assumption is valid or not. Further, the recovered data will also help to correct for the effect the missing data have on conclusions. The proposal makes use of optimal design techniques to decide which missing values to follow up. Essentially certain missing values might yield more information about the type of missing data mechanism than others; in addition some values might be more likely than others to be recovered. In this way we would ensure maximum information from the recovered data is obtained. This will allow data analysts to determine whether the presence of MNAR is likely and take appropriate action.We will collaborate with our project partners, the Office for National Statistics and NHS Blood and Transplant in the development of these methods. Our project partners will provide relevant data for us to consider realistic scenarios and we will discuss interim results with them to ensure our methods are most useful for practitioners. We will also present the work as part of a missing data course at the African Institute of Mathematical Sciences (AIMS) to maximise the global benefit of the work.The methods developed in this proposal will be disseminated through papers and presentations. In addition, we will create a free to use R package that will implement the methods to allow easy uptake by users. We will provide training in using this R package as part of a two-day workshop where we will describe our methods to users. A dedicated website will be updated throughout the project to describe developments and facilitate engagement with interested parties.