Perhaps you’re of the mind that the only way to learn anything of value is by randomized controlled trial (RCT). I disagree with that position. For some things, RCTs are the right approach. But I think the topics of study that can’t benefit from some sound observational study, perhaps in preparation for a future RCT, are few. And then there are many topics and questions that can’t be studied by RCT due to methodological, ethical, practical, or financial considerations. So, there is plenty of room for observational studies and, given that, one ought to use the best available techniques, including instrumental variables (IV).
However, even if one only wishes to do RCTs, IV can assist.Very often the perfect randomization contemplated by the researcher is compromised. Some assigned to the treatment group don’t comply. Some assigned to the control group receive treatment. When randomization breaks down things get messy, and it may not be clear what can be learned. Colleagues and I faced this very problem on a study of the Community Nursing Organization Demonstration years ago. We solved it by analyzing the randomized groups using the notion of “intent to treat” (ITT), essentially ignoring the contamination of treatment and control groups, arguing that it is akin to what would happen in the real world anyway. We supplemented the analysis by comparing those treated to a comparison group not involved in the study.
But we could have done something else, and I wish we had. We could have considered the random assignment as an instrument for actual receipt of treatment. One has to admit, it is a very good instrument. It is highly correlated with treatment/control assignment (since most subjects comply) and it is not related to outcomes (which is the whole point of random assignment).
The math and statistics of this approach are very straight forward. It’s all explained in a 2006 paper by Angrist in the Journal of Experimental Criminology (and elsewhere). I won’t go into the details. Suffice it to say, even if you love RCTs and only RCTs, sooner or later you’ll come across one for which randomization has failed. In that case, IV can assist. The method is credible, sensible, and sound. Moreover, it fully exploits the beautiful properties of the randomness with which the RCT was designed.
What one obtains with such IV-corrected RCT analysis is an unbiased estimate of the causal effect of treatment on those whose treatment status was affected by randomization (called “compliers”). This estimate is known as the local average treatment effect (LATE). In the case of an RCT of a therapy that can’t be obtained outside the experiment and in which no individual in the control group received treatment (so that all individuals who received treatment did so due to randomization and would not have otherwise), one can obtain the IV-corrected treatment effect by dividing the ITT treatment effect by the probability of treatment assignment compliance. In this simplified (but common) setting, this calculation also provides the average treatment effect on the treated (ATET). It is clear from this example that the ATET differs from an ITT estimate when compliance with treatment assignment is not perfect (ATET > ITT). Also, in this example, but not in general, the LATE is the ATET.
The key point is that the IV-corrected estimate is as valid, meaningful, and useful as the ITT estimate. It’s just an answer to a different question. ITT examines the effect of the intervention on a population, including that due to lack of compliance. IV techniques provide an estimate of the effect of treatment on those that comply (LATE). In the case for which compliance is one-sided (no one in the control group received treatment and everyone who was treated was randomized as such), LATE = ATET.
