• What did Congress mean by “established by the State”?

    Over at the Volokh Conspiracy, Jonathan Adler, one of the architects of the exchange litigation, has posted a thoughtful response to a post I wrote defending the extension of tax credits to those purchasing health plans on federally established exchanges. You should read it; he’s really sharp.

    But he’s still wrong. Adler argues that I haven’t offered a satisfactory explanation for why Congress used the phrase “an Exchange established by the State” in the tax-credit calculation. He rejects my suggestion that the best way to understand the phrase is that it was a shorthand for exchange, whoever happened to establish it. “When Congress wanted to use a shorthand for ‘exchange,’” he writes, “it did just that— and said ‘exchange.’” As Adler sees it, the use of “by the State” in the tax-credit calculation must serve some distinct purpose.

    In making this claim, Adler is invoking two hoary canons of statutory interpretation: the canon against surplusage and the canon of consistent usage. These are useful canons—usually. But neither can help make sense of Congress’s meaning unless we can safely assume that Congress carefully used “exchange” when it meant “any exchange” and “exchange established by the State” when it meant “only state exchanges.”

    We can’t assume that here. Elsewhere in the statute, Congress referred to state-established exchanges when it clearly meant exchanges more generally. The ACA, for example, limits who can buy insurance on an exchange to those who “resid[e] in the State that established the Exchange.” Read literally, this would prohibit anyone in states with federal exchanges from buying insurance on those exchanges. Federal exchanges would be useless. That can’t be what Congress meant.

    Similarly, the ACA says that states have to maintain their Medicaid eligibility standards until “an Exchange established by the State” is up and running. This provision was meant to provide stopgap protection for Medicaid beneficiaries until the exchanges went live. But, read literally, it would forbid a state that declined to establish an exchange from ever relaxing its Medicaid standards. Again, that’d be batty.

    So Congress wasn’t meticulous about its references to state-established exchanges. At times, it did use “Exchange established by the State” as a shorthand for exchange. If that’s true elsewhere in the statute, it may be equally true when it comes to calculating tax credits. And reading the “established by the State” language to allow tax credits on federal exchanges makes much better sense of the statute as a whole.

    Can I be completely, absolutely confident about Congress’s meaning here? No. Without question, the statute is a bit of a mess. What do you expect? It’s a big statute, drafted by a lot of different people working under immense pressure.

    But here’s the thing. Adler can’t be completely confident in his interpretation either. At a minimum, Congress’s inconsistent use of the phrase “established by the State” gives rise to an ambiguity as to its meaning. And when you’ve got an ambiguity, it’s up to the agencies charged with interpreting the ACA to resolve that ambiguity. The tie goes to the government.


    Comments closed
  • Too few kids get the HPV vaccine. Did you all not watch my video?

    From the CDC:

    To assess progress with HPV vaccination coverage among adolescents aged 13–17 years,* characterize adherence with recommendations for HPV vaccination by the 13th birthday, and describe HPV vaccine adverse reports received postlicensure, CDC analyzed data from the 2007–2013 National Immunization Survey-Teen (NIS-Teen) and national postlicensure vaccine safety data among females and males. Vaccination coverage with ≥1 dose of any HPV vaccine increased significantly from 53.8% (2012) to 57.3% (2013) among adolescent girls and from 20.8% (2012) to 34.6% (2013) among adolescent boys. Receipt of ≥1 dose of HPV among girls by age 13 years increased with each birth cohort; however, missed vaccination opportunities were common. Had HPV vaccine been administered to adolescent girls born in 2000 during health care visits when they received another vaccine, vaccination coverage for ≥1 dose by age 13 years for this cohort could have reached 91.3%. Postlicensure monitoring data continue to indicate that HPV4 is safe. Improving practice patterns so that clinicians use every opportunity to recommend HPV vaccines and address questions from parents can help realize reductions in vaccine-preventable infections and cancers caused by HPV.

    Still, too few kids are getting the HPV vaccine. I could write another post on this, but I’ve got Healthcare Triage instead. Watch, and spread the word!


    Comments closed
  • My one comment on Gruber and Halbig

    I said it on Twitter, but I’ll expand here.

    Is there a single CBO analysis which documents what would happen if states refused to set up exchanges and would therefore “lose” their subsidies? Were any of Gruber’s models set up in this manner? If not, then I don’t understand how anyone in Congress or who set up the law thought it was going to work that way.

    I accept that the law was written poorly. I accept that there may be individuals who thought it would work in the way the DC Circuit majority said. But there are tons of analyses, reports, interviews, and more that show that no one involved thought that way.

    This blog has been going since before reform was passed. Find me a single post where we discussed this. Find a post where we discussed others discussing this. Do you think we would have ignored this? We wouldn’t have been concerned?


    Comments closed
  • Methods (kinda): Rubin on Rubin and Campbell

    Yesterday I encouraged you to read at least the paper by Stephen West and Felix Thoemmes if not all the papers on Campbell’s and Rubin’s causal frameworks in this 2010 issue of Psychological Methods. I also encourage you to read the response by Rubin. It’s much shorter and so much fun. Here are my highlights.

    Because my doctoral thesis was on matched sampling in observational studies under Cochran, I thought that I understood the general context fairly well, and so I was asked by the Educational Testing Service to visit Campbell at Northwestern University in Evanston, Illinois, which is, incidentally, where I grew up. I remember sitting in his office with, I believe, one or two current students or perhaps junior faculty. The topic of matching arose, and my memory is that Campbell referred to it as “sin itself” because of “regression to the mean issues” when matching on fallible test scores rather than “true” scores. I was flabbergasted!

    Rubin later showed that he was correct about matching but that Campbell was not wrong because Rubin had misunderstood his objection.

    Of course, the situation with an unobserved covariate used for treatment assignment is far more complex, and that situation, coupled with the naive view that matching can fix all problems with nonrandomized studies, appears to have been the context for Campbell’s comment on matching.

    (I may put up a methods post on matching at some point, though I haven’t decided.)

    The drive for clarity in what one is trying to do expressed in this passage resonates deeply:

    Perhaps because of my physics background, it seemed to me to make no sense to discuss statistical methods and estimators without first having a clear concept of what one is attempting to estimate, which, I agree with Shadish (2010), was a limitation of Campbell’s framework. Nevertheless, Campbell is not alone when implicitly, rather than explicitly, defining what he was trying to estimate. A nontrivial amount of statistical discussion (confused and confusing to me) eschews the explicit definition of estimands. [...] My attitude is that it is critical to define quantities carefully before trying to estimate them.

    Elsewhere in the paper, Rubin reveals that even Campbell did not think very highly of his own ability to do math. Rubin studied physics with John Wheeler at Princeton, which one can’t do without a lot of math ability and confidence in it.

    Later in the paper he has a very nice discussion of the stable unit treatment value assumption (SUTVA), which I won’t repeat here. Very roughly, the aspect of it that’s relevant below is that there be one treatment (or at least a clearly defined set of them), not a vague, uncountable cloud of them. (See also, Wikipedia.) It’s due to this assumption that the problem of, say, the causal effect of gender on wages is “ill defined,” as I raised in my prior post.

    For example, is the statement “She did well on that literature test because she is a girl” causal or merely descriptive? If [being assigned to the "control" group] means that this unit remains a girl and [being assigned to the "treatment" group] means that this unit is “converted” to a boy, the factual [the outcome from assignment to "control"]  is well defined and observed, but the counterfactual [outcome due to "treatment"] appears to be hopelessly ill-defined and therefore unstable. Does the hypothetical “converted to a boy” mean an at-birth sex-change operation, or does it mean massive hormone injections at puberty, or does it mean cross-dressing from 2 years of age, and so forth? Only if all such contemplated hypothetical interventions can be argued to have the same hypothetical [outcome] will the requirement of SUTVA that there be no hidden versions of treatments be appropriate for this unit.

    But this does not mean there can be no well-defined study of the causal effects of gender.

    An example of a legitimate causal statement involving an immutable characteristic, such as gender or race, occurs when the unit is a resume of a job applicant sent to a prospective employer, and the treatments are the names attached to the resume, either an obviously Anglo Saxon name ["control"] or an obviously African American name ["treatment"].

    They key here is that though you can’t in a reasonably defined, unique way imagine changing the gender of a person, you can imagine changing the gender as listed on a person’s resume.

    Later still, Rubin explains how, before his work, the “observed outcome notation” that had been the norm made it impossible to be clear how and why certain designs permit unbiased estimates. You really have to read the paper (at least) to see this. I’m still not sure I get it, but I believe him!

    To repeat, using the observed outcome notation entangles the science [all the potential outcomes and observable factors] and the assignments [the mechanism by which observed outcomes are selected among potential ones]—bad! Yet, the reduction to the observed outcome notation is exactly what regression approaches, path analyses, directed acyclic graphs, and so forth essentially compel one to do. For an example of the confusion that regression approaches create, see Holland and Rubin (1983) on Lord’s paradox or the discussion by Mealli and Rubin (2003) on the effects of wealth on health and vice versa. For an example of the bad practical advice that the directed acyclic graph approaches can stimulate, see the Rubin (2009) response to letters in Statistics in Medicine. [...]

    To borrow Campbell’s expression, I believe that the greatest threat to the validity of causal inference is ignoring the distinction between the science and what one does to learn about the science, the assignment mechanism—a fundamental lesson learned from classical experimental design but often forgotten. My reading of Campbell’s work on causal inference indicates that he was keenly aware of this distinction.

    (I may read and then post on Lord’s paradox. I don’t know what it is yet.)


    Comments closed
  • Me! On Vox! Interviewed by Adrianna!

    Over at Vox, Adrianna (whom we miss and hope will come back to TIE soon) has written up an interview with me about common misconceptions relating to Halbig. My favorite bit:

    AM: What do you make of the D.C. Circuit’s use of other examples of “bad policy” within the Affordable Care Act — the problems in Guam and the U.S. territories, for example — to suggest that this might just be another example of Congress not thinking through the consequences of their legislation?

    NB: Well, it’s very strange to point to a portion of the law that’s not working, and say that because it doesn’t work well, other portions of the law might not work well, so we’re relieved of any responsibility to interpret the statute in a manner that could make it work well.

    I would have thought that the best approach would be to try to make the statute work as well as possible given the constraints of the statutory text, and not to spike the legislation out of a misplaced fidelity to a piece of legislative text shorn of context.

    Go read the whole thing! Or just check out Adrianna’s take on Jon Stewart’s rant about the decisions. It’s pretty much the same thing but with jokes.


    Comments closed
  • Methods: Flavors of validity plus a ton of bonus content

    Before I get to the main subject of this post, I want to encourage you to read in full the papers about the frameworks and methods of Campbell and Rubin in this 2010 issue of Psychological Methods. (If you only have time to read one, I recommend that by Stephen West and Felix Thoemmes.) The papers cover a wide range of issues pertaining to causal inference in experimental and observational study designs. To my eye, they do so very well and with almost no math. (I illustrate the style of math used below.)

    Though there are a number of differences and similarities between Campbell’s and Rubin’s frameworks, a few are emphasized:

    • Campbell put greater emphasis on employing study design to mitigate threats to validity (about which more below). Rubin emphasized statistical methods to remedy defects that threaten validity.
    • Campbell’s framework focused more on the direction of causal effects. Rubin was as concerned with their magnitude as well.

    Now, about “validity” and its various types, Stephen West and Felix Thoemmes wrote,

    We designate X as an indicator of treatment (e.g., 1 = Treatment [T]; 0 = Control [C]) and Y as the outcome (dependent) variable. The central concern of internal validity is whether the relationship between the treatment and the outcome is causal in the population under study. Does the manipulation of X produce change in Y? Or, does some other influence produce change in Y? Note that internal validity does not address the specific aspect(s) of the treatment that produce the change nor the specific aspect(s) of the outcome in which the change is taking place—nor does it address whether the treatment effect would hold in a different setting, with a different population, or at a different time. These issues are questions of construct validity and external validity, respectively.

    Granted, that was a bit rushed, so here’s William Shadish’s take on flavors of validity:

    1. Statistical conclusion validity: The validity of inferences about the correlation (covariation) between treatment and outcome.

    2. Internal validity: The validity of inferences about whether observed covariation between A (the presumed treatment) and B (the presumed outcome) reflects a causal relationship from A to B, as those variables were manipulated or measured.

    3. Construct validity: The validity with which inferences can be made from the operations in a study to the theoretical constructs those operations are intended to represent.

    4. External validity: The validity of inferences about whether the cause – effect relationship holds over variation in persons, settings, treatment variables, and measurement variables.

    Originally, Campbell (1957) presented 8 threats to internal validity and 4 threats to external validity. The lists proliferated, although they do seem to be reaching an asymptote: Cook and Campbell (1979) had 33 threats, and Shadish et al. (2002) had 37.

    That’s about all I care to post/quote about validity. (As with all my methods posts, you should read the papers or a textbook for details.) Now for some bonus, though related, coverage of some of the contents of two papers in that Psychology Methods issue.

    Stephen West and Felix Thoemmes conveyed the setup of Rubin’s causal model as follows:

    Formally, each participant’s causal effect, the individual treatment effect, is defined as YT(u) – YC(u), where YT(u) represents the response Y of unit u to treatment T, and YC(u) represents the response of unit u to treatment [or control] C. Comparison of these two outcomes provides the ideal design for causal inference. [...] Unfortunately, this design is a Platonic ideal that can never be achieved in practice.

    Why? Because for each individual unit, u, we never know the effects of both treatment arms, T and C under precisely identical conditions. We only observe, at most, one. The other (or some estimate of it) must be inferred by other means. This is the entire problem of causal inference.

    The model makes it clear that we can observe two sets of participants: (a) Group A given T and (b) Group B given C. A and B may be actual pre-existing groups (e.g., two communities) or they may be sets of participants who have selected or have been assigned to receive the T and C conditions, respectively. Of key importance, we also need to conceptualize the potential outcomes in two hypothetical groups: (c) Group A given C and (d) Group B given T. Imagine that we would like to compare the mean outcome of the two treatments. Statistically, in terms of the ideal design what we would ideally like to have is an estimate of either μT(A) – μC(A) or μT(B) – μC(B) [actually, ideally, both] where A and B designate the group[s] to which the treatment [and control, respectively] was given. Both Equations [] represent average causal effects. Of importance, note that [they] may not represent the same average causal effect; Groups A and B may represent different populations. [...]

    [W]hat we would like to estimate is a weighted combination λ[μT(A) – μC(A)] + (1 –λ)[μT(B) – μC(B)], where [...] λ is the proportion of the population that is in the treatment group. [...]

    What we have [from study data] in fact is the estimate of μT(A) – μC(B). [...]

    For observed outcomes, only half of the data we would ideally like to have can be observed; the other half of the data is missing. This insight allows us to conceptualize the potential outcomes as a missing data problem and focuses attention on the process of assignment of participants to groups as a key factor in understanding problems of selection.

    Basically, the entire enterprise of causal inference is to design and employ methods to better estimate (in the sense of minimizing the threats to validity defined above) the unobserved counterfactual means μC(A) and/or μT(B) or, what amounts to the same thing, their difference from those that are observed.

    I found the following fascinating:

    The researcher will need to conceptualize carefully the alternative treatment that the individual could potentially receive (i.e., compared with what?). Of importance [] this definition makes it difficult to investigate the causal effects of individual difference variables because we must be able to at least conceptualize the individual difference (e.g., gender) as two alternative treatments. If we cannot do this, Rubin (1986) considers the problem ill defined.

    Wow! So, for example, Rubin (1986) would consider the problem of the causal effect of gender on, say, wages “ill defined” because there is no conceivable possibility of a male being female or vice versa in the same sense in which someone can take a drug or not. I probably don’t need to point out that the causal effect of gender on wages is a significant research question of considerable cultural, policy, and political import at the moment. What exactly it means for it to be “ill defined” I don’t know, though I could speculate. But I’ve downloaded Rubin (1986) and one day I may read it and find out.

    Here are a couple of passages I highlighted in the paper by William Shadish:

    [Campbell] is skeptical of the results of any single study, encouraging programs of research in which individual studies are imbued with different theoretical biases and, more important, inviting criticisms of studies by outside opponents who are often best situated to find the most compelling alternative explanations.

    Endorse! Also,

    The regression discontinuity design [] was invented in the 1950s by Campbell (Thistlewaite & Campbell, 1960), but a statistical proof of its unbiased estimate was provided by Rubin (1977) in the 1970s (an earlier unpublished proof was provided by Goldberger, 1972).

    This I did not know. I may post more on the content of a few other papers in the collection. I’m still working my way through them.


    Comments closed
  • JAMA Forum: CBO Report Brings Some Good News About Health Care Spending

    The latest CBO forecast on the budget and the deficit is the most optimistic I’ve seen in a long, long time, and it’s mostly due to healthcare spending. I discuss it in my latest post over at the JAMA Forum.

    Go read!


    Comments closed
  • Stand Up! – July 23, 2014

    I am a frequent guest on Stand Up! with Pete Dominick, which airs on Sirius/XM radio, channel 104 from 6-9AM Eastern. It immediately replays on the channel, so those on the West Coast can listen at the same times. Eric Segall, a Georgia State University College of Law Professor, was on with me.

    Today it was all-Halbig. Great discussion. I encourage you to listen!

    You can play the audio right here, after the jump…

    @aaronecarroll Read the rest of this entry »

    Comments closed
  • Halbig said it was applying the law as written. Don’t believe it.

    In the Wall Street Journal this morning, Jonathan Adler and Michael Cannon, the two principal architects of the exchange litigation, have an op-ed about Halbig and King. They write that “[t]he president’s defenders often concede that he is doing the opposite of what federal law says. Yet he claims that he is merely implementing the law as Congress intended.”

    The not-so-implicit suggestion is that Adler and Cannon are the staunch defenders of the law, and that those who disagree with them just ignore the law when they think Congress didn’t really mean what it said. That suggestion pervades the debate over the exchange litigation: either you apply the law as written or you just don’t care about the law.

    Let me be blunt: the suggestion is false. Those who disagree with Adler and Cannon care every bit as much about the law as they do. They just don’t buy their artificially cramped understanding of how to figure out what the law is.

    To understand the legal fight, keep in mind that words don’t have meaning in a vacuum. Words have meaning only because they mean something to those who speak and to those who listen. They’re communication devices. (As Ludwig Wittgenstein famously explained, you can’t have a private language. Language is public or it’s not really language at all.)

    So if you’re trying to make sense of a statute, the question for an interpreter is what Congress meant to communicate by the words that it chose. Usually, that’s easy to figure out. Words are not infinitely malleable. They have meaning in our linguistic community. And it’s a good rule of thumb that Congress means what it says and says what it means.

    But not always. Sometimes a statute uses words that don’t track what Congress meant. Sometimes that’s because Congress made a mistake. But more often, loose language fails to get corrected because, when taken in the context of the statute as a whole, the meaning of the statutory text is pretty clear.

    This is not an especially controversial point. Words always accrue meaning from context, and that context can affect the meaning of the words that the speaker selects to convey that meaning. If my wife says to me, “Do you mind taking out the garbage?” and I say “yes,” even as I dutifully take out the garbage, it’s clear from context that by yes I meant no. And yelling “fire!” means something very different in a crowded theater than it does at a firing squad.

    So too with statutes. Text really is the best guide to meaning. But sometimes the broader statutory context demonstrates that Congress meant to convey something very different than what a literal construction of an isolated snippet of a statute might suggest.

    Now, one might adopt the view—and I take this to be something like the D.C. Circuit’s view in Halbig—that courts are really bad at figuring out the meaning of words from their broader context. They can too easily justify whatever outcome they prefer by referring some amorphous “context.” In the face of uncertainty, maybe courts do better if they blind themselves to that broader context and mechanically apply the statute’s literal terms.

    But I don’t think that’s an attractive way of making sense of what Congress meant by the language that it chose. Statutory drafters aren’t writing a computer program that will be interpreted by a machine. They’re human beings using words to communicate meaning to other human beings. Why not use every contextual tool available to understand what Congress was driving at, much in the same way we labor to understand what people really mean when they speak to us?

    This case showcases the hazards of a myopic approach to statutory interpretation. The plaintiffs’ claim is that Congress meant to withdraw subsidies from people in states with federal exchanges because doing so would encourage states to set up exchanges. Congress was basically making a threat: set up your own exchanges or you’ll lose out on tax credits.

    But it’s easy, from context, to demonstrate that Congress didn’t mean anything of the kind. For threats to work, they have to be communicated. When Vito Corleone made the proverbial offer that can’t be refused, he didn’t just say “sign the contract.” He had Luca Brasi hold a gun to the head of a guy and “assured him that either his brains or his signature would be on the contract.” Without the gun, there’s no threat.

    In the ACA, it would have been simple for Congress to threaten the states. But it didn’t. Instead, it crafted a statute that gave the states the option to set up their own exchanges and used federal exchanges as a backstop. In context, that’s all it did. It’s outlandish to think that Congress meant to threaten the states it inserted the innocuous phrase “established by the State under 1311” into the complex calculation of the tax credit. Why? Because the supposed threat was so well-hidden that the states didn’t even notice it. No gun, no threat.

    So if the “established by the State under 1311” language wasn’t an implicit threat, what was it? In context, Congress used the phrase as shorthand for an exchange, whether established by the state (the default) or the federal government (the fallback). In his concurring opinion in the Fourth Circuit, Judge Davis makes the point beautifully:

    If I ask for pizza from Pizza Hut for lunch but clarify that I would be fine with a pizza from Domino’s, and I then specify that I want ham and pepperoni on my pizza from Pizza Hut, my friend who returns from Domino’s with a ham and pepperoni pizza has still complied with a literal construction of my lunch order. That is this case: Congress specified that Exchanges should be established and run by the states, but the contingency provision permits federal officials to act in place of the state when it fails to establish an Exchange. The premium tax credit calculation subprovision later specifies certain conditions regarding state-run Exchanges, but that does not mean that a literal reading of that provision somehow precludes its applicability to substitute federally-run Exchanges or erases the contingency provision out of the statute.

    In its decision yesterday, the D.C. Circuit draped itself in the mantle of legislative supremacy: “Within constitutional limits, Congress is supreme in matters of policy, and the consequence of that supremacy is that our duty when interpreting a statute is to ascertain the meaning of the words of the statute duly enacted through the formal legislative process.”

    I wholeheartedly agree that the point of statutory interpretation is “to ascertain the meaning of the words of the statute.” But I deplore a method of statutory construction that leads to a result so manifestly at odds with what Congress actually meant.


    Comments closed
  • Podcast on Halbig and King

    For those who want to hear more about Halbig and King, the Federalist Society hosted a podcast yesterday where Jonathan Adler and I discussed the cases and their potential fallout. Adler is one of the architects of the lawsuits and, although we disagree on the merits, he’s both fair and thoughtful about the cases. I couldn’t wish for a better sparring partner.


    Comments closed