• COPD and Generalizability

    Ken Covinksy with a comprehensive comment on a recent NEJM study on preventing  Chronic Obstructive Pulmonary Disease (COPD) exacerbations.  This randomized control trial compared two medicines to assess which performed best. COPD is the fourth leading cause of death, causes a great deal of morbidity for patients, and is a particularly expensive condition for the Medicare program.  The problem with the study from Covinsky’s perspective is that it focuses on relatively young patients with COPD (mean age 63).  In fact, he argues the exclusion criteria of the trial limits the usefulness (generalizability) of the study.  Three of the exclusion criteria with his comments

    • Exclusion: “Significant diseases other than COPD, i.e. disease or condition which, in the opinion of the investigator, may have put the patient at risk because of participation in the study or may have influenced either the results of the study or the patients’ ability to participate in the study” (Comment: Yes. Co-existing conditions may influence the outcomes of treatment for COPD. But real world patients have more than one disease. Exluding them from a study makes the study of questionable relevance to real world practice)
    • Exclusion: Patients with a known symptomatic prostatic hyperplasia or bladder neck obstruction. (Comment: Seems this would exclude most men over the age of 70, and virtually all over the age of 80.).
    • Exclusion: Patients with moderate or severe renal insufficiency (Creatine clearance less than 50 ml/min). (Comment: You’ve got to be kidding)

    He concludes with this comment:

    These issues, especially the exclusions, prevent this study from informing the care of the vast majority of older persons with COPD (and as a result, the majority of all persons with COPD). This is unfortunate, because knowing which of these drugs should be used as first line therapy in older persons with COPD could have resulted in significant improvements in care. It is time for our leading medical journals, funding agencies, regulatory agencies, and the public to insist that clinical research be conducted in a way that signficantly informs the care of older patients.

    He is calling for increased focus on building the concept of external validity into the development of clinical trials. Further, Covinsky notes the study focused on measures of lung function in isolation from general mobility, which are more commonly of direct concern for patients.  I wrote about the need for broader external validity in the context of palliative care research last week. I think lack of generalizability due to the study population not being representative is the most commonly misunderstood problem with published medical/health policy research.

    Update: I initially used the term selection bias in last sentence when I meant to say lack of generalizability.

    • Not that I want to brag, but in my specialty, Anesthesiology, we have learned the value of functional status in evaluating patients. It has changed the way we evaluate patients for surgery. We have learned that patients with good functional status usually do not need lots of other tests to prepare them for surgery. I wish that other specialties would include that in their studies. It would have been helpful, I believe, in this particular study to correlate functionality with other measures of improvement. I think this is one way that physicians can contribute to holding down costs, ie, by following functional indicators when possible.


    • I understand and agree with the main point you are making about generalizability. In repeating Ken Covinksy criticism of the randomized trial of tiotropium and another agent in a recent NEJM article, you may have picked a poor example.

      Covinksy listed three exclusion criteria to which he took exception. The first was presence of a significant co-existing disease. However, simple existence of a co-existing disease wasn’t the exclusion criterion. It was the existence of disease that the investigator believed may have put the patient at risk. Certainly that’s parallel to real-life experience. No physician will prescribe medication he believes may put his patient at risk.

      The second exclusion was the existence of symptomatic prostatic hyperplasia. Covinksy believes that would exclude nearly all men over 70, but the exclusion is for SYMPTOMATIC disease. Note this from the tiotropium package insert: Worsening of urinary retention may occur. Use with caution in patients with prostatic hyperplasia or bladder-neck obstruction and instruct patients to consult a physician immediately if this occurs.

      The third exclusion was moderate or severe renal insufficiency. Again, from the package insert: Patients with moderate to severe renal impairment should be monitored closely for potential anticholinergic side effects