Ken Covinksy with a comprehensive comment on a recent NEJM study on preventing Chronic Obstructive Pulmonary Disease (COPD) exacerbations. This randomized control trial compared two medicines to assess which performed best. COPD is the fourth leading cause of death, causes a great deal of morbidity for patients, and is a particularly expensive condition for the Medicare program. The problem with the study from Covinsky’s perspective is that it focuses on relatively young patients with COPD (mean age 63). In fact, he argues the exclusion criteria of the trial limits the usefulness (generalizability) of the study. Three of the exclusion criteria with his comments
- Exclusion: “Significant diseases other than COPD, i.e. disease or condition which, in the opinion of the investigator, may have put the patient at risk because of participation in the study or may have influenced either the results of the study or the patients’ ability to participate in the study” (Comment: Yes. Co-existing conditions may influence the outcomes of treatment for COPD. But real world patients have more than one disease. Exluding them from a study makes the study of questionable relevance to real world practice)
- Exclusion: Patients with a known symptomatic prostatic hyperplasia or bladder neck obstruction. (Comment: Seems this would exclude most men over the age of 70, and virtually all over the age of 80.).
- Exclusion: Patients with moderate or severe renal insufficiency (Creatine clearance less than 50 ml/min). (Comment: You’ve got to be kidding)
He concludes with this comment:
These issues, especially the exclusions, prevent this study from informing the care of the vast majority of older persons with COPD (and as a result, the majority of all persons with COPD). This is unfortunate, because knowing which of these drugs should be used as first line therapy in older persons with COPD could have resulted in significant improvements in care. It is time for our leading medical journals, funding agencies, regulatory agencies, and the public to insist that clinical research be conducted in a way that signficantly informs the care of older patients.
He is calling for increased focus on building the concept of external validity into the development of clinical trials. Further, Covinsky notes the study focused on measures of lung function in isolation from general mobility, which are more commonly of direct concern for patients. I wrote about the need for broader external validity in the context of palliative care research last week. I think lack of generalizability due to the study population not being representative is the most commonly misunderstood problem with published medical/health policy research.
Update: I initially used the term selection bias in last sentence when I meant to say lack of generalizability.
