Peter Kramer is a psychiatric educator and the author of the widely-read Listening to Prozac. In a recent New York Times editorial, he argues for the value of stories, including case studies, as a genre of medical literature. Unfortunately, his essay doesn’t convey the difference between anecdotes and evidence.
But first, Kramer gets important things right.
A story narrates a sequence of events driven by the actions of one or more characters. We are built — perhaps even wired — to understand the world through stories. Stories can explore how medicine links to larger human concerns. Through stories we can imaginatively connect to the experiences of others, identify with their suffering, and locate their needs for care.
Stories also matter in the practice of medicine. Doctors tell stories to teach other physicians, to brief colleagues on a case, and to inform patients about a possible course of care so that decision making can be shared. Learning to tell good stories — to be concise yet complete, to be concrete yet vivid, to motivate action while cautioning about uncertainty — is a critical skill for care givers. Kramer describes how stories about cases are the only source of guidance when he reaches the end of medical knowledge and
…I exhaust the guidance that [clinical] trials can give — and then I consult experts who tell me about this case and that outcome.
Finally, important clinical advances often begin with an observation about a case.
[V]ignettes can do more than illustrate and reassure. They convey what doctors see and hear, and those reports can set a research agenda.
So far, so good. But Kramer goes off the rails when he seeks to rehabilitate stories as a kind of medical evidence. He argues that in the case of modern antidepressants.
[D]octors had not waited for controlled trials. In advance, the [hypothesis that Prozac made you “better than well”] had served as a tentative fact. Treating depression, colleagues looked out for personality change, even aimed for it. Because clinical observations often do pan out, they serve as low-level evidence — especially if they jibe with what basic science suggests is likely… this approach, giving weight to the combination of doctors’ experience and biological plausibility, stands somewhat in conflict with the principles of evidence-based medicine.
In the absence of evidence, doctors must decide based on experience and biological plausibility. But Kramer doesn’t convey why evidence-based medicine deprecates anecdotes as evidence. Kramer seems to have forgotten the harms that can occur when clinical stories are taken as evidence.
Siddhartha Mukherjee’s The Emperor of All Maladies documents the risks associated with medicine by story. The great surgeon William Stewart Halsted (1852-1922) had a biologically plausible belief — confirmed, he thought, by his experience with cases — that extensive removal of tissue surrounding a tumour was necessary to cure breast cancer. He taught the procedure to the next generation of surgeons and for decades, hundreds of thousands of women were radically disfigured by removal of their breasts and extensive tissue in their chests. A clinical trial published in 1985 showed no benefit for radical compared to conservative surgeries.
This isn’t just a 19th century problem: Read Mukherjee’s history of bone marrow transplantation and high dose chemotherapy for metastatic breast cancer, also covered on TIE. Again, a treatment entered practice far ahead of the completion of clinical trials and again, tens of thousands of women suffered from a painful and expensive treatment that lacked benefit.
Medicine by case study has been a particular menace in mental health care.
I began working in mental health after college, as a therapeutic child care worker at a children’s mental health institute that was based on the ideas of Bruno Bettelheim. Bettelheim is now forgotten, but in the 1970s he was a renowned psychoanalyst and public intellectual. There were many therapists like him in that era: charismatics with compelling stories about rescue through psychotherapy.
Bettelheim published highly influential books reporting cases in which he cured autism through psychotherapy. His theory of autism attributed the disorder to the emotional frigidity of the mother. This made sense in the light of the psychoanalytic theory of the day, itself built on case studies.
We know now that Bettelheim’s therapy was useless, that his theory was absurd, and that the man himself was a charlatan who may have abused his patients. Most importantly, we now validate psychotherapies through clinical trials, like any other medical intervention. The age of charismatic therapists is over and we have modest but realistic estimates about what psychotherapy can achieve.
So why do stories fall short as sources of trustworthy medical evidence? Case histories often lack the basic elements of science: objective measurement, public demonstration of effect, and replicability.
But the most important problem with a case study is epistemic: it cannot carry information about causality. An action is a cause if doing it leads to an outcome and not doing it would not. So to make an inference about causation we need more than a biologically plausible theory. We also need information about would have happened to a patient not only in the factual case where the treatment occurred but also in the counterfactual case where it did not. Valid clinical studies arrange observations so that we can make inferences about both the factual and counterfactual cases, and thereby infer causality. But a case history can present only the factual side. The case study makes sense to us because we supply the belief about the causal linkage. It doesn’t provide independent evidence about that link.
So, medical humanists and ordinary caregivers must tell stories and yet stay in the realm of science. How? A challenge of writing strong science narratives is to spot the implicit assertions of causality and make sure that they are consistent with scientific evidence, wherever possible. And otherwise, to convey to the reader the uncertainty about causality.
TIE has many posts on causality.
