• Health Care Quality Illusions

    David Lazarus is a reporter on consumer issues for the LA Times. His family recently became big-time consumers of health when his wife spent a week in an intensive care unit. Coming out of this ordeal, he concluded that the

    US health system has flaws, but not in quality of care.

    This is reminiscent of Senator Ron Johnson’s Wall Street Journal editorial in 2009. Thinking about quality rather than price, and generalizing from the successful care that his daughter received, Senator Johnson argued that the US has

    the finest health-care system in the world.

    Senator McConnell similarly believes that

    we have the best health care in the world.

    These gentleman acknowledge that there are important concerns about the cost of health care and how families finance that cost. But leaving these concerns aside, they believe that the quality of US health care is superb.

    Aaron wrote a definitive series of posts on the quality of US healthcare (start here) and concluded that on most important measures, the US has poorer quality of care than comparable countries. If Aaron can’t persuade you on that point, there’s not much I can say.

    What I want to add, however, is that thinking about quality of care as a debate about whether the US is number 1 is too simple and unhelpful. We need to ask questions about quality in the right way to make real progress. Here are three questions to challenge how you think about quality of care.

    1. Is quality of health care a single dimension?

    Lazarus and the Senators are confident that we can discuss quality of health care on a single dimension. To the contrary, there are at least four dimensions of health care quality.

    • The hotel experience. Some hospitals are nicer places to stay than others. This may seem trivial in the context of life and death, but any hospital manager will tell you that ‘hotel quality’ matters to patients.
    • The relationship experience. Did the health care providers treat you with respect? Were they considerate of your religious beliefs? The well-educated readers of this blog may have difficulty imagining that they would not be well-treated by health care providers. But disrespect may be a primary consideration if you are poor, speak a language other than English, live outside the mainstream culture, or are mentally ill.
    • The rightness of treatments. By the ‘right’ treatment, I mean the one that was most likely to benefit you.
    • Safety. Were you harmed through error or neglect while you were in care?

    Because quality has multiple dimensions, it’s not clear how we should rank order countries (or hospitals, or doctors, etc). If A provides better hotels and more of the right treatments, but B treats patients more respectfully and delivers care more safely, who is better overall? To say that health care is better in one country than another, we must either identify which dimension we are talking about or, if we are thinking holistically about quality, we need to specify how we weight the dimensions to come up with an overall ranking. Aaron was writing about safety and getting the right treatments, but in most public debates it’s not clear what people are talking about.

    2. Can you judge the quality of care from personal experience?

    Lazarus and Senator Johnson judged health care quality based on their family experience. Should we take this testimony seriously? If we are talking about the hotel experience or the relationship experience, we certainly should. In these matters the patient or the family are the relevant authorities.

    If we are talking about the rightness of treatments, however, things are more complex.

    There are some aspects of care that any of us can judge. I’ve been cared for or worked in hospitals that were filthy and where I’ve seen insects that shouldn’t be there. Once, I laid in a hospital bed for 10 hours with a heart rate topping 200 beats per minute waiting for a cardiologist to see me. (This problem was resolved when my physician wife informed the staff that she was leaving my bedside and would return with a rocket launcher if they didn’t promptly find someone who knew what they were doing.)

    In most cases, however, it is impossible for someone to judge the rightness of treatment from his personal experience of care. This is not just because the issue is technical and requires professional knowledge. The bigger issue is that you can’t judge the rightness of treatment from personal experience, because rightness depends on a counterfactual. The right treatment is the one that was expected to produce outcomes that were better than the expected outcomes of alternative treatments. But we only have personal experience of the care that we actually received. By the way, your doctor, despite all of her training, is in the same fix you are. She can only see the outcome of the care you got, not the outcome of the care you might have gotten.

    The only way to judge the rightness of your treatment is to look at the outcomes of many people like you who got your treatment and compare them to many other people similar to you who got other treatments. This involves data, ideally collected through randomized trials. Expert medical opinion — unless it is based on this kind of evidence — is a very poor alternative to data. And personal testimony is worth nothing at all.

    3. Should we think about the quality of care comparatively, or in absolute terms?

    There are heated debates about where the US ranks in the quality of care (and I am with Aaron about where the US ranks). But really, I don’t think that this matters. I don’t care whether US surgical wound infection rates are worse than, say, the Swedes. What I want is that the US wound infection rate, whatever it is, should be cut in half. Instead of thinking comparatively, we should look at quality in absolute terms: how good is contemporary US care relative to how good it can be?  

    So here is how I would like to see discussion of quality change. First, recognize that quality has many important components. Second, the rightness of treatment can’t be evaluated from anyone’s testimony about their wife or daughter; or by the opinions of white-coated experts. You need to look carefully at the evidence.  Finally, let’s get past the question of ‘Is American First’? Even where America is first, what matters is that American health care could be better.


    • One more dimension, perhaps: I suspect that often when people think about U.S. “quality” (particularly with a chauvinistic bias), they’re thinking not so much of the appropriateness of treatment as of the likelihood of getting a difficult or expensive or cutting-edge treatment if you think you need it.

      A heightened possibility of getting such treatment may be statistically eclipsed on quality measures by frequent overtreatment, or poor safety procedures, or lack of access to decent care for large segments of the population. But I think it’s psychologically important to a lot of people — e.g. those with a fair amount of power in the patient role, that is, those with good insurance.

      • Agreed, Andrew. And as someone pointed out on Twitter, people sometimes think that technology = quality care. After all, they wouldn’t be using a freaking _proton_beam_ unless there was real SCIENCE behind it, right?

      • Let’s not forget that cutting edge technologies usually come with marketing campaigns, both to doctors and to patients. Part of the reason that patients expect the latest and greatest is because they’ve heard about it in the media (probably partly influenced by PR), they’ve seen ads, and their doctor has had a number of sessions with a marketing rep, which is the way doctors tend to get information.

    • Hi Bill
      So much to say about this. But I will limit myself for now. Within the healthcare world there exist at least two fundamental approaches to health care and with it two approaches to quality.
      1) technical care. This is the idea that what health care does is to provide technical solutions to ill health (science is the false rubric for this). High quality is the provision of the technical solution in a way that doesn’t further damage the individual or subject them to other harms.
      2) relational care. This is the idea that what health care can provide is a useful relationship that escorts and guides people through illness- and increasingly through health. The technical solutions are part of this approach but they are not the most highly valued (by this i mean esteemed). Rather its the combination of relationship based wisdom and support and care that is treasured.

      At this moment it’s clear that the technical approach is what’s most paid for. And it’s clear how ineffective it is beyond generating enormous costs for marginal (at best) gain when viewed from population health perspective. The call is out for new try at the relationship approach- health homes and in the world of commercialized relationships with branded organizations (what we in the psychiatric biz call institutional transference) ACOs.

      We will find out if they can gain any purchase and if the technology juggernaut is beginning to ebb. Given the limited role health care has in improving or protecting health- especially in environments where the social determinants of disease are more powerful then the health protection or healing offered by health care services

      So I would add to your list- any discussion of quality might usefully specify the theory of health care underlying the measures of health care quality

      • Ken,
        I like how you describe the relational model of care. Say more about how that affects our understanding of quality.

    • AVIP has a good experience during a hospitalization; therefore, America has the best healthcare in the world!
      Actually, before we draw any conclusions about the healthcare system, we should get some more information. The article in the LA paper does not tell us very much. Mrs. Lazarus is reported to have had a seizure due to hyponatremia. Very severe hyponatremia does not just happen. Often, it is associated with medications or iatrogenic events. Or is there an underlying and still undiagnosed neoplasm?
      A grateful husband’s testimonial is understandable in human terms, but it is not very useful evidence about the whole system.
      I am able to make these comments because I am the best oncologist in the world!

      • _You_ are the best oncologist on the planet? That’s terrific! If you have a second, let me ask you a question about my aunt…

        • Go ahead. I also have a trophy for world’s best dad given to me by my daughter.
          My point is that one episode of care that turned out well tells us almost nothing about the system as a whole, and Mr. Lazarus’ understandable gratitude for his wife’s recuperation can not be used as a marker of the entire US.

    • I strongly believe you need to talk about consistency when you talk about quality. On an aggregate basis, the existence of excellent care is not all that meaningful. You can find fantastic doctors and excellent care in most countries not noted for good medical systems, it’s a just crapshoot when you go to the hospital. This is why anecdotal data is a poor indicator.

      Hotel quality is something that is easy to measure and judge. I suspect many patients use it as a gauge of healthcare quality. Further, it is not priced. Without a price tag, all of us would rather stay in the Ritz-Carleton than Motel 6.

      Countries with cheaper health systems don’t have facilities like hotels. I spent a week in a NHS burn center. Utilitarian was the vibe, not hotel. They had put a cheapo trailer in the courtyard for classroom space for children who were stuck there long term. There was no atrium. The old Victorian hospital could have been the setting for a Dickens novel. However, this was a world-reknowned burn center.

      I’d rather stay at Motel 6 on the rare occasions that I am overnight in a hospital and pay half the price for insurance for the whole of my life. I’m an expat, and worldwide health insurance excluding the US and Canada for expats is half the price of worldwide health insurance including the US and Canada.

    • I have had the (mis)fortune of having to visit the ER both in the US and in Italy. In Italy, despite my limited language skills, I was seen within 10 minutes. In the US, it took 20 minutes just to clear the insurance checks. I also note that I was treated with more respect in Italy (where I was assigned an English-speaking doctor) than by the amazingly condescending resident I was seen by in the US. Based on my admittedly anecdotal experience, it’s hard for me to accept that we have the “best medical care in the world.”

    • Taking a similar line as SAO – the existence of good experiences does very little to prove [or disprove] a judgement about our healthcare system in general. I worked a long time in Marketing Research and after looking at a lot of qualitative and quantitative data I came to one conclusion. VARIANCE is always bigger than we think it is – or we determine via calculations. I think the same applies to healthcare.

      Doctors vary
      Care centers very
      Patients vary
      Treatments vary
      Treatment effects vary
      And with things like cancer – the disease varies

      All this leads to a pretty complex set of independent variables that in the end determine the dependent – Health Care Quality.

      The results one obtains are based on a pretty complex interaction – and maybe as much art as science.

      • LL,
        Totally agree with you message about how much variance there is. One place I observe this is in the overestimation of the importance of known, valid risk factors. Having a family of history of cancer increases your risk of cancer, but typically only by a few percent. How much should that affect your decision making, or your doctor’s? Not much, in my view.

        • Ignoring or limiting the discussion of “effect size” and it’s meaning is one of my other pet peeves. I love “The Cult of Statistical Significance” and thought it did a good job of pointing out the costs we are paying for being so focused on finding “Significance” and not on trying to “hit home runs” in health care innovation.

          When you combine a lot of variance with small [but statistically significant] effect[s] you can – and do get a very wide range of outcomes – not consistent “excellence”.

    • So grateful for this post and the comments which I agree with. Perhaps the most frustrating aspect of health care talk in this country is that, even among those who should know better, it does NOT represent what is on TIE. Like others have stated there is clearly a bias towards tech, towards the “experience” i.e. hospital as “hotel” and what you have personally experienced. And, as pointed out, this has a lot to do what is aggressively marketed, note to the free market types out there perhaps the only thing that is not promoted is actual price. This says a lot about people’s perception of health care and why health care is just not like any other commodity.

      • Jennifer, thanks for the kind words. You get a free lifetime subscription to TIE.

        Seriously, if we can encourage _data-based_ debate on these issues, we will have done our job. Taking the long view, I’m encouraged. The quality of argument in the best newspapers was, in my opinion, substantially better during the ACA debate than during the ClintonCare debate.

    • How does the US compare with other countries in terms of medical errors? Studies have found that malpractice kills more people in the US than car and workplace accidents combined, but that’s only things that are clearly identifiable as malpractice.

      The anecdotal evidence here is that things are far worse in the US than books such as “The Medical Malpractice Myth” describe: the “care” my parents received at “the best hospital in Boston” was worse than bad. (They dropped my mother from a bed, hurting her badly, and I’ve mentioned my father’s problems before.) I’m quite sure that the problems with their care never made it into any malpractice statistics, but they should have.

      • David,
        Google suggests that some work has been done on international comparisons of medical error rates. That’s too far from my field to be able to evaluate that research, so with your permission I will leave it to you.

        I am sad about the care that your parents received.

        Being nearly illiterate in the law, I wonder whether all medical errors count as malpractice?

      • David,

        I have tried to stress elsewhere the importance of understanding that we deal with two [at least] types of errors in healthcare…

        The first – and easiest one to deal with – is applying the wrong treatment when a clear – better – correct treatment is available. That I think is what we should all agree is malpractice…

        The problem is that there are many cases where the correct treatment is not known [or even worse “knowable”] – that the Doctor[s] are not sure what to do – they are using there experience and intuition to try and solve a problem that they want desperately to solve – but are in the kind of diagnostic mess that we see on House – or other medical dramas. Unfortunately this happens far more often [I believe] than health care pros would like us to believe.

    • “two [at least] types of errors in healthcare…”

      I was about to argue for a third type, since my impression of my parents’ problems were that they were due to a failure to _care_ for the patient, i.e. a failure to look at and think about the patient. In retrospect, the system (at least at “the two best hospitals in Boston”) needs extremely proactive involvement of a close family member capable of getting in the doctors and staff personnel’s faces and making sure that the patient is actually cared for. My father spent three days going rapidly downhill intellectually (becoming more and more confused and disturbed), yet the doctor in charge thought he was doing fine and would be discharged as soon as they got the (hospital-acquired) infections under control. Father died less than 12 hours later. It’s rather odd that I could diagnose a life-threatening condition from the other side of the planet better than the best and the brightest at “the second best hospital in Boston” could. (And that was after a failure of care at the ER at “the best hospital in Boston”, in which father wasn’t treated for the problem that brought him there.)

      So when people get started about how wonderful US health _care_ is, I tend to go postal. It looks to me that US health care is seriously bad unless you have someone to keep a close watch on said care and make sure they’re actually caring for the patient.

      But maybe this falls under your first type. Or maybe it’s a necessary concommittant of this thing we call medicine.

      Japan, traditionally, had the idea that when someone was hospitalized, a family member had to come along and live with them at the hospital. Much of that was just to do laundry and the like, since Japanese hospitals were rather Spartan in those days (before the, say, 80s), but after seeing the failures in the US, I’m thinking there was more sense there than I gave them credit for. A common theme in Japanese film and TV drama was a family member pleading with the doctor to look more carefully at the patient. Seems that’s necessary in the US as well.

    • Excellent discussion. I was surprised and disappointed when I read Lazarus’ article. He’s usually better than that.

      I spend much of my time educating the public and others about health reform and the ACA (not necessarily promoting it, but simply explaining what’s in the law and its intentions – often the first rational discussion many people have had about Obamacare, but that’s a different issue). Before discussing the law, I always talk about health care, health insurance, and the need for reform. The most important take-home message, more important than understanding what’s in the ACA, is that the U.S. does NOT have the best health care system in the world. Not even close.

      If people leave and understand only that, I’m satisfied.