Very early in my research career (actually, as a grad student) I noticed that researchers were not consistent in their use of temporal tense (past, present, future) in journal articles and manuscripts. It was (and still is) all over the map. You’ll see stuff like this, even within the same paper:
“We ran OLS regressions and find …” [“ran” is past tense; “find” is present]
“We estimate a model with fixed effects.” [“estimate” is present]
“The coefficient is … which meant that.” [“is” is present; “meant” is past]
Yikes! Which of these is correct? This bothered me, and I wondered if there were rules about whether research was or is or results are or were. I think I asked around and got different opinions. Then I made up my own set of rules, which I’ve followed to this day, and attempt to get others to follow. They’re below. Do you agree with them? Comments open for one week.
All the things I (or my colleagues and I) did methodologically, I (we) did in the past, so I use past tense for methods and data collection. However, all the results are true for all time (so we think), so I like to report results in the present tense, with one exception.
Example: “We ran [past] an OLS and found [past] that people with bigger feet are [present] smarter than people with smaller feet, even controlling for age.”
The only exception is if I’m explicitly referencing a prior year. Example: “However, we found that in the 1990s, people with bigger feet were [past, because we’re explicitly referencing a past year] stupider than people with smaller feet.” It would read oddly to say that people in the 1990s with bigger feet “are” stupider …
That’s it. Not hard to follow. Even if you disagree, it has the advantage of consistency. At a minimum, authors should at least figure out what tense they want to use for methods and results and then do so consistently in a single paper. Read closely, and you’ll find that researchers don’t/didn’t/and may never pay attention to tense.
The epidemic has led every state but Missouri to establish one of these programs, which allow doctors and regulators to track how many opioid medications and other controlled substances have been dispensed to patients. A new analysis shows that prescription drug monitoring programs can reduce the overuse of narcotics — but that many states have adopted relatively weak versions.
Opioid medications, like Vicodin, Percocet or OxyContin, can be useful in treating pain. But when patients receive many prescriptions — whether from multiple doctors at the same time or from the same one for a long period of time — it can signal a problem. Patients with more pills than they need could endanger themselves or divert them to the black market. Longer-term use increases the risk of addiction and other bad outcomes.
Data from the Centers for Disease Control and Prevention show that opioid overdoses and prescriptions grew in parallel between 1999 and 2010. Though prescriptions have fallen more recently, they have been written for longer durations. Black-market opioids — heroin and, in particular, fentanyl — also contribute to overdose deaths. But many who use these drugs also use prescription opioids and may have become dependent on them first.
That’s where prescription drug monitoring programs come in. They collect data from pharmacies to track what prescriptions for controlled substances patients have filled. The databases can be used to assess whether patients are getting more opioids than they can safely use. In addition, they can be used to tell if patients are getting other drugs, like a benzodiazepine, that are dangerous to use in combination with an opioid.
According to research summarized by the Leonard Davis Institute of Health Economics at the University of Pennsylvania, prescription drug monitoring programs can help reduce the amount or strength of opioids prescribed and dispensed. When physicians or dentists check the database and see a worrisome pattern of dispensed opioids, they can deny or change a prescription, screen for an opioid or other substance use disorder, and even counsel the patient to seek other forms of pain management or addiction treatment, if warranted.
Dr. Zachary Meisel, an author of the Leonard Davis review, uses a drug database when he practices in the Hospital of the University of Pennsylvania emergency department. In related work, he found that the databases often prompt conversations about opioids between provider and patient. In other cases, he said, prescription drug monitoring programs “help dispel a suspicion that a patient is seeking additional opioids.”
Monitoring programs are mitigating the opioid epidemic. One study, published in Health Affairs, found they’re associated with a decline in the chance a patient with pain will receive a Schedule II opioid prescription, to 3.7 percent from 5.5 percent. The study was based on a sample of 26,275 doctor’s office visits in the 24 states that started drug monitoring operations during 2001-2010. The results of another study— of Medicare beneficiaries over 2007-12 in 10 states — suggest that such programs are associated with reductions in the strength of opioid medications dispensed and the duration of opioid prescriptions. Deaths related to oxycodone use fell 25 percent in 2012, after Florida created a monitoring program.
But other work shows that having access to a prescription drug monitoring program is not enough. States can make the programs much more effective by mandating prescribers to engage with it. Twenty-five states require prescribers — physicians and dentists — to register with their state database. This forces prescribers to push through the first barrier to use — just signing up — and seems to make a difference.
One study, published this year in Health Affairs, found that states that required prescriber registration saw a 10-percentage-point reduction in use of Schedule II opioids among Medicaid enrollees, relative to states that did not require registration.
Most, but not all, states with mandatory registration also require prescribers to consult their state databases before prescribing an opioid. A study published in Health Services Research this year found that states requiring this experienced a reduction in the duration of opioid prescriptions in the Medicare population. Another study, also of the Medicare population, found that mandatory use was associated with fewer patients getting opioids from multiple doctors — so-called doctor shopping — and with patients holding a smaller supply of the drugs.
Studies expanding beyond the Medicare population to include younger Americans find that use mandatesreduce admissions to treatment facilities for opioid use disorder. A New York study of prescriptions by dentists in an urgent care center found that when its mandatory program went into effect, opioid pills prescribed went down 78 percent.
When states roll out monitoring programs, opioid-related overdose deaths fall, according to one study. And they decline more in states that mandate their use. Though some studies have not found that such programs reduce opioid use or opioid-related mortality, it could be because they do not distinguish between programs with such mandates and those without.
Nevertheless, prescription drug monitoring programs have limitations. They can track only dispensed drugs, not black-market drugs (like heroin and fentanyl). And though they can be used to tell how many and what kinds of opioids are dispensed, they can’t tell who takes them or if they’re diverted to the black market. The programs could also be more effective if more prescribers used them. One study found that only about half of primary care physicians use the database. Among those who use them, many do not do so routinely.
Prescription drug monitoring programs are not the only tool to combat the opioid crisis. Other state laws that tighten regulations of pain clinics and combat doctor shopping can help, too, reducing overdose deaths and admissions to treatment facilities. Wider distribution of naloxone, which can reverse an opioid overdose, and public education on its delivery can also help, as can greater access to safe means of disposing of unused pills.
Prescription drug monitoring programs have shown promise, but so long as relatively inexpensive heroin and fentanyl are available on the street, they will never be a full solution on their own.
The following originally appeared on The Upshot (copyright 2017, The New York Times Company).
In recent days, Democrats have stepped into the health policy vacuum created by the Republicans’ failure to repeal and replace the Affordable Care Act. Proposals making the rounds include allowing Americans to buy into Medicare at age 55 or to buy into Medicaid.
Both Medicare and Medicaid pay lower prices to health care providers compared with private market plans offered by employers and in the Affordable Care Act marketplaces. On that basis, you might think these public programs are more cost-efficient. Are they?
Imagine that I take my car to the cheapest mechanic in town, while you take yours to the most expensive. My repairs, though costing less, don’t always fix the problem or last as long. You get what you pay for.
Let’s take a look at whether something similar is happening with public health programs. One study examined claims data for 26 low-value services and found that as much as 2.7 percent of Medicare’s spending is on these services alone, which include ineffective cancer screening, diagnostic testing, imaging and surgery. That sounds pretty bad.
But a paper that appeared in Health Services Research this year suggests that private plans do not perform better. Looking at the years 2009 to 2011, the authors compared the rates at which Medicare and private health plans provided seven low-value services. The services compared were among those identified as unnecessary by national organizations of medical specialists as part of the Choosing Wisely campaign.
The researchers found that four of the seven services they examined were provided at similar rates by Medicare and commercial market plans: cervical cancer screening over age 65; prescription opioid use for migraines; cardiac testing in asymptomatic patients; and frequent bone density scans. Medicare was less likely to pay for unnecessary imaging for back pain, but more likely to pay for vitamin D screening.
This finding might seem counterintuitive. Commercial market plans pay higher rates and confer higher profit margins, meaning there is more financial incentive for physicians to provide privately insured patients more of all types of care, whether low or high value.
“What kind of insurance you have does affect your access to health care,” said Carrie Colla, associate professor of the Dartmouth Institute for Health Policy & Clinical Practice and the lead author of the study. “But once you’re in front of the doctor, by and large you’re treated the same way as any other patient.”
One apparent exception found in the study involved the seventh service it examined: cardiac testing before low-risk, noncardiac surgery. This service was provided to 46 percent of Medicare beneficiaries and 26 percent of privately insured patients. The large difference could reflect the fact that cardiac problems are more prevalent among older people. So a doctor with equal concern for all her patients might test Medicare patients at a higher rate for that reason. Nonetheless, such testing is considered low value even for the Medicare population.
Another recent study, published in JAMA Internal Medicine, also found little relationship between insurance status and low-value care. The study found no difference in the rates at which seven of nine low-value services were provided to patients on Medicaid versus those with private coverage. Six were also provided at the same rates for uninsured and privately insured patients.
Moreover, the study found that physicians who see a higher proportion of patients on Medicaid provide the same rate of low- and high-value services for all their patients as other physicians do. This is an important finding because Medicaid pays doctors less than private plans do, raising concerns that higher-quality doctors would tend not to participate in the program.
“Despite concerns to the contrary, Medicaid patients don’t appear to be seeing lower-quality doctors,” said Dr. Michael Barnett, lead author of the study, a physician with the Brigham and Women’s Hospital and an assistant professor at the Harvard T.H. Chan School of Public Health. “Though raising the prices Medicaid pays doctors may increase physician participation, enhancing enrollees’ access to care, it isn’t likely to change the quality of care patients receive once they are in the doctor’s office.”
If insurance status doesn’t influence how much low-value care patients are being offered, what does? In part, it seems related to the history and organization of local health care markets. A big culprit, according to Ms. Colla’s study, is a market’s ratio of specialists, like cardiologists and orthopedists, to primary care physicians. In areas where there are relatively more specialists, there is also more low-value care. That’s not to say that specialists don’t provide valuable services — but it suggests that they tend to provide more low-value care as well.
In a way, this is good news — the medical system doesn’t seem to discriminate by insurance status. It also means that public programs appear to be relatively cost-efficient, spending less than private payers for care of similar quality. That bodes well for Democrats’ proposals to expand Medicare or Medicaid.
But the bad news is that the study results imply that the value of care is hard to influence by adjusting prices. In a normal market, paying less for something would send a message of its low value, prompting people to provide less of it. The fact that price apparently does not influence doctors’ decisions is just another way in which health care does not seem to function like other markets.
But detoxification is actually extremely dangerous. Nearly every addict who successfully completes a week-long detox program without further treatment relapses, and in a world with increasingly powerful synthetic drugs on the market, the risk of overdosing and dying during a relapse has become ever more threatening.
That’s from a nicely written and brief piece in WaPo by Michael Stein, an internist and the chairman of the department of health law, policy and management at Boston University. Read the whole thing.
The following originally appeared on The Upshot (copyright 2017, The New York Times Company) while I was on vacation.
When you have a health problem, your first stop is probably to your primary care doctor. If you’ve found it harder to see your doctor in recent years, you could be tempted to blame the Affordable Care Act. As the health law sought to solve one problem, access to affordable health insurance, it risked creating another: too few primary care doctors to meet the surge in appointment requests from the newly insured.
The study, published in April in JAMA Internal Medicine, found that across 10 states, primary care appointment availability for Medicaid enrollees increased since the Affordable Care Act’s coverage expansions went into effect. For privately insured patients, appointment availability held steady. All of the gains in access to care for Medicaid enrollees were concentrated in states that expanded Medicaid coverage. For instance, in Illinois 20 percent more primary care physicians accepted Medicaid after expansion than before it. Gains in Iowa and Pennsylvania were lower, but still substantial: 8 percent and 7 percent.
“Given the duration of medical education, it’s not likely that thousands of new primary care practitioners entered the field in a few years to meet surging demand,” said the Penn health economist Daniel Polsky, the lead author on the study. There are other ways doctor’s offices can accommodate more patients, he added.
One way is by booking appointment requests further out, extending waiting times. The study findings bear this out. Waiting times increased for both Medicaid and privately insured patients. For example, the proportion of privately insured patients having to wait at least 30 days for an appointment grew to 10.5 percent from 7.1 percent.
The study assessed appointment availability and wait times, both before the 2014 coverage expansion and in 2016, using so-called secret shoppers. In this approach, people pretending to be patients with different characteristics — in this case with either Medicaid or private coverage — call doctor’s offices seeking appointments.
Improvement in Medicaid enrollees’ ability to obtain appointments may come as a surprise. Of all insurance types, Medicaid is the least likely to be accepted by physicians because it tends to pay the lowest rates. But some provisions of the Affordable Care Act may have enhanced Medicaid enrollees’ ability to obtain primary care.
The Affordable Care Act also included funding that fueled expansion of federally qualified health centers, which provide health care to patients regardless of ability to pay. Because these centers operate in low-income areas that are more likely to have greater concentrations of Medicaid enrollees, this expansion may have improved their access to care.
Other trends in medical practice might have aided in meeting growing appointment demand. “The practice and organization of medical care has been dynamic in recent years, and that could partly explain our results,” Mr. Polsky said. “For example, if patient panels are better managed by larger organizations, the trend towards consolidation could absorb some of the increased demand.”
Although the exact explanation is uncertain, what is clear is that the primary care system has not been overwhelmed by coverage expansion. Waiting times have gone up, but the ability of Medicaid patients to get appointments has improved, with no degradation in that aspect for privately insured patients.
1. Isn’t it true that the government pays Medicare Advantage plans a lot less today than they did in 2010, the year of focus of the study you wrote about?
Almost! According to government statistics, in 2010, Medicare Advantage plans received payments from the Medicare program equivalent to 113 percent of what it would cost a similar beneficiary to be covered by the traditional program. In 2017, that figure is 100 percent, but grows to 104 percent if you more accurately account for differences in the health of Medicare Advantage enrollees and traditional Medicare beneficiaries.
2. If the government is now paying the same (or almost the same) for an enrollee in Medicare Advantage as for a traditional Medicare beneficiary, what’s the problem?
Well, some believe that we should be taking advantage of market efficiency to save the government money, which was part of the original motivation for including private plan alternatives in Medicare. One key point of my piece was to compare what Medicare Advantage plans receive from the government to what it costs the plans to provide care, including marketing, administration, and profit as well. On that basis, in 2010, the plans received 8.5 percent more in government revenue than their costs. In 2017, that figure is 11 percent. What this means, as I wrote, is that Medicare Advantage plans are more efficient at managing care than the traditional program, but that taxpayers aren’t benefiting from that efficiency as much as they could be.
3. Where does that extra money go?
Medicare Advantage plans are supposed to use the additional revenue to enhance their benefits — either by providing coverage for things traditional Medicare does not cover (e.g., eyeglasses and hearing aids) or by reducing cost sharing. There is no doubt plans do this, as I wrote in my piece. And this is of tremendous benefit to enrollees, particularly lower income ones that cannot afford a supplemental plan to fill in the gaps in traditional Medicare.
A post on the Health Affairs blog documents in greater detail the kinds of additional benefits Medicare Advantage plans provide, beyond what’s covered by traditional Medicare. Just over half get basic dental benefits, three-quarters get eye exam coverage, just under half get a hearing aid benefit, and about one-third get help paying for gym memberships. The vast majority of Medicare Advantage enrollees that get these benefits, and others, do so with no additional premium.
4. Isn’t it true that sicker patients tend to leave Medicare Advantage?
Yes. This is something I wrote about in another Upshot post. Because Medicare Advantage plans have networks, enrollees are not covered for just any doctor they wish. Medicare Advantage plans may also impose other restrictions on care, like requiring prior authorization for some services. For sicker patients, such practices impose a heavier burden, because they need more care and see more doctors. Some of those patients choose to leave Medicare Advantage and return to the traditional Medicare program, which has an open network and does not attempt to manage care.
5. So, given all this, what is the value of Medicare Advantage?
Medicare Advantage plans have been found to be of higher quality than traditional Medicare. They also reduce wasteful use of health care by managing care, something the traditional program doesn’t do at all. Finally, they fill in gaps in coverage and cost sharing of the traditional program. They’re able to do so when the traditional program is not because changing traditional Medicare would require legislation, and it’s hard to achieve political consensus on anything in health care these days.
The bottom line is that Medicare Advantage plans offer choices that some beneficiaries value. They can deliver the Medicare benefit more efficiently and with higher quality. Yet, taxpayers do pay more to plans than they could, given plans’ own costs. Paying less might mean plans leave the market and that enrollees get less. There are always tradeoffs.
The following originally appeared on The Upshot (copyright 2017, The New York Times Company).
The Medicare Advantage program was supposed to save taxpayers money by allowing insurers to offer older Americans private alternatives to Medicare. The plans now cover 19 million people, a thirdof all those who qualify for Medicare. Enrollee satisfaction is generally high, and studies show that plans offer higher quality than traditional Medicare. But the government pays insurers more than they pay out for patient care — in some years, it turns out, a great deal more.
Concern about Medicare Advantage’s cost has found sharp expression in a recent suit brought by the Justice Department charging UnitedHealth with excessive billing of the government. While that suit plays out, research published by the National Bureau of Economic Research provides context.
The study, released in January, found that the revenue Medicare Advantage plans received in 2010 exceeded the amount they paid out for medical care by a hefty 30 percent. At more than $2,000 per enrollee per year, that probably topped $20 billion dollars, nearly all from federal payments, not enrollee premiums. The study relied on Medicare Advantage billing data obtained from three large insurers across 36 states, a type of data the government doesn’t yet release.
Paradoxically, even though Medicare Advantage plans cost taxpayers more than traditional Medicare, they spend less on care. In fact, one of the motivations of the program is to capture that lower spending as savings for taxpayers. It hasn’t worked out that way.
“Our study found that health care spending for enrollees in Medicare Advantage plans is 10 to 25 percent lower than for comparable enrollees in traditional Medicare,” said Amy Finkelstein, an M.I.T. economist and one of the study’s authors. “Yet government payments to plans is far above their lower health care costs.” The study was also conducted by four economists at Stanford: Vilsa Curto, Liran Einav, Jonathan Levin and Jay Bhattacharya.
The analysis raises two questions: How do Medicare Advantage plans spend so much less on care? And, given that, how do we account for their higher costs to taxpayers?
One reason for the lower spending is that Medicare Advantage enrollees use less care or use lower-cost care. For example, compared with traditional Medicare patients, Medicare Advantage patients are more likely to go home after a hospital visit, rather than to a skilled nursing facility. Medicare Advantage patients see specialists relatively less often and receive fewer inpatient operations, but more outpatient ones, which are cheaper. All of these are what you’d expect from care management techniques used by Medicare Advantage: referral requirements and narrow networks of doctors, for instance.
“This is exactly what Medicare Advantage plans were designed to do,” said Dr. Bruce Landon, a physician with Harvard Medical School. “They manage the utilization of services while also assuring that enrollees receive recommended care, all at lower cost to patients.” Dr. Landon’s research on the program found that Medicare Advantage enrollees use 20 percent to 30 percent less emergency department and outpatient surgical care, as well as receive fewer hip and knee replacements.
Medicare Advantage plans also attract enrollees who tend to be healthier than traditional Medicare beneficiaries, a feature that yielded intriguing results in light of the lawsuit against UnitedHealth. When the M.I.T.-Stanford team compared the two kinds of Medicare patients, they found that Medicare Advantage patients were 25 percent less costly than traditional Medicare patients. But when the team more rigorously matched the health of both sets of patients, the Medicare Advantage patients were just 10 percent less costly. This drop does not prove the suit’s claims of overbilling, but it allows for the possibility.
Why does the government pay Medicare Advantage plans so much more than it costs them to cover care? It’s partly an intentional, if controversial, design of the program. Congress has established payment formulas and authorized bonus programs intended to help the private market.
The government also pays insurers for administrative and marketing expenses. Yet even when these additional expenses are factored in, the government still pays plans an excess. According to the Medicare Payment Advisory Commission, federal payments to the plans exceeded health care costs and other expenses by 8.5 percent in 2010. Though the Affordable Care Act has reduced payments to plans and limits the amount they can attribute to administration and marketing, they still receive government payments in excess of their costs today.
Not all of the “excess” federal money goes to the insurers’ bottom line. Traditional Medicare entails significant cost sharing for beneficiaries; they are responsible for 20 percent of the costs of doctors visits, for example. Most Medicare Advantage plans don’t require as much cost sharing or out-of-pocket payments. And some of the influence of Medicare Advantage plans’ managed care techniques rub off on the traditional program, too, reducing spending — a spillover effect that partly explains the slowdown in growth of Medicare spending.
But is the cost of Medicare Advantage worth the benefits it delivers? It’s hard to know without knowing more about patients’ diagnoses, services used and other data. The Medicare program had been collecting such data since 2012 and was planning to release it, but, expressing concerns about its quality, recently put off doing so.
Whether through an employer, Medicare, or an Affordable Care Act marketplace, many Americans have choices of health care plans — sometimes from among dozens of possibilities. I’ve written a lot on the AcademyHealth blog and The Upshot about how hard it is for people to make good plan choices from such a bounty of options. They often end up paying more than they should, either because they actively choose a suboptimal product or because they passively remain in a product that drifts away from optimality over time.
Some studies indicate that with reminders and assistance — provision of calculating tools, better ways of comparing and viewing options, and other guidance — people can be encouraged to make active choices and, moreover, make better ones. Availability of such things is not widespread, but it is growing.
But is encouraging annual plan shopping, providing selection aids, and improving choice architecture just treating the symptom of the real problem? Maybe more choices isn’t better? What if helping consumers navigate them is merely second best to reducing the number of choices in the first place?
It sounds a bit crazy, because a premise of an ideal market is that there are so many choices, everybody can find a product to perfectly match his or her preferences. No market achieves that ideal, but clearly fewer choices moves in the wrong directly, theoretically.
Theory is useless if it is wrong. It must be tested.
In a study published as an NBER working paper, Jason Abaluck, Jonathan Gruber did so. They compared people’s choices of health plans under various interventions: (1) promotion of active choices, (2) assistance with choice support software, and (3) reduction in plan choices. According to their analysis, across these intervention types, consumers achieved the best outcomes when choices were restricted. Less is more!
These interventions were implemented across school districts in Oregon between 2008 and 2013 where, according to the analysis, only 36% of covered employees had made the cost-minimizing choice. On average, employees spent about $1,000 more on coverage per year than they could have. The number of plan choices varied across district and time. Through 2011, the vast majority of employees had four choices. After 2011, some had as many as ten. Choices ranged in plan style — closed panel HMOs (e.g., Kaiser) and broader network PPOs — as did employer contributions to premiums and cost sharing, all of which was controlled for in analysis.
The authors analyzed nearly 400,000 employee-year observations in several ways. One analysis exploited the fact that 12% of employee-year observations were forced to switch plans when their current plan was dropped in their district. Did forcing people to switch cause them to choose better plans, compared to people who did not switch? Nope. Forced switchers actually spent even more than they could have, compared to non-switchers.
Another analysis exploited the randomization of enrollees to access to a tool that used each individual’s claims history to provided the total cost of each available plan. Based on this information, it also displayed plans in order of total cost and stared the lowest cost plan. In other words, it relieved consumers of most of the cognitive effort of comparing plan costs — just look for the star and done!
The authors estimated that users that received the tool’s recommendations were only 8% more likely to choose the best plan, relative to those that did not have access to the tool. Unfortunately, the tool’s recommendations were not always in alignment with true plan costs, because the tool had to make some assumptions and also due to user input error. The authors estimated that a perfect tool fed perfect information — which is essentially impossible to achieve — would induce 17% of users to choose the lowest cost plan, reducing foregone savings, coincidentally, by only 17% as well.
Finally, the authors examined forgone savings as a function of choice sets. Foregone savings are almost twice as large when employees face a choice of eight plans versus only four plans. Now, this result could occur if what’s going on is that as additional choices include better ones, but that enrollees don’t pick better ones. That is, they could do better with more choices, but they don’t know how to make the right ones.
That’s not what’s going on. The authors wrote,
[S]maller choice sets lead to beneficiaries being enrolled in lower cost plans because larger choice sets have more plans which are higher cost on average – and individual choices do little to offset this “more dangerous” choice environment.
In other words, when choice sets are small, plan administrators have made relatively good selections already. But as administrators expand the choice set, they add worse (more expensive) plans. Because consumers have a hard time picking the best plan, they are more likely to end up in a more expensive one as choices grow.
The conclusion is that consumers may save far more if they are presented with fewer choices than they save by either being induced to shop or by provision of output from a plan cost estimator. Yes, we can help some people make modestly better choices with cost calculators and better choice architecture — and we should provide such things. However, according to this study, people are so bad at choosing plans that they are even better served by simplifying their choices.
The following originally appeared on The Upshot (copyright 2017, The New York Times Company).
The relatively recent movements toward transparency and quality in health care have collided to produce dozens of publicly available hospital quality metrics. You might consider studying them in advance of your next hospital visit. But how do you know if the metrics actually mean anything?
There are valid reasons to be suspicious of measurements of hospital quality. One longstanding concern is that some hospitals may disproportionately attract sicker patients, who are more likely to have worse health outcomes. That could cause those hospitals to appear less effective than they actually are. Statistical techniques can mitigate but not completely eliminate this bias.
A related problem is that measurement of the quality of a hospital can be biased if it doesn’t take into account the socioeconomic status of the population it serves — and many such metrics do not. For example, a hospital in a wealthy region serves patients with more resources, relative to a hospital in a poorer region. If greater patient resources translate into better health — and a lot of research suggests they do — the hospital in the wealthy region may appear to be of higher quality. But that isn’t necessarily because of the care it delivers.
Because of issues like these, one study found that approaches to rating hospitals don’t agree on which hospitals are high or low in quality. “We have a vast number of quality measures,” said Dr. Ashish Jha, a co-author of the study and a scholar of health care quality at the Harvard T.H. Chan School of Public Health, “but which are signal and which are noise? It can be incredibly tricky to sort out.”
A recent study, however, shows that there is at least a bit of signal within the noise. The study, by health economists at M.I.T. and Vanderbilt, found that hospitals that score better on certain metrics reduce mortality. Among the ones they examined were patient satisfaction scores.
“We found that hospitals’ patient satisfaction scores are useful signals of quality, which surprised me to some extent,” said Joseph Doyle, an economist at M.I.T. and one of the study’s authors. “Hospitals with more satisfied patients have lower mortality rates, as well as lower readmission rates.”
According to the study, a hospital with a satisfaction score that is 10 percentage points higher — 70 percent of patients satisfied versus 60 percent, for example — has a mortality rate that is 2.8 percentage points lower and a 30-day readmission rate that is 1.9 percentage points lower. This is consistent with earlier work, described by my colleague Aaron Carroll, that found an association between better Yelp ratings of hospitals and lower mortality rates and readmission rates for certain conditions.
Mr. Doyle’s study, published as a National Bureau of Economic Research working paper, is exceedingly clever in its design. The ideal study would be to randomly assign patients needing hospital care to facilities with high or low quality. Then, this ideal study would see what happened to those two groups of patients: Did the group randomized to more highly rated hospitals live longer and stay out of the hospital longer? If so, the metrics are, in fact, providing useful guidance.
For ethical as well as practical reasons, we cannot randomly assign patients to hospitals. But it turns out that in emergency situations, like heart attacks, which ambulance service picks up patients who live in the same neighborhood is effectively random in many cases.
In some locations, patients are assigned to services in an orderly rotation. In others, services compete to see which can reach a patient first. In others still, it’s the ambulance that happens to be closest to the patient that gets the business. In all of these cases, exactly which ambulance picks up a given patient with a given condition is random. It also turns out that ambulance companies have preferences for certain hospitals, and the random assignment of ambulance companies to patients leads to an effectively random selection of the hospital at which they receive care.
The authors exploited this randomness as a natural experiment to test how different kinds of hospital quality measures predicted mortality and readmissions. Using data from 2008 to 2012, they compared Medicare patients needing emergency care who lived in the same ZIP code but were served by different ambulance companies and, therefore, tended to be delivered to different hospitals with different quality scores. The approach was validated in earlier research that showed that higher-cost hospitals have lower mortality rates than lower-cost ones.
In addition to testing the predictive ability of satisfaction scores, Mr. Doyle’s study examined indicators of high-quality care — things that a hospital does that are believed to improve outcomes, like the rate at which a hospital gives heart attack patients aspirin upon arrival.
Here, too, hospitals with better such indicators had lower mortality and readmission rates. The very best hospitals by these measures can reduce the odds of death within a year by 14 percent relative to the very worst hospitals, for example.
“Though hospital quality measures are not perfect, our work provides some reasons to be optimistic about some of them,” Mr. Doyle said. “Hospitals that score well on patient satisfaction, follow good processes of care and record lower hospital mortality rates over the prior three years do seem to keep patients alive and out of the hospital longer.”
For each marketplace plan, they quantified network size as the ratio of the number of participating physicians to the number of physicians eligible to participate in the plan’s service area. They did this for all physicians and by specialty. They categorized network extent as follows:
x-small: <10% of physicians participating
small: 10%-25% of physicians participating
medium: 25%-40% of physicians participating
large: 40%-60% of physicians participating
x-large: ≥60% of physicians participating
Across all plans and marketplaces, 12% of networks are x-small, 19% are small, 24% are medium, 31% are large, and 15% are x-large. That’s a lot of numbers. Since consumers, wonks, and policymakers are probably most concerned about narrow networks, it may be simpler just to pay attention to the proportion that are either x-small or small: 31%.
According to their analysis, for the most part, there is little correlation between network extent and metal tiers. As shown in the chart below, within metal tier, the distribution of plans’ network extent is fairly stable. The one exception is platinum plans, which have a substantially higher proportion of narrow networks, with 41% x-small or small. However, platinum plans only attract 5% of enrollees.
The Data Brief also breaks down network extent by physician specialty. Prior work suggests that narrow network plans help control health care spending so long as they don’t disrupt access to primary care physicians (PCPs) and do reduce network extent of specialists. The 2016 results show that network extent across primary care and other specialists is largely similar.
There are two exceptions. First, networks for psychiatrists tend to be much more narrow (45% x-small or small) than other specialists (e.g., 31% x-small or small for PCPs). This raises concerns about adequate access to mental health care in marketplace plans. Second, hospital-based physician networks are extremely narrow: 72% x-small or small. As the authors point out, “This is notable given that this is the group of physicians most likely to lead to a surprise out-of-network bill.”
Network extent varies tremendously by state. The chart below shows the percent of networks in each state that are x-small or small. It would be valuable to understand what accounts for such variation and its implications. How does it relate to provider and insurer market power, for example? What patterns of care and outcomes correlate with network extent? I’d also like to know how geographic variation of network extent looks by specialty, again with the implications for access and health care outcomes.
If you’re concerned about narrow networks, you might want to know how their prevalence has evolved over time. The authors compared network extent from 2014 to 2016 for silver-rated plans. By and large, the proportion of narrow network plans didn’t change, though there was a shift from small networks (declining from 31% to 29%) to x-small networks (increasing from 6% to 12%).
You’ll find even more stats in the Brief. Though the Leonard Davis Institute investigators have made aggregate marketplace network extent more transparent to policymakers and the public, making network extent — including within specialty — transparent to consumers at the time of plan purchase is an ongoing challenge. Also particularly troubling is
[t]he high prevalence of narrow networks among hospital-based physicians […]. Given that these physicians are the ones most likely to send surprise out-of-network bills, this remains a concern for those with narrow network plans and broad plans.
Austin and Aaron are participants in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com.