I don’t know a lot about healthcare.gov’s inner workings, so maybe the following is incorrect. With one caveat, which I mention below, it passed a vetting on Twitter (FWIW).
- “The internal 80 percent target is the basis of a promise that has become an administration mantra in recent weeks: HealthCare.gov will ‘work smoothly for the vast majority of users’ by the end of November.” (Source.)
- “This past Friday, the average [per page] error rate was approximately .75 or three quarters of one percent.” (Source.)
- A CMS demonstration video showed that a healthcare.gov application can require over 100 pages. (Source.)
Let’s do the math. Given the above, the probability of success of getting through one page is 1-0.0075 = 0.9925. Therefore, the probability of successfully getting through n pages is 0.9925n. If n is 100, the probability of completing an application successfully is 0.47 or, as luck would have it, 47%. That’s below 80%, the administration’s target. (If an application only took 30 pages, the probability of success would be just under 80%.)
This has the appearance of a contradiction.
The big assumption here is that errors are evenly distributed over individuals. If only 20% of the individuals are generating all the errors, there’s no contradiction, for example.
Let’s suppose the errors are evenly distributed. It would then seem that reaching an 80% success rate would require individuals to make multiple application attempts. How many retries would be necessary, on average? If I’m doing the math right, the answer is the smallest integer m, such that
(1-0.9925n)m < 0.20
With n = 100, I get m = 3. The simpler way to think about this is that with one try, 53% of people fail (47% succeed, per above). If those people retry, 53% of 53% fail, or 28%. It takes a third try to get below 20% failure, or 80% success, the administration’s goal. Now, the answer is fewer than 3 if one doesn’t have to go all the way back to the beginning of the process with each retry (e.g., if the system saves your place upon error). But, even in that case, it takes more than one try, on average, to successfully apply.
This is why you can get stories of people having to try repeatedly to apply alongside with administration statistics that might suggest a more user-friendly experience. Again, there may be details that make the above moot. I just don’t know them. Delighted for corrections in the comments, on Twitter, or by email.
