UPDATED: My response to Megan McArdle re. the McKinsey Study

In an article over at BloombergView, Megan McArdle (formerly of Newsweek/The Daily Beast, The Atlantic and The Economist) invoked my name regarding the recent "27% Previously Uninsured" McKinsey study.

Two reasons have been offered for why McKinsey might be wrong. For starters, Charles Gaba asks why, if McKinsey is correct, these numbers are so different from the numbers in New York, which collects data on prior insurance status and has found that a majority of purchasers are the previously uninsured.

Actually, I can answer that: New York had almost completely destroyed its insurance market by implementing guaranteed issue and community rating (insurers have to sell you a policy at the same price as they'd sell one to anyone else), but not a mandate. Prior to Obamacare's implementation, policies in New York's individual market were astronomically unaffordable for all but the very rich and very sick. It's actually pretty unlikely that what is happening in New York will be a good guide to the rest of the country.

I have no problem with McArdle's response; I've heard the same thing from several other people. I've never claimed that New York's ratio (currently over 70%) is representative of the nation at large, and my personal suspicion is that the true "previously insured / uninsured" breakout nationally is closer to 50/50 overall.

What I do have an issue with is that even though she cited me and linked to my piece, she completely ignored the main point of the article: That the McKinsey study includes people who bought insurance outside the new marketplaces (that is, off-exchange or direct enrollments). You know, that mysterious data that I've been desperately trying to track down (hopefully with your help).

As I explained...in what I thought was pretty clear terms...if you're going to do a survey that includes people from Group A and Group B, but you have absolutely no idea how large Group B is, nor do you have any idea how many of the results come from Group A vs. Group B, then any conclusions that you draw from that survey are pretty much worthless.

Let me reiterate my point:

The Exchange-Based QHP enrollments were at around 3.6 million at the time of the McKinsey study. Let's say that Off-Exchange QHP enrollments were also at 3.6 million, for 7.2 million total.

McKinsey is saying that 27% (in this example, around 1.94 million of the combined total) were previously uninsured, while 73% (5.26 million in this example) were already insured.

However, without knowing how many of that 27% came from the ACA exchange pool and how many came from the off-exchange pool, this tells us absolutely nothing about what percent of ACA exchange enrollees were previously uninsured.

Perhaps all 3.6 million off-exchange enrollees already had insurance, and only 1.66 million of the exchange-based enrollees did. That would mean 54% of exchange-based enrollees were uninsured, not 27%.

Or, perhaps it's the reverse, and every single exchange enrollee was already insured (0% uninsured, a disaster!).

I'm not saying that either of these is the case; the point is that by mixing the two together, the only conclusion we can draw from this is that the correct number is somewhere betwee 0% - 54%, which isn't terribly helpful.

But wait, it gets worse!

The above example assumes that an equal number of exchange & off-exchange enrollees are out there, when in fact we only know one of the correct numbers: 3.6 million exchange-based as of mid-February.

The off-exchange number could easily be 10 million. Or 5 million. Or 1 million. I know it's at least 465,000, but that's only out of a handful of companies and one state.

So, if the off-exchange number is 10 million (or, for that matter, anything higher than 5.3 million), that could, in theory, mean that every single previously insured enrollee was done off-exchange, and 100% of exchange enrollees are newly insured. This obviously isn't the case (my own family switched from a non-compliant policy to a compliant one via HC.gov), but you get my point.

Don't take my word for it, however; the McKinsey study itself (PDF) stresses the unknown off-exchange factor not once, not twice, but SEVEN TIMES, quite emphatically:

In February 2014 we conducted another round of our national survey of consumers eligible to purchase individual coverage during the open enrollment period (either on or off the exchanges).

Each round has included consumers who enrolled in healthcare coverage for 2014 (either on or off exchange), those who have shopped but not yet enrolled, and those who have not yet shopped.

Three important points help clarify how the survey findings should be interpreted. First, some of the respondents purchased coverage through channels other than the exchanges; as a result, the numbers in our survey cannot be directly compared to the publicly reported exchange enrollment numbers.

In all cases, the products could have been selected on or off the exchanges.

The objective is to understand the shopping and purchasing behavior of consumers who are eligible to purchase individual coverage on the ACA exchanges or elsewhere.

We used the following characteristics to focus on the consumer segments eligible to purchase individual coverage on the ACA exchanges or elsewhere:

Select new 2014 product: Either previously insured respondents who switched (either switching from one carrier to another, or switching policies but staying with the same carrier), or previously uninsured who enrolled; policies can be selected on or off exchange; includes those who paid their premium and those who hadn’t yet at time of survey.

Ms. McArdle makes some valid points in her article, and does a good job of answering my "but what about New York" question. However, she chose to completely ignore this factor, even though it was the entire point of my piece. Why she did this I can't say.

Having said all this, I'll repeat: My personal guess is that, nationally, about half of the exchange QHP enrollments to date were previously uninsured, while the other half were already covered, and it may turn out that McKinsey's 27% is actually accurate...but if it is, that would be pure chance.

UPDATE: Shortly after I posted this, I heard back from Ms. McArdle via Twitter. She and I engaged in a lively discussion of the issue; her excuse for not mentioning the main point of my article was:

@charles_gaba I ignored it because I don't think it's really relevant; I want to know net insurance signups, not exchange signups per se.

— Megan McArdle (@asymmetricinfo) March 8, 2014

Well...the thing is, I want to know the total net insurance signups as well. HOWEVER, as I've shown above, the very study that McArdle herself is citing itself specifically states repeatedly that the "off-exchanges included" factor is very much relevant.

As for McArdle, take another look at her exact wording:

Overall, McKinsey says, just one quarter of the people who bought insurance on the exchange were previously uninsured.

That's pretty clear to me--McArdle is claiming that the 27% (higher than the 25% she claims, by the way) of uninsured enrollees are specifically on the exchange, when the McKinsey study she's quoting states seven times that the survey was not limited to "on exchange" enrollees.

If Ms. McArdle didn't realize that the McKinsey study was including off-exchange enrollees, then this error would be understandable...but a few moments later, when I pointed out that most of the people wondering about the "previously insured" issue don't seem to include off-exchange enrollments at all, she responded:

@charles_gaba Sure, but I've discussed off-exchange purchases extensively in the past, so I don't appear to be among those people.

— Megan McArdle (@asymmetricinfo) March 8, 2014

So, she's very much aware of the off-exchange issue, to the point that she's discussed it "extensively" in the past...but didn't happen to notice the SEVEN TIMES that the issue is referred to in the study, including one that specifically says "the numbers in our survey cannot be directly compared to the publicly reported exchange enrollment numbers."

Alternately, she chose not to make any mention of off-exchange enrollments whatsoever--and in fact falsely claimed that the survey included exchange-based enrollments only--because the other ones "aren't relevant" to her.

IMPORTANT: ALSO SEE MY FOLLOW-UP POST, "A RESPONSE TO AVIK ROY" PUBLISHED ON MARCH 25, 2014.