A Response to Avik Roy

Forbes' Avik Roy and I butted heads last week on Twitter regarding the "How many enrollees were previously uninsured???" issue, and with it, the McKinsey study which claimed that only 11% (in January) or 27% (in February) of the QHP total were previously uninsured.

In a nutshell, I never said that McKinsey's 27% claim was wrong, I just said that the methodology they used to come up with 27% didn't tell us anything about the previously uninsured percent of exchange QHPs because McKinsey mixed together both exchange and off-exchange enrollments...without noting how many of their respondents came from each pool. Since we have no idea how many off-exchange enrollments there are, and we don't know how many of the respondents came from either pool anyway, there's no way of knowing how much of that 27% comes from either. It could be 0% and 54%. It could be 54% and 0%. It could, in theory, be 100% of one and 0% of the other.

In any event, Roy first responded by claiming that:

@UriManor @charles_gaba Off-exchange enrollments don’t affect exchange risk pool; usually previously insured. Not a big number either way.

— Avik Roy (@Avik) March 19, 2014

...which is not only wrong (both ON and OFF-exchange enrollments are part of the same risk pool, as I noted this morning and the Washington Post confirmed this afternoon, which means that the pool is certainly impacted (for good or for bad) by both groups), but which has absolutely nothing to do with my point.

He then concludes his tweet by stating, quite matter-of-factly, that the Off-Exchange number "isn't big" anyway.

Considering that I've documented nearly 560,000 off-exchange QHP enrollments to date (from only 2 states and a dozen or so companies across other states, out of the 200+ health insurance companies nationwide), this seems like a rather odd thing to say. Even if there isn't a single off-exchange enrollment that I haven't found yet (and there are most certainly many, many more out there), 560K out of over 5.6 million is still 10%, a sizeable number no matter how you slice it (and, at the time that he wrote his column on March 8th, Mr. Roy only knew of the 4.24 million exchange QHPs which had been announced by HHS; the 5M announcement wouldn't come for another week. This means that just about all of the 560K off-exchange QHPs that I have would have made up either 11% or 13% of the total that Mr. Roy knew of at the time that we had our exchange).

Anyway, he then made another rather curious point:

@charles_gaba @urimanor off-exchange enrollees (mostly) ineligible for subsidies. So far less likely to dent uninsured figures.

— Avik Roy (@Avik) March 19, 2014

...which I agree with completely, and which supports my point that off-exchange enrollees are far more likely to already have insurance, since these are mostly people who had a high enough income to afford health insurance without the subsidies already.

I encouraged Mr. Roy to read my piece explaining my issue with the McKinsey study (not that they did anything wrong, just that it doesn't mean what Mr. Roy and others keep insisting that it means), and left it at that.

So, anyway, that was mostly it until today, when he got back to me and said...

.@charles_gaba @UriManor Read your blog post. You overstate impact of off-exchange #Obamacare enrollment. See update: http://t.co/DZqoW1FWOT

— Avik Roy (@Avik) March 25, 2014

So, I read the update to his piece, which reads as follows:

UPDATE: A Daily Kos blogger named Charles Gaba, at acasignups.net, argues that the McKinsey study “tells us pretty much nothing,” because it did not distinguish between off-exchange and on-exchange individual-market sign-ups. “My personal suspicion,” he writes, “is that the true ‘previously insured / uninsured’ breakout nationally is closer to 50/50 overall.”

Fortunately, we don’t have to rely on personal suspicions. As my colleague Scott Gottlieb notes, we have actual data from insurers on this topic; most analyses estimate that approximately 20 percent of sign-ups to date have been off the Obamacare exchanges.

If we (generously) assume that every single off-exchange signer-upper was previously insured, in order to make the math as flattering to Obamacare as possible, and that 20 percent of sign-ups were off-exchange, we arrive at the following paid enrollee breakdown on the exchange: 76 percent previously insured, 24 percent previously uninsured.

24 percent is better than 14 percent, but not so much better as to justify a furious defense of the exchanges’ performance in this regard. And the percentage of off-exchange uninsured signups is unlikely to be zero.

OK. First of all, yes, I'm a "Daily Kos blogger". Of course, my primary blogging platform for the past few months has been here, not at dKos, but I guess I can't fault him for pointing out to his readers that yes, I'm a Lefty. I don't try to hide that and am quite frank about it on my FAQ, but whatever.

Anyway, on to the substance of his response: He brings in his colleague Scott Gottlieb to resolve the issue, by claiming that Dr. Gottlieb claims that "most analyses" estimate that off-exchange enrollments only make up 20% of the total QHPs. He then uses the "20%" figure to come up with only 24% of exchange QHPs being previously uninsured.

First, let's assume that the 20% number is correct. That would mean that if there are 4,000,000 exchange QHPs, there are 1,000,000 off-exchange QHPs (5M total).

If 27% of the total are uninsured, that means 73% were insured, or 3.65M. Mr. Roy is "generously" allowing all 1M off-exchange QHPs to already be insured, leaving 2.65M of the 4M on-exchange enrollees to also be insured already. That means that 1.35M of 4M exchange QHPs are uninsured...or 33.75%, not 24%. Hmm.

OK, let's assume that he meant that the number of off-exchange QHPs was 20% of the number of exchange-QHPs: 1 million vs. 5 million, for 6M total. In that case, his figures work out: 73% of 6M = 4.38M. Subtract 1M insured off-exchange QHPs and you have 3.38M on-exchange QHPs who were insured, leaving just 1.62M uninsured...or 32.4%. Huh.

Now, you're probably saying, "but wait! In his analysis, he also subtracts the unpaid enrollments from the totals; that's why his numbers are so much lower than yours!"

Yes, that's true. Except that as I've noted many times before, at least 1/3 and as much as 40% of the total exchange QHP payments aren't even due yet, and up to 1/3 of them won't even be due until May anyway.

HOWEVER, guess what?? This is all academic for another reason:

Nowhere in Gottlieb's article--the sole source of Mr. Roy's "only 20% off-exchange" claim--does he (Gottlieb) say anything about "most analyses" giving the number as 20%. Here's what Gottlieb says:

Off exchange enrollment could be as high as 20%, according to one Wall Street analyst. If that holds true nationwide, these enrollees would probably offset the number of people who sign up on the exchange, but will never pay their first premium (and thus never be truly enrolled).

ONE. One (unnamed) Wall Street analyst. Not "most". Not "a significant number". One guy. He works on Wall Street. He's an analyst. Of the healthcare industry? I assume so. This one, unnamed source certainly doesn't qualify as "most analyses". You have to have at least 3 of anything before you can have a "most". Hell, why not just say "100% of analyses", while you're at it?

I'm not trying to beat up on Gottlieb here. He's not the one who said that his single source counted as "most". In fact, I very much agree with the larger point of his article, which reiterates the importance off-exchange QHPs, as I have been for months, and he's likely to be taken more seriously than me by the Forbes crowd, so I have no problem with his piece.

UPDATE: I forgot to credit Uri Manor (who actually kind of started the discussion with Mr. Roy and myself in the first place) for pointing this out yesterday, before I wrote my own response:

@charles_gaba @Avik @ScottGottliebMD says 'Off exch enroll could b as high as 20%, according to one Wall Street analyst." Not real data.

— Uri Manor (@UriManor) March 25, 2014

Then, take a look at what Gottlieb says later on:

The one data point we have on “off exchange” enrollment trends comes from Highmark, which recently announced that 31% of the 133,000 people it has enrolled across the three states where it operates (Pennsylvania, West Virginia, and Delaware) signed up off exchanges.

31%. Not 20%, but 31%--and that's hard data, not theory from an unnamed analyst. Funny that Mr. Roy didn't mention this.

Of course, there is another company mentioned: WellPoint, which does indeed report the 20% figure:

WellPoint WLP -1.27% has also said that about 20% of its applications came off exchanges, from customers going directly to the insurer.

...except that a) WellPoint isn't an "analyst", it's a single company; and b) if you average the two together (WellPoint enrolled over half a million new members as of 12/31; that's where the 20% came from), it actually comes to: (41.23K out of 133K) + (100K out of 500K) = (141.23K out of 633K), or 22.3%.

So, ok; that's a bit higher than 20%, but not by much. Other individual companies may report a higher percentage...or a lower one. CoOportunity of Iowa and Nebraska, for instance, reports 32,252 off-exchange enrollments vs. 17,926 on-exchange so far...or 64% of their total QHPs.

But guess what?

There's also Washington State, which reported 184,317 off-exchange enrollments vs. 84,287 exchange QHPs as of the end of January. Yeah, that's right: 68.6% of all QHP enrollments in the state of Washington were off-exchange through the end of January.

Now, I do also have Wisconsin documented through mid-January at only 7,900 off-exchange out of around 57,900 total, which comes to only 14%, although I didn't get that data until after both Mr. Roy's column and our discussion. However, if you average together Washington and Wisconsin (the only 2 full states for which we have data, and even that's only through the end of January), you get 192,217 / 326,504, or almost 59%.

I could go on, but I think my point is made.

Mr. Roy made a statement ("MOST analysts say 20%" when there's only a single analyst cited, without a name or source), while failing to mention two other hard data sources which give either 31% or 22.3% depending on your POV. He also completely ignores the multiple additional examples on this site which range as high as 68.6%.

However, even this is all academic, because thanks to CNBC's report yesterday, we finally have some real clarity on the "percent of exchange QHPs which were previously uninsured", at least in the 2 states which have actually released their data: New York and Kentucky.

You know what it is? 59% in New York and 75% in Kentucky.

Combine them and you get an average of 62% between the two states.

Mr. Roy's response to this?

@charles_gaba @urimanor You can’t cherry-pick two states, especially given New York’s uniquely dysfunctional pre-ACA market.

— Avik Roy (@Avik) March 25, 2014

I can't cherry-pick? He used as his source a single, unnamed, unspecified "Wall St. analyst" to insist that off-exchange QHPs only make up 20% of the total, while ignoring a bevy of more solid numbers that ranged up to 69%, and then says that I'm cherry-picking?

Oh, and about the "dysfunctional NY market"...yes, that would be the same dysfunctional market that caused Ms. McArdle, the writer who I was originally responding to about the McKinsey study in the first place to explain was likely to have a much higher previously uninsured exchange QHP rate than other states:

Two reasons have been offered for why McKinsey might be wrong. For starters, Charles Gaba asks why, if McKinsey is correct, these numbers are so different from the numbers in New York, which collects data on prior insurance status and has found that a majority of purchasers are the previously uninsured.

Actually, I can answer that: New York had almost completely destroyed its insurance market by implementing guaranteed issue and community rating (insurers have to sell you a policy at the same price as they'd sell one to anyone else), but not a mandate. Prior to Obamacare's implementation, policies in New York's individual market were astronomically unaffordable for all but the very rich and very sick. It's actually pretty unlikely that what is happening in New York will be a good guide to the rest of the country.

In other words, Ms. McArdle was quite certain that the only reason NY's "QHP previously uninsured rate" was so high is because insurance was so insanely high before the exchange subsidies came around.

And yet, Kentucky...with vastly different demographics, economics, geography, etc...is actually reporting a 16% higher previously uninsured exchange QHP rate than New York.

Mr. Roy's response to this?

@charles_gaba @urimanor Individual state data can’t be extrapolated, pure and simple.

— Avik Roy (@Avik) March 25, 2014

From someone who just extrapolated a single, unnamed source into an absolute while ignoring large amounts of solid evidence to the contrary.

Pure and simple indeed.

Now, here's the thing: I'm not saying that Mr. Roy's 20% is wrong, or that McKinsey's 27% is wrong. Maybe they're correct. I've only documented about 10% of the QHPs as being off-exchange; perhaps it really is only 20 or 27% of the total. I'm just saying that there's too many unknowns for anyone to conclude that it is 20% or 27% for exchange QHPs either. It's still a big unknown.

Furthermore, I do appreciate him at least bothering to read my own analysis. He's a Big Established Expert and I'm just some web developer in Michigan. He's (from his Forbes bio) the Opinion Editor for Forbes, a Senior Fellow at the Manhattan Institute for Policy Research, and was a health care policy advisor to Mitt Romney...while I create websites for small businesses, often while wearing a bathrobe.

But that doesn't mean that he's right, either...and unless I'm missing something important here, nothing that he's said proves that he is.