(UPDATED) A look at Avalere Health's "1.1M - 1.8M New Medicaid" Analysis

So, after yesterday's big CBO report which had some interesting (and, in some cases, extremely misinterpreted) points to make about the impact of the Affordable Care Act, today there was another report released today from Avalere Health which has gotten a lot of attention, regarding the number of new Medicaid/CHIP enrollees which have been added specifically due to the ACA. I don't know much about Avalere Health, but they tout themselves as a healthcare think tank/analysis firm. From their "About Us" page:

"Avalere is a vibrant community of innovative thinkers dedicated to solving the challenges of the healthcare system. We deliver a comprehensive perspective, compelling substance, and creative solutions to help you make better business decisions. We partner with stakeholders from across healthcare to help improve care delivery through better data, insights, and strategies."

OK, fair enough. According to Avalere:

Avalere estimates that from October through December 2013, between 1.1M and 1.8M people have newly enrolled in Medicaid as a result of the Affordable Care Act (ACA).

These estimates are concentrated in states expanding Medicaid—with expansion states representing roughly three-fourths of total estimated new Medicaid sign ups.

Now, on the surface, even the high end of this range (1.8 million people) would appear to be wildly divergent from my own recent analysis, which currently has "new Medicaid/CHIP enrollments" at around 7.5 million. However...

...Avalere isn't trying to count how many "new Medicaid/CHIP enrollees" there have been; they're only trying to count how many of those new enrollees are included specifically due to the ACA itself. In other words, they're not including the so-called "Out of the Woodwork" enrollees: People who were already eligible for Medicaid prior to the expansion provisions, but who never enrolled before for a variety of reasons. These reasons include a) not knowing that they already qualified; b) not even knowing that Medicaid existed before (believe it or not, these folks do exist); c) not knowing how to go through the application process; d) being too ashamed/proud to admit they needed help; or e) finding the application process to be too complicated. For these people, the ACA has operated primarily as a massive outreach/promotional/advertising campaign, along with streamlined/simplified application processes and so on. Note that the "woodwork" enrollees aren't mentioned at all in the Avalere article:

Reports from the Centers for Medicare & Medicaid Services have indicated that at least 6.3M individuals have been determined eligible for Medicaid between October and December 2013. These figures include a number of individuals who would have normally enrolled in Medicaid absent the ACA, including regular program churn and renewals.

Now, whether these people should be included or not is a matter of perspective. I personally feel that they should be included, but you could certainly argue that they shouldn't. This is why, while I do include the full 7.5 million figure in the Graph, I also make sure to include a strict "Expansion Only!" number prominently at the bottom of the Medicaid/CHIP spreadsheet as well. As you can see, as of this writing, that number stands at about 3.7 million.

I arrived at the 3.7 million figure by first eliminating all Medicaid/CHIP enrollments from the 25 non-expansion states. If the state hasn't expanded Medicaid, obviously no Medicaid enrollees would "count" as being part of ACA expansion using this strict interpretation. This drops the total down to about 5.45 million people. Then, for the remaining 25 states, I'm using Washington State as a baseline--they're the only state which specifically separates out "expansion" enrollees from "woodwork" enrollees (ie, previously elgible but hadn't enrolled before), and in their case, the ratio is about 68% "expansion" to 32% "woodwork". Expanded across the 5.45 million, that comes to about 3.7 million. I realize this is pretty crude, but it's all I have to go on at the moment. I really wish other staes would separate these out, but it is what it is.

OK...so the apples-to-apples comparison now appears to be  3.7 million to 1.8 million. This is quite a bit less dramatic of a difference, but it's still pretty steep, so let's see if we can find the remaining 1.9 million people.

My own methodology is laid out in detail here. Let's take a look at how Avalere arrived at their numbers:

Avalere developed the estimates through comparing reported data on new enrollments from October through December 2013 to enrollment rates from the summer of 2013.

Avalere Health compared the number of new Medicaid and CHIP applications from October through December to the average monthly number of applications submitted from July through September of 2013 (“the control”).  We then applied this comparative rate to the total number of determinations, with two sets of assumptions for those assessed/determined by exchanges, which comprise our lower and upper bound estimates outlined below.  

Right off the bat I'd want to know whether Medicaid enrollments are seasonal--that is, do enrollments typically fluctuate dramatically between the summer and late fall months, or is it usually fairly even between the two? I'd also want to know whether enrollments this past summer were especially unusual, perhaps in anticipation of the upcoming exchanges launching? I have no idea whether either of these are a factor, but they're both worth looking into. (Note: See UPDATE below)

In their notes, this one jumped out at me right away:

In addition, in a limited number of cases, states have reported households as opposed to individual applicants, and this is not adjusted for. 

According to the CMS reports, there are 5 states which report households instead of individuals: Alaska, Connecticut, Nevada, North Carolina and Oregon.  According to the US Census Bureau, the average U.S. household has roughly 2.6 people. However, Covered California uses an average of around 1.8 people per application, so I err on the side of caution and use this instead. The tallies for those 5 states comes to around 280,000 households; multiplying those out by 1.8 gives about 504,000. This immediately accounts for around 224,000 of the difference. We're now down to a difference of less than 1.7 million: 3.71 million vs. 2.02 million. (Note: See Update below)

The following states did not report control data for applications submitted; thus, we use the average expansion state rate as a proxy: DE, IL, NY, RI, and WA.  UT also did not report control data so we use the average non-expansion state rate as a proxy.

About 25 states have expanded Medicaid and another 25 haven't. On the non-expanded side, I don't see this as making too big of a difference, since Utah's 61,000 uninsured residents who are eligible for Medicaid/CHIP only represent about 1.7% of the 3.625 million people eligible for Medicaid/CHIP in these states.

On the other hand, Delaware, Illinois, New York, Rhode Island and Washington combined have 2,292,000 people eligible for Medicaid/CHIP out of a total of 10.342 million. That's over 22% of the eligible population of the ACA-expanding states. Given that these states have very different demographic breakdowns from most of the other 20 states, I'm not sure how reasonable it is to average them out.

MA did not report Medicaid and CHIP determinations in any month.  Thus, applications are used as a proxy, which may overstate the estimate for this state.

I'll come back to Massachusetts later, as this is part of a major distinction in what Avalere and I are including.

The final 3 notes are also pretty important:

PA did not report Medicaid and CHIP applications or determinations in the December CMS report.  To adjust, Avalere used a 2-month rate calculation for applications and the average of October and November determinations for December.

OH did not report Medicaid and CHIP determinations in the December CMS report. To adjust, Avalere used the average of October and November determinations for December.

WI did not report Medicaid and CHIP determinations in any month.  Thus, applications are used as a proxy, which may overstate the estimate for this state. 

I wouldn't make too much of the Wisconsin note, but the Pennsylvania and Ohio notes seem significant. Using the average of October and November would be fine if every month was roughly the same...but as everyone knows, December saw a huge increase in enrollments on both the private and Medicaid/CHIP side (via the exchanges, at least; overall direct Medicaid enrollments seems to have been only a bit higher in December). In fact, the HHS reports (which only cover the actual exchange enrollments for Medicaid as well as Private QHPs) show roughly 400,000 Medicaid enrollments in both October and November, but almost double that (780,000) in December. Ohio and Pennsylvania are both pretty large states, so I wouldn't shrug off their impact.

On the other hand, I've already admitted to using 1 state (Washington) as a basis for the expansion-to-woodwork ratio across the other 24, so who knows. I don't really know what to make of these various "averaging" effects. They may all cancel each other out, but should at least be kept in mind.

However, all of this goes back to the most important factor: What are each of us "counting" as "New Medicaid/CHIP enrollments" and what aren't each of us "counting"?

Here's where the Massachusetts note comes into play. As far as I can tell, Avalere isn't counting the 130,000 people that Massachusetts has automatically transferred from their state-run healthcare program, Commonwealth Care, over to MassHealth (MA's name for Medicaid/CHIP itself), due specifically to the ACA's Medicaid expansion provision.

From what I can tell, Avalere also isn't counting the 630,000 people in California who were transferred over from the LIHP (Low Income Health Program), a similar state-run healthcare program which was created, again, specifically thanks to the ACA's Medicaid expansion provision.

In all, there are over 984,000 people in this sort of situation in various states--they were previously enrolled in some sort of state-run healthcare program, but have been automatically moved over to Medicaid/CHIP itself, specifically due to provisions in the ACA. The problem is that technically speaking, none of them were transferred over until January 1st...AFTER the December cut-offs of the HHS & CMS reports.

Once you either add these to Avalere's number (and yes, in this case I really think they should be counted as "newly enrolled in Medicaid due to the ACA"), we're now talking about 3.71 million vs. 3.00 million (or, if you use the low-end Avalere estimate, 2.3 million).

But wait, there's more!

As noted above, Avalere's data only includes Medicaid/CHIP enrollments through the end of December...and in fact, not even quite that...the HHS report only runs through 12/28, leaving out 3 days. My own data has been updated continuously as it's released. Medicaid data on my spreadsheet has been updated for 15 states since the end of December. The largest state, California (over 12% of the total population), is updated through 1/15; Colorado, Kentucky, Maryland, Minnesota, New York (3rd largest state), Oregon and Washington (which, combined, represent 14.7% of the total population) have all been updated through the end of January. 

Now, I'm not saying that these partial post-December updates have added another 700,000 to the total (or up to 1.4 million if you use Avalere's low-end number), but they certainly cut the difference down to a respectable level.

So, am I saying that Avalere is "wrong" that that I'm "right"? Not at all. We're trying to answer slightly different questions here, and are just using different methodologies to get there. This doesn't mean that one of us is "correct" and the other one is "wrong", it just means that the answer really depends on what your question really is.

Deep Thought is a computer that was created by the pan-dimensional, hyper-intelligent species of beings (whose three dimensional protrusions into our universe are ordinary white mice) to come up with the Answer to The Ultimate Question of Life, the Universe, and Everything. Deep Thought is the size of a small city. When, after seven and a half million years of calculation, the answer finally turns out to be 42, Deep Thought admonishes Loonquawl and Phouchg (the receivers of the Ultimate Answer) that "[he] checked it very thoroughly, and that quite definitely is the answer. I think the problem, to be quite honest with you is that you've never actually known what the question was."

Deep Thought does not know the ultimate question to Life, the Universe and Everything, but offers to design an even more powerful computer, Earth, to calculate it. After ten million years of calculation, the Earth is destroyed by Vogons five minutes before the computation is complete.

The bottom line is that figuring out the "correct" Medicaid/CHIP enrollment numbers is a royal pain for anyone. There's a tremendous amount of "churn", with people moving onto and off of the program as their circumstances change--women getting pregnant or giving birth; people gaining or losing jobs that change their income level; people getting married or divorced; people moving from one state to another (which may have different rules in place for qualfying for coverage) and so on. When you add to that the philosophical question of whether or not to include people who were encouraged to enroll because of the ACA, it becomes pretty confusing, and I give props to Avalere for taking their own shot at it.

I should note that since I posted this last night, I've already heard claims that Avalere is either "anti-ACA" or "pro-ACA". I honestly can't tell, which is probably a good sign. On the one hand, they aren't "counting" several million people who I think they should, making their final tally considerably lower than mine. On the other hand, the language of their actual press release doesn't seem to include any negative red flags (ie, it doesn't say "only" 1.8M, and they include a fairly positive quote from their Vice President concluding “If the increased rate of enrollment continues, we could see Medicaid rolls grow substantially throughout 2014.”

(UPDATE): I just read the Washington Post's story on the Avalere report, which includes quotes from Diane Rowland of the Kaiser Family Foundation which fully support two of my major points above:

Some Medicaid experts were skeptical of Avalere’s conclusion because it didn’t take seasonal variations into consideration. During the summer months, “we tend to have an influx of farm workers who are temporarily on Medicaid,” said Diane Rowland, executive vice president of the Kaiser Family Foundation.

Again, this is important because if the July - September period normally has a spike in enrollment anyway, it could dramatically reduce the apparent impact of the following quarter. An easier example of this would be retail sales during the holiday shopping season. Let's say that year in and year out, Apple sells 10 million iPhones per quarter during the Winter, Spring and Summer quarters but sells 30 million iPhones over the Holiday quarter. If they then sold 35 million the following quarter (Jan - Mar of the following year), it's not reasonable to compare those sales against the 30M sold in the Holiday quarter; you would compare it against the 10 million they sold in the previous Jan - Mar quarter. Instead of a 17% increase (35 vs 30) you'd be talking about a 350% increase (35 vs 10).

In addition, Rowland said, the study uses new and incomplete data. Some states count the number of individuals who sign up, while other states count applications, which may include four or five people.

Again, I'm being very conservative here, only multiplying household numbers by 1.8x, not "4 - 5".