Regarding Prof. Theda Skocpol's ACA Enrollment Brief

Yesterday, Theda Skocpol of Harvard University cited the data at this site in a brief about state progress in implenting the Affordable Care Act posted at the Scholars Strategy Network website. This brief--particularly the accompanying visual graph--was in turn cited in articles posted at both Talking Points Memo and Mother Jones. The gist of both the brief as well as these articles is that the states which have been cooperating with the ACA have been far more successful in enrolling people in both private insurance plans as well as publicly-funded plans such as Medicaid, SCHIP and related state-run programs.

This conclusion isn't especially shocking or unexpected, but the brief and again, especially the chart both make the differences between the "Full-Go" states (those which are fully on-board in trying to make the ACA work) and the "Just Say No" states (those which haven't expanded Medicaid and dumped their exchanges onto the federal government) seem especially striking.

The only problem (or at least, perceived problem) with this is that there was what I feel is a vitally important chunk of data not included in the brief (and therefore the other articles): The roughly 1 million unspecified private enrollments and roughly 1.85 million unspecified Medicaid/CHIP enrollments that are clearly noted at the bottom of the spreadsheet, just below Wyoming. These enrollments have been officially announced by the HHS Dept, but simply haven't been broken out between the 36 states running off of the Federal exchange, because HHS hasn't released that data yet. In most cases, those states' data only runs through November 30, although there's 10 states which have partial updates since then (Alaska, Delaware, Iowa, Mississippi, Nebraska, New Mexico, South Dakota, West Virginia, Wisconsin and Wyoming). However, even in those cases the data is very incomplete (either only covering part of December or only covering one of multiple insurance companies operating in those states).

If the missing data was evenly distributed across both the 36 states in the Federal exchange as well as the 14 state-run exchanges, this might not matter since you could extrapolate (roughly) the numbers fairly well. However, we're talking about a total of 2.85 million enrollments out of roughly 6.5 million total...all of which is completely on the 36 Federal exchange side.

As a result, there's been some understandable confusion about how to interpret Prof. Skocpol's brief and accompanying chart. The TPM article does make a brief mention of the missing data very late in their article:

The data that Skocpol used included more recent enrollment figures for state marketplaces, but nothing more recent than Nov. 30 for the states using HealthCare.gov. The Obama administration has said that about 975,000 enrolled nationwide through the federal site in December.

...and the Mother Jones article has since issued the following, somewhat more pointed correction to theirs:

POSTSCRIPT 2: There's nothing in Skopcol's report that says this, but apparently the state data is through January 4 while the federal data is through November 30. This makes comparisons between the two impossible, and it makes the chart above worthless. Please ignore this entire post.

Unfortunately, as the guy running the spreadsheet which was the original source of the data, I can't simply ignore either the MJ post nor the original brief. There are those who might read these articles (and the corrections) and start to question the integrity of my data, which is as accurate and transparent as I can make it. To this end, I've been in touch with Prof. Skopcol, who has given me permission to post the following statement. It's rather lengthy, but there's one important point (bold-faced by me) which I need to respond to:

Yesterday, the Scholars Strategy Network posted a lead-off spotlight on my new brief about state progress in implementing Affordable Care and an accompanying chart showing where enrollments in Medicaid and the exchanges reported by state stand at the very start of 2014.  My collaborators and I understand that enrollments are a dynamic process, and reporting is sporadic and uneven. We plan to regularly update our chart and expect the averages of subgroups of states to change in relation to one another over coming weeks and months.  We especially anticipate changes when the federal government issues big data reports that allocate new enrollments by state.

The only data source I had for enrollments allocated by state as of early 2014 was Charles Gaba's spreadsheet as of 1/4/14, as I indicated in my brief.   For a few states in his spreadsheet, the last reported updates were right around that time.  But for most, they are earlier.  Some have argued that the reporting date for exchange enrollments was only 11/30 for all 36 states relying on the federal exchange.  That is not the case according to my reading of Gaba's spreadsheet. 

In quite a few cases, he had and comments on sources dated later than that. Overall, final reporting dates for the data I used on exchange enrollments vary a lot across the states.  I saw -- and see -- no way around the problem that data are not perfectly parallel in date or completeness of reporting, except to regularly repeat the exercise embodied in my chart. I will do that and make the results public.

I am glad to see the issue of state level developments getting a lot of discussion on the web because, frankly, Affordable Care implementation is not really a single national story; it is much more a story of state-level efforts and accomplishments.    As we look forward to the next big jump in state-allocated data, when the federal government reports in January, we can imagine various ways in which the gaps between more and less cooperative states in my first chart might change. Here I refer only to exchange enrollments, not to Medicaid:

(a) We may continue to see large exchange enrollment gaps between states with their own exchanges and states, whether pro-reform or not, that use the federal site for sign-ups.  This could happen if the effects of the first two months of web mess-ups on Healthcare.gov "live on" once steady progress happens across the board.

(b) We may see the 7 "Supporter" states that rely on the federal site begin to "catch up" with the Full-Go states, while the 23 "Just Say No" states continue to lag in adding exchange enrollments.   This could happen if a disproportionate number of the December enrollments at Healthcare.gov (NOT reported by state as of 1/4/14) turn out to have been happening in supportive states that were simply held back by the early woes of the federal site.

(c) We may see equal surges in exchange enrollments in all states without their own exchanges, regardless of whether they support the law or not.

It may also be the case that the groupings of states I use break down completely as time passes. There are variations within each category -- Nebraska, for example, has moved faster on exchange enrollments than other officially non-supportive states.    I plan to try to develop better measures of cooperation that capture degrees of quiet collaboration.   Not all "Just Say No" states are like Texas!

These are all empirical questions.  I will be interested to see how trends play out as updated data becomes available, and I will share the comparable chart publicly when I have it, no matter what it shows.  The first chart is a snapshot in time with the data available as of the turn of the year.  Perfect data will never be available, but we can keep tracking and filling in the blanks.  I am always happy to hear comments from colleagues who also care about the facts.

Theda Skocpol, Harvard University

I appreciate Prof. Skocpol's citing my work as well as her response, but I still have a problem with the bold-faced part above:

• As I noted above, as of this writing, the "reporting date" for just 10 of the 36 states relying on the federal exchange is later than 11/30.
• Those 10 states only represent around 20 million people total, or less than 10% of the total population of those 36 states.
• The "reporting date" for those 10 states is as early as 12/09 in some cases.
• Even then, the "reporting date" is only a partial figure in many case; for New Mexico, Iowa and Nebraska, for instance, the number only represents one out of 2-4 insurance companies operating in those states.
• Most importantly, the vast majority of enrollments for all 50 states (+DC) came in mid to late December...the very section of time which is least represented by the 36 Federally-run states, even accounting for those 10 partially-updated ones.

Now, my guess is that even after those 2.85 million enrollments (1 million private, 1.85 million Medicaid) are broken down state-by-state, there will still be a significant discrepancy between the "Full-Go" and "Just Say No" states anyway; it will most likely just be less dramatic. However, until that state breakdown is provided, this point must be taken into consideration when considering this study. I look forward to seeing the updated chart once that data is made public, however, and am honored to be a part of this study.