ACS Data Users Group

I hope everyone is healthy.

ESRI has some estimates of when estimates are unstable in the ACS tables

They use MoE to calculate coefficient of variation and have flags for low, med, high reliability. Looks like the formula you wrote below.

Coefficient of variation (CV) = 100* [(ME/(1.645)] / estimate

https://storymaps.arcgis.com/stories/514a53bbcfd44b4ea2992b4c40059ef4

and see

https://doc.arcgis.com/en/esri-demographics/latest/regional-data/acs.htm#ESRI_SECTION1_805FF6F174ED48059E26696F0A440571

Does the Census Bureau have something that says when the MoE indicates estimates are unstable?

Thanks

Gene

Hi Gene -

Hi Gene -

I don't have a direct answer to your question. However, there was a pretty good discussion about interpreting coefficient of variation in this forum about a year ago

Different coefficients of variation with same information - Forum - Discussion Forum - American Community Survey Data Users Group (prb.org)

From what I remember, Esri tends to apply their reliability standards pretty indiscriminately - often to proportions / percentages. This has the problem that Matt and Jonathan describe in the above post.

-- Dave

Thanks, a very interesting discussion.

David Wong and Min Sun wrote about this set of issues in a paper published back in 2013 in Spatial Demography. This may be useful to some

Handling Data Quality Information of Survey data in GIS: A Case of Using the American Community Survey Data.

https://link.springer.com/content/pdf/10.1007/BF03354884.pdf?pdf=button

Best

- Stephen

Thanks very much for the link to this paper.

I am afraid that the Bureau actually miss handled there computation of MOE, if one is a frequentist, if one is a Bayesian that approach is even worse. On the frequentist, I had a debate with Bureau a long time ago, where they indicated that they realized that they had made a mistake/ (Some of their margins of error go negative, in other words they are saying that their is chance that there will be negative cases.) This is obviously absurd. They used the wrong approach to computing error altogether. That has been corrected for some data using Random Replicates, which are also available in the PUMS data. Here is a link to my correspondence with the Bureau and their response.

https://www.dropbox.com/s/lfjo11wm5ma1axx/Memo_Regarding_ACS-With_Response.pdf?dl=0

Here is a link to the Random Replicate Explanation.

www.census.gov/.../variance-tables.html

For small numbers or when any of the percentages are less than 30% or more than 70% one should use random replicates. The MOE assumes a normal distribution which is not true in such cases. As another example their estimate of the MOE of median income assumes a normal and symmetric distribution of income. There are analyses that show that the MOE of median income approaches zero as it becomes more skewed. Guess what Income is highly skewed. They use a multiple well above one for MOE of median income based upon the standard error of the average, These more accurate approaches to standard error on covered in all of the major statistical packages.

Andy

Thanks very much!

Interesting discussion happy to see Andy refer to the frequentist vs Bayesian issue I read with great delight in The Signal and the Noise. Weighting is another issue you can't address this way, right? Couldn't weighting be happening in the background to adjust some of the sampling issues showing up in these MOEs?