Data Discussing

A couple of friends recently posted Facebook items related to this link at Makers, which says it shows the "best and worst states for women in 2017." The Makers link goes to a study reported at MoveHub, with a story written by Patrick Gilligan.

Makers is a networking site for women in business, with stories, videos and interviews with successful women in a variety of fields. MoveHub is a site that offers information about different areas for people considering moves, focusing on moves overseas.

It doesn't take a lot of time on the internet to find a host of infographic-y items that rank states, countries or regions according to different criteria. Even the significant chunk that aren't outright clickbait dangle the ranking lure in front of readers, banking on curiosity and the competitive impulse to grab an eyeball and make it hit that link. Depending on the criteria you like to use, it's not hard to get a set of rankings that puts a preferred area on top or on the bottom.

But the majority of these pieces, clickbait or no, are junk -- or at the very best can offer no proof that they're not junk, unless the ranking being discussed is a narrow one based on clearly measurable data. The MoveHub piece, for example, cross-referenced "data for the gender pay gap, political representation in the state legislature, equality in education, accessibility to health insurance, reproductive rights and the number of incidents of violence against women at the hands of men." The colored map in the story lists the sources for different studies used in the cross referencing, but leaves out the individual studies themselves. We also don't know which group was responsible for which set of data -- sure, you can guess that the National Conference for State Legislatures is the source for statistics about how many women serve in elected office per state, but what data do the Americans United for Life or Kaiser Family Foundation provide?

We don't know if these studies dealt with the same kinds of populations, beyond a reasonable assumption that they compile data about women. So we don't know if a combination of their results to produce rankings is reliable -- unless similar groups were studied, then the data won't combine well. Did the MoveHub researchers weight some criteria more heavily than others? For example, did rates of violence against women by men figure more heavily into the final tally than equality of education? It's not out of line to note that female victims of violence by men come from many educational backgrounds, nor is it out of line to wonder if some women might consider the safety of themselves and their children a little more important than access to doctoral programs. Perhaps they don't -- but nothing in the MoveHub story clues us in about which it might be.

When we look at the criteria MoveHub used, we can raise some more questions. How were these chosen? Were women themselves surveyed to find out that these five were their top concerns? If so, where is that data? Once they were chosen, were rankings from just one study put into the MoveHub formula, or were several combined to help correct for the possibility of outliers or inadvertently skewed results? If just one study was chosen: Did it have the best data set and survey methodology, or was it at the top of the Google search results? Or did it have the result that best matched the author's intent? 

Why did MoveHub only use the number of incidents of violence against women by men? Wouldn't women also be concerned about incidents of violence against them by women, too? Hawaii is ranked at the top of the 50 states, singled out for its extremely low rate of women murdered by men. Would the rate change if it was simply women murdered? A woman murdered by a woman is just as dead as a woman murdered by a man. It might not change the rate or the rankings in that category much at all, but we don't know based on the info MoveHub uses.

And what do some of these categories actually measure? Did the gender pay gap study compare men and women with equal or similar jobs, or did it lump all jobs together without considering what difference that might make? "Reproductive rights" is a pretty broad term, even though a lot of folks on both sides of the issue seem to want to limit it to abortion. Access to abortion? Public funding for in vitro fertilization for women who can't have children? What markers did the study or studies MoveHub uses measure in ranking the states? In the paragraph about the worst state, Oklahoma, we're told that legislation was introduced that would require a woman seeking an abortion to have permission from the baby's father. But did that legislation ever become law? (Spoiler: No) So what quantifiable difference does it make?

What's "equality in education?" The state's two top universities split the difference on male-female enrollment, with the University of Oklahoma 51-49 in favor of women and Oklahoma State University 51-49 in favor of men. The third-largest school, the University of Central Oklahoma, goes 59-41 in favor of women, and most of the other smaller regional colleges in state cluster around that 3:2 ratio. How did whichever group that conducted that study rank states? What made a state do well on their study?

And to close, what was the actual statistical difference between the top and bottom tiers in the study? Yes, there's a nice map and numerical ranking that puts Oklahoma and four other states in a deep warning shade of red to show how awful they are, but what were the actual differences in that murder rate, mentioned above, between Hawaii and Oklahoma? Or between any other two states? If I make a billion dollars a year and eight others make a billion dollars plus one apiece, I rank at the bottom of the income stats. But I probably won't care, because the difference is so very small and because I've got a billion reasons not to.

You might think I'm having a little hissy about this one because it puts my state at the bottom and there may be something to that, although I dismiss the same kinds of ratings fluff that put us towards the top as well. And for mostly the same reasons: No clarity about how the data are selected or weighed, no clear definition of categories or rationale for choosing them, and nothing to suggest that any of the data have been compiled or researched by anyone with any training or background in statistics or surveying.

Oklahoma is like a lot of places in that it could move faster to help women gain real parity and equality in its society, culture and governing systems. It probably lags behind some but is ahead of others. Arguments that it's the best place in the world for women will lean on some weak reeds indeed and will get shut down before very long. But "the worst state for women?"

I'm not a woman. Except for three periods of five years or less in other states, I have only lived in Oklahoma, so my experience of other states is limited. Moreover, I wasn't a woman when I lived in them, either. So is Oklahoma the worst state for women? I don't know, but neither does MoveHub.

The difference is I'll admit it.