A Land Line Only Poll? How Quaint!
This morning on Twitter I saw a lot of conversation about a streetcar poll done by the Cincinnati Enquirer. In it they kick off with the following statement [Poll: Most Oppose Streetcars]:
Residents of Cincinnati overwhelmingly oppose the proposed $128 million streetcar project, objecting 2-to-1 to City Hall’s plan to borrow tens of millions of dollars for a plan widely viewed as a waste of taxpayer money, an Enquirer poll shows.
Okay, so it’s a poll. A telephone poll. A telephone poll that only called people with land lines. Hmmm….
That would have all been well and good a decade or two ago, but with an ever increasing array of people ditching landlines and going exclusively cellular its validity is highly questionable. I have not had a landline since 2005 and know many others like myself.
Let’s see what the Pew Research Center has to say about that divide, after all they deal with number and polling every day [How Serious is Polling's Cell Only Problem] :
While the cell-only problem is currently not biasing polls based on the entire population, it may very well be damaging estimates for certain subgroups in which the use of only a cell phone is more common. This concern is particularly relevant for young adults. According to the most recent government estimate, more than 25% of those under age 30 use only a cell phone. An analysis of young people ages 18-25 in one of the Pew polls found that the exclusion of the cell-only respondents resulted in significantly lower estimates of this age group’s approval of alcohol consumption and marijuana use. Perhaps not surprisingly, excluding the cell-only respondents also yields lower estimates of technological sophistication. For example, the overall estimate for the proportion of 18-25 year olds using social networking sites is 57% when the cell-only sample is blended with the landline sample, while the estimate based only on the landline sample is 50%.
I’m in my mid forties and more than half of my friends and colleagues have no land line. I’m not talking kids here , I’m talking about technology professionals ranging from early 30s to late 50s. Since a lot of the comments on the poll include people talking about cell phone only households being just kids I feel obligated to point that out.
So this is a survey of 600 people out of over 300,000. One five hundredth of the population as a representative sample. Hmmm….
Next let’s take a look at John Schneider’s post on the UrbanOhio forums:
Just got off the phone with someone who knows a lot about polling. He says the Enquirer is really parsing the results of this poll.
Here’s the gist of his take: 24% of Cincinnatians think it will revitalize the city’s core, while another 20% say its a risky project but that it ought to be built. That’s 44%. Another 48% think it’s a waste. He says for a project that’s not built, which the public understands very little because they have nothing to compare it to, being only four points down in the poll is pretty good. Plus 7% are not sure. And there’s a 4% margin of error.
Here’s another thing that pops out at me: 19% of Cincinnatians think they would use the streetcar almost every day. Besides water, sewers and streets, do you think 19% of Cincinnatians can name another City of Cincinnati service they expect to use every day. I can’t.
Someone wrote me earlier: “Looks like the Enquirer has declared war on the streetcar.”
Wanting to get some info of my own I looked to our local voices on twitter. Here are a few of them:
margyartgrrl good survey design is key -@cincienquirer has bad ordering and framing of questions; poor design on answer options.kate_the_great We’ve had convos in my office about these polls and how they skew the audience b.c so many people now landline-less. No brainerRandySimes Kind of the same. Here it was when the Enquirer decided to appeal to northern ‘burbs by trashing city.CinNewTon It would be nice if the old guard in Cincy would stop & consider the future instead of worrying about the past.LivingInGin Enquirer probably conducted the poll via telegraph, so only Tom Luken’s nursing home companions could respond.Kate_The_Great (again) While I was napping: great twitter convo about polls via telephone re: streetcar. Y’all are spot on. Landline= flawed BC narrow audience.
So what are your thoughts? I see a bad case of flawed methodology in execution of the poll. Bad data.
-Loki, Founder and Publisher