The Critical 'I'

Read. React. Repeat.

Thursday, January 22, 2004

Take it from someone who knows about opinion polls: Amateurs should never undertake them. The American Family Association found that out the hard--and, for the rest of us, funny--way, as they had to cancel their plans to present results of their online poll on same-sex marriage after the responses skewed heavily toward acceptance of such unions, to the consternation of the Association:

"We're very concerned that the traditional state of marriage is under threat in our country by homosexual activists," said AFA representative Buddy Smith. "It just so happens that homosexual activist groups around the country got a hold of the poll -- it was forwarded to them -- and they decided to have a little fun, and turn their organizations around the country (onto) the poll to try to cause it to represent something other than what we wanted it to. And so far, they succeeded with that."

Note the quote: " cause it to represent something other than what we wanted it to". News flash, chief: You don't conduct a poll with preconceived notions about what the outcome will be; you run a poll to discover what a population thinks about an issue or set of issues. Obviously, they weren't interested in finding out anything--rather, they just wanted to throw some numbers together that would validate their agenda. It's the wrong way to approach a survey like this.

Now, Smith says, his organization has had to abandon its goal of taking the poll to Capitol Hill.

"We made the decision early on not to do that," Smith admitted, "because of how, as I say, the homosexual activists around the country have done their number on it."

So what happened?

Against the wishes of the AFA and its members, the poll leaked to the outside. And soon, people like Gabe Anderson began posting it to blogs, social-networking sites such as Friendster and sundry e-mail lists. When Anderson posted it to his blog on Dec. 18, 2003, the anti-gay-marriage position was leading, with 51.45 percent of respondents opposing gay marriage or civil unions.

It goes without saying that you don't open up a poll to the whole online world and expect it to remain a secret shared only by a select few. Especially with this subject matter, the opportunity for abuse is obvious, and is going to be exploited by people with strong opinions on both sides of the debate. You have to employ actual sampling techniques and have a controlled polling environment--usually meaning direct feedback from participants via personal, telephone and email interviews. A free-for-all via the Web isn't going to stand up to scrutiny.

The truly funny thing about this is that, at root, the AFA is correct about the results of its flawed survey being way off. For months, national and regional polls have shown that most people are opposed to legitimizing homosexual marriage, although not by a huge margin, and always with some qualification regarding the details. In fact, one of the most recent such surveys, from ABC News and the Washington Post, reinforce this: 55 percent of Americans believe such unions should be illegal, but don't feel that warrants something as drastic as a Constitutional amendment.

So there was no need for the AFA to even embark upon their own survey, and thus wasting their time and resources (although judging from the way this was carried out, I doubt they devoted much of either toward this) ; they would be better off just citing all the existing evidence to back up their case. In the event that they feel 55 percent assent somehow isn't convincing enough, they could always do a polling among just their own membership; that should get them something like 90 percent, minimum.

(Via MemeMachineGo!)