There's a saying out there that "you can get statistics to say anything." Folks often refer to this saying as a way of disparaging pollsters, researchers, statisticians, or intellectuals in general. The implication is that you just can't trust any of that pointy-headed bunch.
The truth is, of course, not so simple as that.
Actually, statistics never themselves lie; it's people who lie about statistics. In order to understand what a statistic really means, you have to know what the question behind the statistic is, and who the people answering the question are. When the people who answer a question accurately represent the population you're interested in talking about, and when the question asked offers an exhaustive, mutually exclusive and balanced set of possible answers, statistics about responses can have importance and meaning. On the other hand, if questions only offer limited possibilities, promote one answer in a leading fashion, pair up multiple answers under one option or are simply incomprehensible, resulting statistics only represent the researcher's warped reality. Any attempt under these conditions to tie answers to the actual state of folks' opinion is a bad idea, to put it mildly. In other words, Garbage In means Garbage Out.
A good example of rotten poll-taking can be found at Poll on Politics. TomG, a self-styled Matt Drudge "reporter" and member of the Conservative Coalition, asks a series of questions about political issues of the day. One simple problem with his poll page is that it's linked to by a conservative commentary page. Not surprisingly, conservative opinions are strongly represented in his results. What does this tell us, but that conservatives are conservative? It may nice to know that this trivially true notion is confirmed, but so what?
[An aside: TomG stuck this note on his site: "The writer of the piece on my 'Poll on Politics' says one problem with it is that the poll is linked to a conservative web site and the results are strongly of conservative opinions. Why is that a problem for the writer? Does he think conservative views shouldn't be heard?" Well, it's not a problem if you call it "The Poll of People Who Already Agree With Me." It sure as heck is a problem if you're trying to portray your poll as a general "Poll on Politics," with a picture of a donkey on one side and an picture of an elephant on the other.]
Another big problem lies with the questions themselves. A number of polls offer a combination of two statements in every option. One of the two statements inevitably skewers Clinton or some other Democrat. Take, for instance, the poll, "How successful will the Bush administration be?" The available options are:
A) Very successful like the Reagan administration
B) Somewhat successful like his fathers administration
C) Unsuccessful like the the Carter administration
D) Corrupt like the Clinton administration
Anybody who answers the poll registers their support for a Republican President or their disdain for a Democratic President. The only option is to agree with TomG. This is very convenient for TomG.
|Another question that either sneakily or unwittingly lends only credence to George W. Bush asks "What is the most impotant [his spelling - a Freudian slip?] issue that George Bush should promote?" The options, "tax cuts for taxpayers," "rebuild our military," "develop missile defense system," and "privatize social security," are all issues that George W. Bush already promotes. There is absolutely no choice for the poll's respondent but to either endorse the Bush agenda or not respond to the poll -- and, of course, non-respondents are not counted by TomG. The results' seeming endorsement of the Bush agenda doesn't really tell us anything except, again, what the pollster thinks.
Other polls offer false dichotomies - sets of options that aren't mutually exclusive. One poll asks "Do you want more laws on guns or do you want greater enforcement of existing laws?" This pollster doesn't allow for someone who wants both, or who wants neither. The resulting poll numbers are gibberish because they don't represent what people might truly believe.
I could conclude this essay simply by saying I find sites like Poll on Politics distasteful, but in tribute to unfairly loaded polls, I'll sum up with a loaded poll of my own. Please choose the option you like best: Unbalanced and misleading web polls are:
C) a waste of time
D) all of the above.
Visit Poll on Politics to read TomG's unintentionally humorous response to this article.
Had Enough? Didn't think so...return to Irregular Times
Visit the Irregular Bin
Riled up? Don't let it bottle up. Talk back!
Got irregular thoughts of your own?