“If The Polls Are To Be Believed…”

Did you ever notice that polls seem to be subject to more journalistic skepticism than other numbers? Perhaps it’s because there are so many polls available, and of such varying quality, that it’s easy to compare and contrast differences.

On the other hand, nearly all other statistics – especially those from “official” sources – are accepted prima facie. For example, the media ran unquestioningly with a Social Security Administration story about the dramatic increase in baby boys being named Barack, although a simple journalistic inquiry would have quickly deflated that claim. Also, every press story about autism in New Jersey now includes the tag that New Jersey has the highest autism rate in the nation. However, this claim is based on a study that was done, not nationally, but in only 14 states – and even the comparability of those state results is suspect. (I’ll be blogging more about this issue in a few weeks.)

Not that I’m arguing for less scrutiny of polls – far from it. My main concern is that while basic questions about all numbers go unasked before being reported to the public, it seems that only with polls do the media express doubt on the veracity of the numbers they themselves report, many times with banner headlines.

On NJN’s Reporters Roundtable this past weekend, one panelist after another – including a reporter from Monmouth’s own media partner – used the phrase, “if the polls are to be believed.” They were discussing the New Jersey GOP gubernatorial primary and, specifically, Chris Christie’s large lead over Steve Lonegan in polls released by Monmouth/Gannett and Quinnipiac last week.

Skepticism of election polls – as with any number provided for public consumption – is extremely healthy. Polls are designed to capture a snapshot of attitudes and behaviors at the time they are taken. And indeed, we tend to see a lot of consistency among reliable polls on “here-and-now” measures (e.g. a politician’s job performance rating). However, polls are imperfect tools when asked to predict the future, such as an election outcome. Yet they work reasonably well and are the best tools we’ve got – so we use them.

What concerns me about the “if polls are to be believed” line is that there tends to be little discernment about why one should or shouldn’t believe the polls. And that’s because there is a tendency to evaluate election polls only in their capacity to predict outcomes. Among the five journalists on RR, I recall only one actually discussing a poll finding other than the top-line “horse race” results.

There’s such a media fascination with the horse race numbers that it’s easy to shoot down polls as insubstantial, if in fact that was all the polls were asking about. However, some polls actually do strive to go beyond the vote intention question to understand the dynamics of the electorate.

Monmouth/Gannett’s GOP primary poll included 19 questions about the campaign. But media outlets tended to report only the horse race results, or at most, the results of one or two additional questions.

In fact, the poll included a wealth of information that could help shape media coverage of the campaign and indeed make that coverage more relevant to voters’ actual concerns. For example, in poll after poll, property taxes is named by New Jersey voters as the number one issue they want to hear the gubernatorial candidates talk about. However, less than half of the GOP primary electorate say they have heard any of the candidates articulate a plan on this issue, or indeed on just about any of the issues voters consider to be important.

Perhaps the media could better serve its readers and viewers by pressing the candidates more to address these issues rather than focus on horse race and strategy. That is, of course, if the polls are to be believed.