Wednesday, April 18, 2012

Gov. Christie Poll Rating Differences

Cross-posted at PolitickerNJ

There have been a couple of queries about Gov. Christie’s poll ratings released by Monmouth University/NJ Press Media this week compared to the numbers put out by Quinnipiac University last week.  Among registered voters, Monmouth has the governor’s positive job rating at 50% while Quinnipiac put it at 59%.

Each organization’s prior polls put the governor’s approval at 55%.  In other words, Quinnipiac zigged (up 4 points) while Monmouth zagged (down 5 points).  Why?

There have been 17 occasions over the past six years where Monmouth and Quinnipiac released New Jersey governor ratings within two weeks of one another.  These most recent results mark the largest ever difference between the two.

Since Chris Christie took office, the two organizations released three polls prior to today that were conducted within two weeks of one another.  In each of those instances, the governor’s job approval rating differed by exactly 3 points – in two instances the Quinnipiac number was higher, while it was lower in the other.

It’s worth taking a dive into the two polls’ demographics to see if anything there accounts for the difference.  Overall, the polls have very similar racial compositions, but Monmouth includes more cell phone interviews (19% compared to 12.5% for Quinnipiac).  This probably leads to a somewhat younger voter group for Monmouth.  In our most recent poll, 24% of the voter sample was under the age of 30.  Quinnipiac did not release their age demographics, but past polls hovered between 18% and 20%.  Younger voters tend to be more Democratic in their political leaning, so this could have an impact.

In fact, there are notable differences in the partisan composition of the two samples.  Monmouth’s poll puts self-identified Democrats at 37% and Republicans at 23% – a 14 point difference.  Quinnipiac’s sample is 34% Democratic and 25% Republican – a smaller 9 point gap.

Over the last 9 months, Quinnipiac released 6 New Jersey polls and Monmouth released 4.  The Democratic advantage in Quinnipiac’s sample ranged from 6 to 12 points during that time.  Monmouth’s Democratic edge was larger, but more stable at 13 to 14 points. New Jersey’s voter rolls puts the registered partisan split at 33% Democratic to 20% Republican – a 13 point gap.

During the past few months, Monmouth’s voter sample ranged from 34% to 37% Democratic and 20% to 23% Republican.  Quinnipiac’s polls ranged from 32% to 35% Democratic and 23% to 27% Republican.  That means Monmouth’s sample tends to be 1 to 4 points more Democratic and a similar 0 to 3 points more Republican than the official voter rolls.  Quinnipiac’s partisan sample tends to range near the Democratic registration numbers – from 1 point below to 2 points above – but is consistently 3 to 7 points higher in its Republican share of registered voters.

All this explains why Quinnpiac’s gubernatorial ratings have been more “Republican” than Monmouth’s in 7 of the last 8 polls conducted in close proximity of one another.  However, it doesn’t explain why the job ratings diverged so much in their recent poll releases.

So, we turn our attention to another culprit: the questionnaire. Both Monmouth and Quinnipiac use consistent question wording when rating the governor.  Monmouth also makes sure the question appears in exactly the same spot on the questionnaire each time we conduct a non-election poll – for the record, that would be question number 2, after a general evaluation of the state of New Jersey.

Quinnipiac, on the other hand plays around with the order in which they ask the governor’s job rating question.  In 8 polls over the past year, they asked Gov. Christie’s job rating as the first question in 3 cases and the 3rd question in one case.  For the remaining four polls, the governor’s rating question was slotted from #10 and #13 in their questionnaire.

When it was the first question, the governor’s positive job rating was only 44% to 47%.  At the number 3 slot, it was 53%.  At #10 or later in the interview, it ranged from 55% to 59%.  It's worth noting that the lower poll numbers came early last year, and were either closer to or even lower than other polls conducted at that time.  Hmmm.

In the most recent Quinnipiac poll, one of the questions preceding Gov. Christie’s rating presented him as a potential nominee for Vice President.  In other words, the survey framed the governor as a national figure before asking voters to rate his job performance.  Could this be why his rating among Republican voters in particular shot up to an astronomical 92%?

Pollsters know that job approval ratings can be impacted by the context of a poll interview.  That’s why most pollsters try to place these key trend questions in the same place in every questionnaire.  This increases our confidence that any changes in a politician’s ratings are due to real shifts in opinion and not an artifact of questionnaire inconsistencies.

I’m willing to venture that first naming Chris Christie as Mitt Romney’s potential running mate before asking New Jerseyans to rate their governor might have had a wee bit to do with the two polls’ divergent trends.

Other theories are most welcome.