Adventures in Campaign Message Polling, part 2

Cross-posted at PolitickerNJ

I recently wrote that a publicly released campaign poll memo was from a message testing survey, with the results presented out of context. I’ve had some experience with message testing polls, specifically working with non-profit organizations on crafting communication strategies.

My most recent experience with message testing polls, though, was as a respondent. A few weeks ago, I was called on my home phone to participate in a message testing poll conducted on behalf of a local campaign. With a plethora of campaign polls now underway, this recent experience provides a good lesson on what goes into a message testing survey – and why the media should be wary of reporting any results from an internal campaign poll. [It also provides a good lesson on the difficultly of avoiding at least a little bias creeping into partisan polls.]

The first question is how my name got chosen for this poll. Simply, I vote in every general election and so I am very likely to turn out for this off-year election. Furthermore, as an unaffiliated (i.e. independent) voter, I’m part of the “persuadable” electorate for whom campaign messages are specifically crafted.

After establishing that I didn’t work for a political or media organization, the poll interviewer’s first question was whether I thought my local area is headed in the right direction or on the wrong track. This was followed by a generic horse-race question, i.e. whether I was likely to vote Democratic or Republican for the local offices up for election this November. This is a standard question to establish a baseline, since most voters use party ID cues as their primary vote decision tool. It was also the first of three times I would be asked to state my vote intention during the course of the interview – a key characteristic of message testing polls.

The next set of questions asked me whether I have heard of the incumbent officeholders up for re-election and what my overall opinion of them was. Again, this is standard stuff – incumbent elections are typically referenda on the current officeholders. The next question then presented head-to-head matchups for each office, but this time naming the two candidates for each office. This was my second shot at expressing a vote choice, because any change from the generic party ballot question asked earlier could indicate underlying strengths or weaknesses of the named incumbents.

The next questions asked me to name my top local issue and assess my local government’s performance. The purpose of these items is to uncover any unknown issues before the poll measures the impact of potential messages already drafted by the campaign.

We then moved on to the meat of the matter. The interviewer read some fairly long positive descriptions, i.e. messages, about both candidates for each of the offices on the ballot. After which I was asked again about my vote choice – for the third time.

Two things are important to note here. First, an internal poll “memo” which releases the results of this third question without mentioning the context would be misrepresenting actual vote intention of the existing electorate – because the poll respondents had more information about the candidates than typical voters have – and that information was coming one side only.

Second, this is the point where I figured out who sponsored the poll (i.e. the challengers). As hard as this pollster tried to be balanced in wording the positive descriptions for both party’s candidates, the descriptions for one slate of candidates had just a little more “zing” in the wording. This subtle difference could have an unintended impact on the results of the third vote choice question.

To be fair, the word choice may not have been the pollster’s. I’ve worked with partners who insist that a particular word or phrase “needs” to be included in the question. Sometimes, you are successful in talking them out of it, and sometimes you just go along in order to move the project forward.

Question wording is at the core of the art of polling. It deserves as much scrutiny as the demographic composition of a sample and the poll’s margin of error. This is why reputable pollsters release the full wording of all the questions they ask. And it’s why the media should never report a poll where the pollster refuses to release the complete questionnaire.

Back to the survey interview. The final set of questions – before closing with basic demographic information – presented some negative information about the incumbents (confirming my suspicions about the sponsoring party). I was asked whether knowing this information would influence my vote. Again, this is standard stuff.

Interestingly, very few messages were tested in this poll. In a competitive high-profile race, each campaign will test a variety of pro and con statements to narrow down their communication strategy to the most effective messages. In this instance, only one or two messages about each incumbent were tested. This indicates a race where the decision may not be which messages to choose, but whether spending any resources will be worthwhile and, if so, how to identify the most pliable segments of the electorate.

By the way, this was a pretty good message testing poll given the election in question. The interviewer was of very high quality and the questionnaire was well-crafted, my observations about the positive candidate description imbalance notwithstanding.

There is also an interesting side note to this story. I confirmed the identity of the poll sponsor through an Internet search of the firm name and a review of Election Law Enforcement Commission expenditure reports. When I called representatives of both the pollster and party organization to corroborate, they were noticeably flustered. One said he’d call me back, but never did. The other answered my questions mainly with “um” or “er.”

Their reaction underscores the fact that campaigns tend to treat their internal polls as state secrets. Typically, they don’t want anyone outside the campaign organization to know what their poll results reveal. Indeed, they usually don’t want anyone to get wind of the fact they are polling at all. All of which makes any publicly released internal poll immediately suspect.

So, my advice to the media is if a campaign is suddenly eager to release poll results to a wider audience of “interested parties,” consider the motive. And then just file it away.

[Note: I wish campaign pollsters would be more forthcoming with their contact information at the end of the interview, since their conduct reflects on the whole profession. However, I decided not to identify the sponsor of this poll since their practices were sound and the primary purpose of this article is to foster a more critical eye toward the public release of internal campaign polls rather than “out” any particular campaign.]

Is Obama in Need of Some Tom Foolery?

Cross-posted at New Jersey Newsroom

One of the hardest things for a president to do is to stay in touch with the concerns of everyday folk. It’s just the nature of holding a public office, but there are certainly degrees of distance. It is easier for a state legislator to mingle among the masses than it is for a governor. And it is much easier for a member of Congress to grab a bite in a local deli than it is for the President of the United States. Every public official is treated with some deference, but the higher up the political food chain, the more likely it is that encounters with the public will lack some authenticity.

I viewed the live news coverage of Barack Obama’s visit to the Tastee Sub Shop in Edison, New Jersey last week with this in mind. I watched as he disembarked from Air Force One to greet Governor Christie and Mayor Booker – shaking the former’s hand for a prolonged 34 seconds! [I’ve looked at the footage a few times and I’m still not sure who refused to let go.]

I watched as his motorcade pulled up to the shop, blocked from view by a strategically placed delivery truck. I watched people gathered behind barricades yards away wondering whether the President had arrived. Not very riveting stuff.

A while later, a news pool camera came to life and Obama spoke for a few minutes to selected members of the press corp. He talked about the meeting he just had with a handful of local small business owners and called for passage of tax credit legislation to aid small businesses. Then he was whisked off to New York to tape an episode of The View and headline a fundraiser.

So, on his trip to New Jersey (population 8.7 million), President Obama apparently spoke with a grand total of eight state residents: the governor, the mayors of Newark and Edison, and five small business owners. And according to reports, the president, typing on his Blackberry, barely acknowledged the crowds lining Plainfield Avenue during his drive from Marine One’s landing site a few miles from the shop.

Some have questioned the purpose of this presidential visit. If he really wanted to push for passage of the tax credit bill in question – which stalled in the Senate the following day – wouldn’t a high profile event with public statements by small business owners been more effective? We don’t even know what those five business owners said to Obama during that private meeting in Edison.

But maybe we should consider for a moment that generating support for legislation may not have been the president’s primary motivation for this visit. In a recent interview, Obama biographer Jonathan Alter said that the president is not avoiding the isolation of the office “as well as he needs to.” This trip may have been an effort to break through that isolation.

But why come to New Jersey at all? Considering the logistical hassle and cost to taxpayers, wouldn’t it have been cheaper to bring those five business owners to the White House? Cheaper, yes. More effective, maybe not.

By all accounts, it is difficult to be entirely honest with the leader of the free world when you meet him. Nearly everyone who has a presidential encounter reports being awestruck in some way, even if they vehemently disagreed with the incumbent’s politics. It follows that the rank and file citizens President Obama meets in his occasional excursions outside the White House are not always completely candid with him. Inside the White House, that likelihood diminishes even further.

We don’t know how frank those New Jersey business owners were during their short time with the president. But I’ll bet that they were more forthcoming sitting among cartons of chips and cases of soda than they would have been in the Oval Office. And the more often a president comes in contact with the public, the more likely he is to run across people who will speak openly with him. Unfortunately, this happens less and less given the security demands of the modern day presidency. [As I type this, I am looking out my window at the grounds where Woodrow Wilson mingled with voters during his 1916 re-election campaign.]

This is why it is increasingly important that presidents (and other elected officials) make sure their circle of advisors includes at least one or two people who are free to speak their mind without fear of repercussion. As to Obama, Alter claims that there are “very few people” within the White House who “are willing to tell him hard truths.” So maybe the president is in need of a court jester, a.k.a. Tom Fool (after Thomas Skelton, one of the last persons to hold the official title of “licensed fool”).

Jesters were de rigueur for the nobility through the early Renaissance. Unlike the common conception of them as simply clowns, jesters were valued members of a monarch’s court. While their prime responsibility was to entertain, jesters were also respected as sounding boards on important issues of the day. Basically, the jester was given license to speak his mind and was frequently the only advisor the monarch could trust to give an honest evaluation of the situation.

Of course, the “licensed fool” has some modern-day analogies – Stephen Colbert comes to mind. But all of these contemporary jesters exist outside the inner sanctum of power. If I may take this idea in a more serious direction, it is important for any leader to have an honest sounding board. And even if a president is surrounded by strong nay-saying counselors, they are also removed from the everyday concerns of citizens by virtue of their position in the halls of power.

Certainly, public opinion polls have been used to fill the void – and the current president appears to use them more than his predecessors. But poll results are cold measures in many ways detached from the concerns of the real people that underlie the percentages.

The conundrum is how a president keeps tabs on the public mood in an authentic way. Maybe the president should consider establishing an official advisor or council of advisors, drawn from the heartland of this country, whose sole purpose is to tell it like it is.

Or perhaps it’s time to bring back the court jester. No world leader should be without one.

State Pension Reality Check

Cross-posted at PolitickerNJ.

Numbers matter. Poll results, budget deficits, health statistics. Attach a number to any issue and it becomes reality. But sometimes a reality check is in order.

When this year’s budget was first unveiled, the administration touted the closure of a nearly $11 billion structural budget gap. There was some debate over that claim because it involved spending that had not been appropriated for years. The front office switched gears later in the budget process and focused on the their reduction of state expenditures by $3 billion from the prior year – a widely accepted fact that is certainly worth crowing about.

The grander claim of an $11 billion deficit solution continued to surface, though, driven perhaps by the national media’s interest in New Jersey’s Republican governor. I believe such a claim is basically “untrue,” because it implies that the structural issues contributing to this gap have been solved. They haven’t.

Indeed, a recent analysis by the non-partisan – and well-regarded – Office of Legislative Services estimates that next year’s budget deficit could top $10 billion. They arrive at that conclusion by looking at the same “on-the-books” programs and obligations that were used to estimate the current year’s $11 billion gap. So, it was more than a little interesting when Governor Christie said the OLS numbers were “completely fake.”

State treasurer Andrew Sidamon-Eristoff clarified the administration position. He said the OLS figures were “wildly inflated” because they assume “that New Jersey is going to return to its spending habits of 2008 and 2009. Those spending commitments were frankly unsustainable and out of control.”

The treasurer added that all parties “need to come to terms with the fact that fiscal ’11, the budget plan that we just adopted, represents a new baseline for New Jersey.” Fair enough.

The problem is that OLS includes those commitments in its fiscal analysis because the programs are still on the books, a fact that the treasurer implicitly conceded in a later interview.

The OLS numbers are a reality check. Those statutory obligations still exist, which the governor was asked about in his first national Sunday morning television appearance on ABC’s This Week.. Specifically, host Jake Tapper asked Governor Christie whether he wiped these programs off the books via “executive fiat.”

Regarding the pensions, the governor said that he was “going to go after current employees” this fall. Ah – a new reality.

It is no secret that the pension obligation will continue to grow, even after a required $500 million contribution is included in next year’s budget. Furthermore, there is mounting evidence that the state will never be able to meet its retirement obligations for current employees. [Jason Method’s piece in the Asbury Park Press on this is a must read.]

The governor has signaled that he is going to tackle this head on before the next budget. The fight is not going to be easy, but win it and the OLS deficit estimates will almost certainly come down. Until then, the numbers are anything but “fake,” especially to the workers who expect to receive these benefits and to the generations of taxpayers who would have to foot that bill.

Adventures in Campaign Message Polling, part 1

Cross-posted at PolitickerNJ.

The political blogosphere recently took note of a poll in New Jersey’s 3rd Congressional district purporting to show first-term Democratic incumbent John Adler with a surprising 17 point lead over GOP challenger Jon Runyan. The poll was conducted for the Adler campaign by their own pollster, but even campaign pollsters have to produce reliable estimates if they want to stay in business. Regardless, this poll – or to be more accurate, the memo that described the poll results – raises some red flags.

First, only 3% of likely voters say they are undecided about their choice. Really? In June, when most voters probably cannot name either party’s nominee?. Second, the vote choice question was posed as a 3-way race, including Adler, Runyan and a third candidate running under the “NJ Tea Party” banner. This candidate – whose name recognition has to be near zero – received 12% of the vote in this match-up.

The poll memo reads more like a campaign fluff piece (e.g. “[Adler’s] record of independence and accountability has put him in an excellent position to win this race.”) than an insightful polling memo. Now, I’m not saying that the poll findings were fabricated. For one, the pollster, Greenberg Quinlan Rosner, is a well-known Democratic firm that is unlikely to jeopardize their reputation by generating poll numbers out of whole cloth. In fact, I’m inclined to believe the results were probably valid. The problem is that they were reported out of context.

Unless the Adler campaign has money to burn on conducting polls solely for the purpose of leaking them to the public, these results were from a longer poll that also tested all sorts of messages and strategies for the Adler campaign. The purpose of an early summer poll is to try out a variety of messages in order to identify the most effective ones for use in the campaign.

What we don’t know about this poll is at what point in the interview this three-way vote choice question was presented to respondents. I have my doubts that this was the first time in the interview that survey respondents were asked to name their vote choice. There were likely some questions about candidate characteristics that preceded this question.

So why does the order of the questions matter? Because questions asked later in a poll allow respondents to use information they heard during the course of the interview to inform their answers. At this point, the poll results no longer reflect the mind-set of typical voters because the poll respondents now have information – i.e. messages – that most voters don’t.

This is why message testing polls rarely get released to the public. Indeed, most reputable pollsters prefer it that way. Unfortunately, their concerns are occasionally overridden by a campaign manager who sees some strategic advantage in releasing the poll results.

One reason a campaign may release an internal poll is to demonstrate to potential donors that they have a viable shot at winning. Considering the healthy state of Adler’s campaign coffers, that’s clearly not a concern. So what advantage did the Adler campaign see in selectively releasing poll results?

First, we need to consider why the campaign even bothered to include an unknown, unfunded third party candidate in one of their vote choice questions. Especially since there will be three independents candidates on the ballot for this race in November. [Side note: There were four, but the state Democrats’ executive director, Robert Asaro-Angelo, successfully challenged Robert “Weedman” Forchion’s petition. This, of course, raises questions about Angelo’s contention that he never heard of the Tea Party candidate listed on that very same ballot.] [UPDATE: Rob Angelo contacted me re this statement. He admitted that he misspoke, since he obviously reviewed the names of all independent candidate filings in June.]

So, why did the Adler campaign only test the Tea Party candidate? Because it makes sense in the current political environment. While we can make a pretty strong guess as to the “Libertarian” candidate’s likely vote total in November, the impact of running under the Tea Party banner is a big question mark.

According to the Adler poll results, a Tea Party candidate may indeed peel off votes from the Republican nominee. But this is by no means a certainty. Why? To start, no one knows who the Tea Party candidate is.

And that gets us to why these numbers were released. Remember, these poll results represent one potential outcome in a context where Adler’s pollster had complete control over information presented to voters. In other words, the message testing effects measured in campaign polls do not always play out so neatly in the real world.

More importantly, a message will certainly not work if no one knows about it. And that is the case with the Tea Party candidate. So, what’s a good way to get a candidate’s name out? Show him exceeding expectations in a poll.

The message testing poll becomes the message! The Greenberg firm issued a memo to “Interested Parties” and sure enough, the story hits the internet, including The Hill, Chris Cillizza’s “The Fix” column in the Washington Post, the National Journal’s Hotline, and PolitickerNJ.

Interestingly, the poll memo was not released to newspapers, at least not to those in Adler’s district. Did the campaign think these media outlets wouldn’t be “interested?” Doubtful. The reason why the poll was released only to internet sites geared to the chattering classes was a strategic one. The intent was to let Tea Party-inclined voters “know” they have a viable option in New Jersey’s 3rd district Congressional race and to suggest to potential GOP donors that Runyan is a shaky investment. The internet is the best way to get that buzz spread with a less critical eye, especially with the burgeoning Tea Party community.

The chosen “interested parties” did their job and disseminated the campaign’s message, any caveats in their reports notwithstanding. Of course, this may backfire in the long run. Tea Party activists have been denouncing the candidate, with stories now focused on whether the Tea Party candidate is a plant. This poll is seen by some as part of a larger Democratic plot.

However, the question remains whether this poll – or more accurately the selective results in the campaign’s memo – should have been reported in the first place. When a campaign simply claims that their candidate is ahead by 17 points, no journalist in his right mind would report it. However, when a campaign has their pollster slap together a memo that purports to show a “51 to 34 percent” lead, suddenly the information is valid.

A good rule of thumb, no poll should be reported – in any venue – unless the pollster is willing to provide the entire set of questions and responses. Otherwise, it’s little more than propaganda, or worse. It’s a little too late for this poll. The Adler campaign achieved its intent – getting out a campaign message under the guise of hard fact.

Homebuyer Credit or Taxpayer Gift?

So what’s the deal with the New Jersey homebuyer tax credit? This program would give homebuyers in the state a tax credit worth up to $15,000 spread over three years. The bill (A1678) passed the legislature with overwhelming bi-partisan support and landed on the governor’s desk on June 10. Yet, Chris Christie has yet to sign it.

On New Jersey 101.5’s “Ask the Governor” program this week, Christie said he likes the idea, but will only sign it if he can figure out where the money is coming from. Proponents of the bill say it will spur construction and create jobs, which will increase revenues in both the realty transfer fund and income taxes, which will offset the cost of the program. A look at the numbers behind the program casts serious doubt that this will be the case.

The program sets aside $100 million from the property tax relief fund as a tax credit for people who purchase a home after the program goes into effect. Of this amount, $75 million is set aside for the purchase of newly constructed homes and $25 million for existing home sales.

Wow! $100 million sounds like it will go far, doesn’t it? Not really. The program would give these purchasers a credit of $15,000 or 5% of the sales price, whichever is lower. For the sake of argument, let’s say that the typical home price is $250,000. That would give the average homebuyer a credit of $12,500.

The $100 million in the kitty would provide tax credits to just 8,000 homebuyers. Hmm, sounds less impressive now. It’s a shame the legislature didn’t know this before they passed the bill, you say. Actually, they did (or at least the Senate did). The Office of Legislative Services provided a fiscal impact statement in late May that laid out just such a scenario – in fact, theirs was even more conservative.

But wait, there’s more. Let’s see how this would actually work if the program was in place. Remember the pot is divided 75/25 between new and existing construction. Using my generous estimate, that equates to 6,000 purchasers of new homes and 2,000 purchasers of existing homes who would qualify for the credit.

There were about 12,000 building permits issued for new homes in New Jersey last year. Even if some went unsold, at least 10,000 new homes were purchased last year. There is some debate on the impact that the now-expired federal tax credit had on those sales numbers. Let’s make a hypothetical assumption that one-third of those sales were driven by the tax credit and two-thirds would have happened anyway because people needed to move. In other words, we could expect that about 6,500 newly built homes will be sold in New Jersey this year without any tax credit.

Now, proponents say that the availability of a state tax credit will spur more buyers into the marketplace and thus increase demand. Let’s accept that premise and estimate that demand doubles and 6,000 homes are sold within the first six months of the program.

Well, what happens after the credit program is shut down? Do those buyers stay in the market and keep housing demand high? If we look at the experience of the federal credit program, the answer appears to be “No.”

The numbers are even more astounding if we look at how the credit will apply to sales of existing homes. According to state figures, more than 110,000 existing houses were resold last year. That translates to about 300 a day. This means the $25 million pot for existing home sales tax credits will be snapped up within a week of the program going into effect. One week!

Considering how lengthy the sales and mortgage process is, a homebuyer’s likelihood of closing on their purchase within the seven day window this credit program is available will be more a matter of luck than planning. “Mr. and Mrs. Jones, here are the keys to your house, and congratulations - you just won $15,000!”

This analysis raises a number of doubts about the economic impact – or ripple effect – of the New Jersey homebuyer tax credit. First of all, if the purpose is to spur economic activity in the form of construction jobs, then why isn’t it limited to new home sales only?

But even if the entire pot was dedicated to new home sales, it doesn’t seem to be large enough to keep demand high for enough time to ride out the current slump in the housing market. If I were a builder, I don’t think I would be hiring more workers on the dubious potential of this program.

And finally, why is the program so generous at $15,000 a pop? The federal program gave an $8,000 credit for first-time homebuyers and a $5,000 credit for current owners. These amounts appeared to be adequate to keep the housing market from dipping lower than it did.

Of course, none of this says anything about whether a tax credit of any sort is particularly smart economics. Some analysts claim that the New Jersey housing market is still overvalued and that prices need to come down further before buyers will return. Basically, they argue that the tax credit program has interfered with the open market and delayed a necessary price correction. Perhaps that’s why the eight Assembly members who voted against the bill happen to be among the most ideologically conservative in the legislature.

In any event, it’s difficult to see economic benefit in taking $100 million dollars in taxpayer money and giving it to 8,000 homebuyers for doing what they would probably have done anyway. Given New Jersey’s current fiscal crisis, maybe the state would be better off using the money to build a few hundred houses itself and sell them for a profit.

Are Nate Silver’s Pollster Ratings “Done Right”

This originally appeared as a guest column on Pollster.com.

The motto of Nate Silver’s website, www.fiverthirtyeight.com, is “Politics Done Right.” I’m not sure that his latest round of pollster ratings lives up to that moniker.

As most poll followers know, Nate shot to fame during the 2008 election, taking the statistical skills he developed to predict baseball outcomes and applying them to election forecasting. His approach was pretty accurate in that presidential race (although it’s worth noting that other poll aggregators were similarly accurate – see here and here).

Nate recently released a new set of pollster ratings that has raised some concerns among the polling community.

First, there are some questions about the accuracy of the underlying data he uses. Nate claims to have culled his results from 10 different sources, but he seems to not to have cross-checked those sources or searched original sources for verification.

I asked for Monmouth University’s poll data and found errors in the 17 poll entries he attributes to us – including six polls that were actually conducted by another pollster before we partnered with the New Jersey Gannett newspapers, one omitted poll that should have been included, two incorrect election results, and one incorrect candidate margin. [Nate emailed me that he will correct these errors in his update later this summer.]

Mark Blumenthal also noted errors and omissions in the data used to arrive at Research2000’s rating. I found evidence that suggest these errors may be fairly widespread.

In the case of prolific pollsters, like Research2000, these errors may not have a major impact on the ratings. But just one or two database errors could significantly affect the vast majority of pollsters with relatively limited track records – such as the 157 pollsters out of 262 pollsters on his list who have fewer than 5 polls to their credit.

Some observers have called on Nate to demonstrate transparency in his own methods by releasing that database. Nate has refused to do this (with a dubious rationale that the information may be proprietary) - but he does now have a process in place for pollsters to verify their own data. [If you do, make sure to check the accuracy of the actual election results as well.]

I’d be interested to see how many other pollsters find errors in their data. But the issue that has really generated buzz in our field is Nate’s claim that pollsters who either were members of the National Council on Public Polls or had committed to the American Association for Public Opinion Research (AAPOR) Transparency Initiative by June 1, 2010 exhibit superior polling performance. For these pollsters, he awards a very sizable “transparency bonus” in his latest ratings.

One of the obvious problems with his use of the bonus is that the June 1 cut-off is arbitrary. Those pollsters who signed onto the initiative by June 1, 2010 were either involved in the planning or happened to attend the AAPOR national conference in May. A general call to support the initiative did not go out until June 7 – the day after Nate’s ratings were published.

Thus, the theoretical claim regarding a transparency bonus is at least partially dependent on there also being a relationship between pollster accuracy and AAPOR conference attendance. Others have remarked on the apparent arbitrariness of this “transparency bonus” cutoff date Nate claims that regardless of how a pollster made it onto the list, there is statistical evidence these pollsters are simply better at election forecasting. I don’t quite see it.

His methodology statement includes a regression analysis of pollster ratings that is presented as evidence for using the bonus.

The problem is that even in this equation, the transparency score just misses most researcher’s threshold for being significant (p<.05). More to the point, his model – using dummy variables to identify “transparent” pollsters, partisan pollsters, and internet pollsters – is incomplete. The adjusted R-square is .03. In other words, 3% of total variance in pollster raw scores (i.e. error) is predicted by the model.

Interestingly of the three variables – transparency, partisan, and internet – only partisan polling shows a significant relationship. He decided to calculate different benchmarks that award transparent polls and penalize internet polls (even though the latter was based on only 4 cases and not statistically significant). And oddly, he does not treat partisan pollsters any differently than other pollsters, even though this was the only variable with a significant relationship to rawscore.

I decided to look at this another way, using a simple means analysis. The average error among all pollsters is +.54 (positive error is bad, negative is good). Among “transparent” pollsters it is -.63 (se=.23) and among other pollsters it is +.68 (se=.28).

But let’s isolate the more prolific pollsters, say the 63 organizations with at least 10 polls to their names who are included in Nate’s first chart. Among these pollsters, the 19 “transparent” ones have an average score of -.32 (se=.23) and the other 44 pollsters average +.03 (se=.17). The difference is not so stark now.

Firms with fewer than 10 polls to their credit have an average error score of -1.38 (se=.73) if they are “transparent” (all 8 of them) and a mean of +.83 (se=.28) if they are not. That’s a much larger difference.

I also ran some ANOVA tests for the effect of the transparency variable on pollster raw scores for various levels of polling output (e.g. pollsters with more than 10 polls, pollsters with only 1 or 2 polls, etc.). The F values for this test range from only 1.2 to 3.6, and none were significant at p<.05. In other words, there is more error variance within the two separate groups of transparent versus non-transparent pollsters than there is between the two groups.

I can only surmise that the barely significant relationship between the arbitrary transparency designation and polling accuracy is pointing to other more significant factors, including pollster output.

Consider this - 70% of “transparent” pollsters on Nate’s list are have 10 or more polls to their credit, whereas only 19% of the “non-transparent” ones do. In other words, Nate’s “bonus” is actually a sizable penalty levied against more prolific pollsters in the latter group. “Non-transparent pollsters happen to be affiliated with a large number of organizations with only a handful of polls to their name – i.e. pollsters who are prone to greater error.

For comparison, re-ran Nate’s PIE (Pollster Introduced Error) calculation using a level playing field for all 262 pollsters on the list. I set the error mean at +.50 (which is approximately the mean error among all pollsters).

Comparing the relative pollster ranking between the two lists produced some intriguing results. The vast majority of pollster ranks (175) did not change by more than 10 spots on the table. Another 67 had rank changes between 11 to 40 spots on the two lists; 11 shifted by 41 to 100 spots, and 9 pollsters gained more than 100 spots in the rankings because of the transparency bonus. Of this latter group, only 2 of the 9 had more than 15 polls recorded in the database.

Nate says that the main purpose of his project is not to rate pollsters’ past performance but to determine probable accuracy going forward. But one wonders if he needs to go this particular route to get there. Other aggregators use less elaborate methods – including straightforward mean scores – and seem to be just as accurate.

His methodology statement is about 4,800 words (with 18 footnotes). It reminds me of a lot of the techies I have worked with over the years – the kind of person who will make three left turns to go right.

This time I think Nate may have taken one left turn to many. We’ll know in November.

New Jersey Property Taxes Far and Away the No. 1 Pet Peeve

This column originally appeared as an Op-Ed in the Gannett New Jersey newspapers on June 13, including the Asbury Park Press, Courier News, Courier-Post, Daily Record, and Home News Tribune.

What do New Jerseyans answer when asked to name the most unfair tax they pay? If you said property taxes, you are correct. If you said something else, I’d like to be among the first to welcome you to our state.

In public opinion polls stretching back nearly 20 years, New Jersey’s property tax consistently ranks as the most detestable when stacked up against the federal income tax, state income tax, and state sales tax. In a September 2009 Monmouth University/Gannett New Jersey Poll, 59 percent of state residents chose property taxes as being the least fair. This is similar to the 61 percent who said the same in 2005, which was up slightly from the prior decade – 54 percent in 1994 and 47 percent in 1991.

The bottom line is that no other tax in New Jersey comes close to raising the public’s ire as much as the property tax. In fact, property taxes are the number one reason people give for wanting to move out of New Jersey.

In a poll we conducted earlier this year, 80 percent of homeowners told us that, if they had to choose, they would like to see their property tax bill reduced rather than their state income tax bill. And that even included a majority of those who currently pay more in state income taxes than they do in local property taxes!

To be fair, we in New Jersey are not alone in detesting our property taxes. An Elon University poll of residents in supposedly low-tax North Carolina last year found 49 percent who said that local property taxes in that state were unfair. This result was nominally higher than the 46 percent who rated North Carolina’s personal income tax as unfair. And this sentiment even extends overseas. A YouGov poll taken in Great Britain in 2007 found that 67 percent rated their local council tax – their property tax equivalent – as unfair, compared to 41 percent who said the same about the income tax.

At its core, there is something about how property taxes are levied that just irks people. While income taxes are based on wealth, property taxes do not take into account one’s ability to pay. And that just strikes people as unfair, regardless of how much their property tax bill is.

The big difference between New Jersey and other places seems to be the all-consuming nature of the issue. During last year’s campaign for governor, the number one issue Garden State voters wanted addressed was – you guessed it – property taxes. I doubt that is true in any other state.

It may be useful to compare New Jersey public opinion today to California in the late 1970s. Back then, a Field Poll found that 70 percent of Golden State residents said their state and local taxes were too high, with 60 percent laying most of the blame on property taxes. In 1978, California voters passed Proposition 13, which placed a 2 percent cap on property tax increases. Two years later, 70 percent of the state still said their taxes were too high, but this time they tabbed the state income tax as the primary offender.

The message from the California experience seems to be that you’ve got to either reduce spending on services or wind up shifting the burden somewhere else. And it’s interesting to note that even 30 years after Proposition 13 capped annual property tax increases at 2 percent, 29 percent of California residents still say their property taxes are too high.

This is certainly food for thought as Governor Christie and others push for a 2.5 percent property tax cap here in New Jersey.

New Jersey's Primary: What's it Mean for November?

So what does the outcome of New Jersey’s Congressional primaries mean? The Tea Party movement certainly made a statement, but there is a lot more to the message voters sent on Tuesday. Basically, voters of all stripes are frustrated, but that frustration was voiced in different ways by Republicans and Democrats.

On the GOP side - assuming Anna Little holds on in the 6th district - the Tea Party can claim only one clear winner. However, nearly all their candidates had stronger outings than are typical for challengers to the party organization’s anointed picks.

Typically, a House incumbent facing a primary challenge in New Jersey will garner a majority of 85% or better. This year, Rodney Frelinghuysen (R-11) and Frank LoBiondo (R-2) were held to about three-quarters of the vote, while Chris Smith (R-4) fell just shy of 70%, and freshman incumbent Leonard Lance (R-7) eked out a 56% majority against three challengers. Scott Garrett (R-5) was the only Republican incumbent to avoid a challenge.

In nearly every other Republican contest, the party-line candidate was held to 60% or less of the vote: 1st - Dale Glading 55%; 3rd – Jon Runyan 60%; 6th – Diane Gooch 50%, 12th – Scott Sipprelle 54%. Only Roland Straten, a sacrificial lamb in the 8th, exceeded 80%.

Clearly there is some disagreement within the Republican Party base about who should bear their standard. But, contrary to most media reports from Tuesday, these near-upsets were not due to low turnout.

More than 245,000 New Jersey voters cast ballots for Republican House candidates on June 8. To put that in perspective, votes cast in GOP House primaries between 2002 and 2008 ranged from 132,000 to 193,000. [The larger number was driven by a hotly contested 3-way race for the U.S. Senate nomination in 2002.] This was, by most measures, a very strong Republican turnout for a primary without a statewide office at stake. [See NJ House turnout trend tables below.]

The fact that 10 out of 13 GOP House primaries were contested (including 4 out of 5 incumbents) is just another indication that voters want to send a message. In a typical year, only one or two GOP Congressional primaries attract more than one candidate (although that number did jump to 7 in 2008). The increased turnout seems largely attributable to these challengers.

For example, Chris Smith typically garners 16,000 to 20,000 votes in an uncontested race. Faced with a challenger, he was only able to increase his total to just over 21,000, while Alan Bateman tallied nearly 10,000 votes.

And, while Leonard Lance increased his vote total from 10,000 in 2008 (when 7 Republicans were vying for an open seat) to 17,000 this year, his predecessor, Mike Ferguson, garnered similar numbers in his uncontested primary races.

On the Democratic side, the story is much different. Only 3 incumbents faced challengers, and 2 won with typically large 86% margins – Rob Andrews (D-1) and Albio Sires (D-13) – while freshman John Adler (D-3) was held to 75%. The remaining five Democratic officeholders ran unopposed.

But there still were a total of 5 contested Democratic House primaries this year, when 2 or 3 is the norm. And yet, Democrats could only manage to get 158,000 of their voters to the polls. Democratic turnout for House races in the prior four cycles ranged between 184,000 and 278,000 (the higher number coming in 2008).

This is exactly the kind of disappointing primary turnout that foreshadowed Jon Corzine’s defeat in last year’s race for governor. After the euphoria of 2008, Democratic voters seem forlorn.

One interesting example is in the 1st district. Between 2002 and 2006, Rob Andrews could count on at least 18,000 votes when he ran unopposed. With a challenger this year, the vote total – including Andrews and his challenger – was less than 16,000. Steve Rothman’s (D-9) 14,000 votes was also a few thousand shy of what he typically gets.

Other Democratic incumbents, Frank Pallone (D-6), Donald Payne (D-10), and Rush Holt (D-12), pulled in numbers similar to their prior races. Only Bill Pascrell (D-8) at 13,000 votes performed slightly better than in past contests. [One caveat: Without a statewide office at stake, uncontested House primary turnout can be driven by down-ballot races, which generate varying levels of interest by district]

So what does this all mean for November?

On one hand, the Democratic base shows very little enthusiasm. Advantage Republicans.

On the other hand, the Republican base appears angry and divided. If that rift can’t be healed and those with Tea Party affinities can’t be persuaded to support the GOP nominees, these voters might sit on their hands this fall. That would be good news for the Democrats.

The bottom line is that a lot of voters – for widely different reasons – are simply unhappy with the performance of their government. The question is: Do they voice their frustration at the ballot box, or do they throw up their hands and refuse to participate in a system they see as unresponsive?

That unpredictability could to lead to a wild fall campaign.

Ironically, though, it could just as easily lead to maintaining the status quo.

At least, that’s what New Jersey’s 13 Congressional incumbents hope will be the case.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

REPUBLICAN HOUSE PRIMARIES – BALLOTS CAST

CD	2010	2008	2006	2004	2002
1	9,274	7,139	nc	8,134	nc
2	24,418	18,051	14,440	13,377	25,335
3	27,799	25,719	16,228	16,010	24,884
4	31,377	16,818	16,109	14,350	19,667
5	29,053	19,914	27,507	21,265	36,080
6	13,399	6,706	5,309	5,613	6,505
7	30,577	25,577	16,423	13,262	20,244
8	7,027	5,859	3,982	4,415	6,437
9	9,378	5,797	6,169	205	7,336
10	1,000	nc	nc	nc	2,005
11	42,653	28,035	26,461	24,410	29,691
12	16,544	9,718	9,383	8,284	11,902
13	2,306	1,739	2,769	3,091	3,420
Total	244,805	171,072	144,780	132,416	193,506

DEMOCRAT HOUSE PRIMARIES – BALLOTS CAST

CD	2010	2008	2006	2004	2002
1	15,816	38,672	18,418	20,431	18,362
2	5,243	16,465	9,971	10,707	9,182
3	15,004	18,130	9,960	11,731	10,431
4	7,908	13,114	8,368	9,539	8,589
5	6,912	15,365	10,341	8,876	6,365
6	10,834	18,609	10,934	14,382	11,005
7	7,843	15,776	8,631	8,906	6,857
8	13,086	19,948	11,083	11,800	10,462
9	13,904	25,418	18,513	17,712	16,362
10	21,464	30,764	22,361	25,397	40,252
11	8,652	15,612	8,235	8,267	6,462
12	14,284	23,653	13,315	14,779	9,618
13	17,418	26,527	34,147	38,473	37,357
Total	158,368	278,053	184,277	201,000	191,304

        2010 numbers are unofficial election night vote counts.
        Contested races with more than one candidate on ballot are in red.
        “nc” = No candidate on the ballot.

More on New Jersey’s Image

Two days after the Gannett New Jersey newspapers ran a feature on New Jersey’s image (which included my analysis of state and national public opinion trends), there is a new poll from Eagleton-Rutgers updating some of those numbers. And they don’t look particularly good.

Wait a minute, you say! Didn’t today’s headlines proclaim that most New Jerseyans like the state? Yes, but the analysis overlooked the fact that these results are the lowest in three decades.

The poll reports that 52% of the public rate the state as either an “excellent” or “good” place to live. That may sound decent in isolation, but how does it stack up to prior years? Here are the numbers from Eagleton’s own poll archive:
            2010     52%
            2007     63%
            2004     68%
            2003     72%
            2001     76%
            2000     76%
            1999     76%
            1994     71%
            1990     59%
            1988     78%
            1980     68%

The current 52% result is clearly an all-time low in the history of these polls! In fact, the only other time positive opinion of the state dipped below 6-in-10 was during the Florio era toilet-paper tax revolt.

Eagleton also asked about state pride and found that 50% take “a lot” of pride in being a resident of the state. There are only three other trend points for this question. The current result is fairly comparable to the 51% recorded 2003 and 52% in 2001, but lower than the 59% in 1994. Moreover, the 24% who currently say they only have “a little” or “no” pride in being a New Jerseyan is slightly higher than the 15% recorded in 2003, 17% in 2001, and 13% in 1994.

Finally, Eagleton also asked a question about where folks prefer to live. Specifically, “If you could, would you move out of your neighborhood or continue to live where [you] are now?” If they want to leave their neighborhood, they are then asked: “Would you move to another part of your town, to another town in NJ, or to another state?”

Eagleton finds that 63% of New Jerseyans would like to stay in their same neighborhood, 14% would move somewhere else within the state, and 23% move out of New Jersey. The number who would move out of New Jersey stood at 19% in 2001 and 23% in 1995 – so there really isn’t any change there.

However, the number who want to stay in their neighborhood has increased from 54% in 2001 and 56% in 1995. That means the number who would like to move elsewhere within New Jersey – currently 14% – is down from 23% in 2001 and 20% in 1995.

That finding is kind of interesting. I’m not sure if true affinity for one’s neighborhood is on the rise or we’ve started to develop a bunker mentality. It’s certainly worth further exploration.

It’s also important to point out how these results compare to a recent Monmouth University/Gannett New Jersey Poll finding that half of the state’s residents want to leave New Jersey.

This just presents another opportunity to get on my soapbox about the framing context in question wording. The Monmouth poll question asks: “As things stand now, would you like to move out of New Jersey at some point or would you like to stay here for the rest of your life?”

The Eagleton question frames the issue in terms of preference to remain in one’s neighborhood. The option of moving out of state is presented as a follow-up only to those who want to leave their neighborhood. The Monmouth question, on the other hand, focuses on the state. Also, the Eagleton question suggests a proximate timeframe, whereas Monmouth specifies moving out during one’s lifetime. Both questions contribute to our understanding of residents’ attachment to the Garden State, but do so from different angles.

As I wrote a couple of days ago, trends are wonderful for providing context in polling. And Eagleton, which has been polling since 1971, is the largest repository of public opinion trends for the Garden State.

I’m not sure why these prior results were omitted from the Eagleton press release. But to me, the trend is the crux of this story.

- - - - - - - - - - - - - - - - - - - - - - -
Update: Eagleton Poll director David Redlawsk posted a response on his blog. He notes that another part of the context is the current economic environment. He is correct. The NJ rating question - much like the president's job approval - is as much a product of macro-trends as any item-specific evaluation. However, I'm not convinced of that as a rationale for excluding prior poll results, especially since reporting these numbers is a standard convention in our field. Context may make the analysis more complex, but it also makes it richer.

Do Other Americans like New Jersey for the Right Reasons? ‎

This column originally appeared as an Op-Ed in the Gannett New Jersey newspapers on April 25, including the Asbury Park Press, Courier News, Courier-Post, Daily Record, and Home News Tribune.

A little over 30 years ago, the nation was astounded to discover that most New Jerseyans actually liked living here.

This shockwave was set off by an Eagleton Poll report that 7-in-10 Garden State residents said their state was a good place to live. This finding was so extraordinary that the New York Times ran a banner headline. Apparently, it was difficult for people outside the state to imagine that anything in New Jersey could make it a desirable place to call home.

This national disbelief raises an important question: What do people living in other states think of us. A recent national survey conducted by the Public Mind Poll found that 47% of Americans have a favorable view of New Jersey, while only 18% have an unfavorable one. That doesn’t sound bad, but it’s not clear if they like us for the right reasons.

When ask what images New Jersey brings to mind, our beaches and coastline were the most commonly mentioned assets. So far, so good. But the second most frequent image evoked by our state is political corruption and organized crime. Umm, not so good.

The poll also found that people who had watched MTV’s “Jersey Shore” were even more likely to have a favorable view of our state than those who hadn’t seen the reality show about a group of self-styled “guidos” and “guidettes.”

Huh? One has to wonder whether other Americans have a “favorable” view of New Jersey simply because they enjoy the caricatures of our state.

Unfortunately, we New Jerseyans may be complicit in perpetuating that view. A 2002 Eagleton Poll found that 28 percent of Garden State residents actually found the HBO Mafia drama “The Sopranos” to be a source of pride for the state. This compared to 24 percent who were embarrassed by it.

Digging into the poll numbers further demonstrates the depth of our image problem. Back in 2003, the folks at Eagleton (which, at the time, included this writer) decided to take the same questions they asked about New Jersey and put them to a national sample.

Across most areas of the country, fully half of other Americans said their own state was an “excellent” place to live. Just 20 percent of New Jerseyans gave the same “excellent” rating to our state. (We can take some solace from the fact that the ratings for other Northeastern states were only slightly higher than our own. Outside the Northeast, though, the state ratings were much higher than for New Jersey.)

That 2003 poll also found that nearly 6-in-10 Americans took a lot of pride in being called a resident of their home state. Here in New Jersey, that number was somewhat lower at 5-in-10.

Overall, 72 percent of New Jerseyans rated their state as a decent place to call home, but when other Americans were asked to evaluate our state, only 38 percent felt that it would be a good place to live.

And so, we continue to endure the national bewilderment noted more than three decades ago. Why would anyone actually want to live in New Jersey?

Of course, the latest Monmouth University/Gannett New Jersey Poll shows that about half of us would, in fact, like to leave the state. But that has more to do with property taxes and the cost of living than any other aspect of Jersey life.

There’s a lot to be proud of in New Jersey. Food? The name “Garden State” was coined in the 19th century to reflect the wealth of our gastronomic offerings. And today, New Jersey offers the cuisine of nearly every world culture.

History? Forget Massachusetts, Virginia and Pennsylvania. During the Revolutionary War, George Washington probably slept more nights in New Jersey than in any of the other 12 states.

The light bulb, motion pictures, the transistor? All invented in Jersey

Being able to access two of the greatest metropolitan areas in the world, and some of the best beaches in the country, along with pristine mountains and lakes – and all in the same day? Only in New Jersey.

Ben Franklin once referred to our state as a keg tapped at both ends. Well, old Ben knew that you only tap a keg if there’s something worthwhile inside.

Now, if we can only get other Americans to discover all the outstanding things New Jersey has to offer.

But then they might want to move here. And we couldn’t have that, could we.

Interpreting the School Budget Vote

Yesterday, New Jersey voters did something they haven’t done in more than 30 years: defeated a majority of school district tax levies. [Note: I’m calling them “levies” here because that is more accurate. Voters don’t really have a say on the spending portion of the operational budgets of their local schools. They only get to vote on the amount in property taxes that the district proposes levying for the year.]

They also turned out in record numbers. The final statewide vote count hasn’t been compiled, but it is somewhere north of 20% of all registered voters. [Update: Turnout was more than 1.2 million voters. That's 24% as a percentage of all registered voters (26% if you exclude districts that don't put the levy up for a vote)! It was 13.5% in 2009.] That may not sound like much, but the previous high for school elections, going back to at least 1976, was 18.6%. 1976 was also the last time a majority of school levies failed. That year, 56% went down. This year, it looks like 59% have been tossed out by voters.

A Monmouth University/Gannett New Jersey Poll released last week found that 29% of registered voters – if they did vote – would support their local school levies, while 37% would oppose them. Based on a sampling of county returns, it looks like that 8 point margin may hold up in the final statewide vote. [Update: The gap closed to 48% for to 52% against -- a 4 point margin.]

There are some other interesting findings as well. Taking Middlesex County as just one example, compared to the April 2009 election, turnout in this one county was up by 65%. The number of “No” votes went up by 90%. But the number of “Yes” votes also went up, albeit by a lower 40%. In other words, turnout increased on both sides of the issue.

So what does this all mean?

Chris Christie and his supporters have claimed victory, saying that New Jersey voters sided with the governor in his battle with the state teacher’s union, the NJEA. However, the governor urged voters to defeat budgets in districts where the teachers made no concessions – and a good number of these actually passed. On the flip side, in the few districts where teachers actually agreed to wage freezes or other concessions – the districts one would expect to be rewarded if voters were out to show support for the governor – a good number (anywhere between 6 and 13 depending on what you count as a "concession") of the school budget levies failed.

The NJEA claims that the school vote was a repudiation of the governor’s draconian cuts in school aid which forced school boards to raise property taxes in order to maintain needed programs and services. Maybe, but polls also indicate that the public expected teachers to be willing to take pay freezes and pay for their benefits.

Local school boards say the vote was the product of a rush to make drastic cuts in a short time frame with few available tools to lessen the pain for both the educational system and the taxpayers. They may be partially right, but polls consistently show that voters believe there is a whole lot of waste in school spending to begin with.

So, here’s what we know about the New Jersey public:
1. They think the size of the cuts in state aid to local schools is unfair.
2. They think the teachers’ unions should be willing to come to the table and agree to a wage freeze and benefit contributions.
3. They don’t want educational programs cut.
4. They don’t want their property taxes raised.

All of these are reasons why Garden State voters voted yesterday. They are the reasons why more people than usual turned out to vote “No.” And they are also the reasons why more people than usual turned out to vote “Yes.”

Anyone who claims with certainty that any of these reasons is the main factor behind a majority of school levies going down yesterday is just blowing smoke.

However, one clarion message did emerge from yesterday’s vote. And the governor got it right when he said today, "[New Jerseyans have] had enough. They want real, fundamental change."

New Jersey voters get very few chances to actually make a statement at the ballot box (considering all our “safe” legislative districts). We’ve seen in the past year, though, that when given a real opportunity to vote for change, they’ll take it. April 20, 2010 was one of those times.

I don’t think yesterday’s vote can be seen as a whole-hearted endorsement of Governor Christie’s policy choices. But it seems clear that he has single-handedly raised the public’s awareness and interest in what’s going on in the state. And for that accomplishment alone, he deserves kudos.

In so doing, though, he has also raised expectations. The message from yesterday’s vote is that it’s time for everyone – governor, legislative leadership, unions, etc. – to drop the childish name-calling, come to the table, and get to work putting New Jersey back on the right track.

Conflicting Polls on the Teachers' Union? Not Really.

A trio of polls were released last week on Governor Chris Christie’s budget, particularly focusing on school aid cuts and state unions. According to at least one report, these polls were “seemingly at odds” with one another (also here). But if you look at what the three polls actually asked, they really tell separate pieces of a cohesive – but nuanced – story.

The Eagleton Poll (and here) found 57% of New Jerseyans feel that school aid should not be cut and 72% are opposed to “making it easier” to lay off teachers to solve local budget problems.

The Monmouth University/Gannett New Jersey Poll found 68% of the public see the cuts as being unfair to some groups (with teachers being among the top “victims”) and Governor Christie is seen as the more negative party in the NJEA dust-up, and ultimately more responsible for the impending teacher layoffs.

The Rasmussen Poll found 65% of likely voters favor having school employees (including teachers, administrators and other workers) take a one year wage freeze to help make up for the deficit in state funding.

I really don’t find anything too contradictory in those results. Public opinion is rarely black and white (as national polling about the health reform debate dramatically illustrates). The real difference in these three polls is that each chose to cover a different facet of the issue.

Both the Eagleton and Monmouth polls asked residents about their opinion of the governor’s proposed budget and how it will affect them personally.

Eagleton also asked quite a few questions about what areas of the budget should or should not be cut and what, if any, tax increases the public is willing to accept in order to avoid those cuts (none, apparently).

Monmouth’s survey included questions on impressions of Christie’s budget in comparison to Jon Corzine’s first budget (trends are a wonderful tool for providing context) and a focus on communication with the general public, including the NJEA battle and reaction to key terms used to describe the budget (e.g. “tough” and “fair”).

Rasmussen’s poll asked four questions, mainly focused on state worker concessions to deal with the budget crisis.

In terms of election polling, Rasmussen has a very good track record and, by my reckoning, had the most accurate final pre-election poll in last year’s gubernatorial race. [And admittedly, Monmouth, along with Zogby, YouGov, and Democracy Corp, came up with the wrong end of the stick in the final days of that campaign. Eagleton did not issue a final election poll.]

But election polling and policy polling are as different as meteorology and climatology. Both start off from the same theoretical premise, but election polling rises and falls on a pollster’s ability to selectively sample and predict the behavior of a subset of the total population. The heart of good policy polling is question wording. [That's why it’s really important to read the actual poll questions on controversial policy issues before reporting the results.]

Differences in question wording can be just a matter of perspective. Simply put, each pollster comes to the field from a different background, e.g. political scientist, policy or communication researcher, partisan strategist, and so on. These hats inform both the topics they choose to cover and how they word the questions.

The pollster’s agenda also informs what segments of the population are considered worth including in the measurement of public opinion on a policy issue. Do they survey the entire general public affected by the issue or just likely voters who can inflict electoral consequences? [Update: Coincidentally, pollster.com's Mark Blumenthal addresses this very issue in his National Journal column today!]

Regardless of the pollsters' varying agendas, these three polls taken together tell a much more complete story than any of the polls separately. You just have to do a little work to put the pieces together.

The notion that the public wants their taxes lowered but recoil from cutting valued services and programs is a phenomenon we’ve seen for years.

So why is it any more difficult to fathom that the public believes the state payroll is a drain on resources and wants the unions to make concessions, but at the same time feels that the governor’s “in your face” approach may not be the best way to go about it?

Another seeming contrast is the recent Public Mind poll that found New Jerseyans evenly divided – 35% favorable to 35% unfavorable – in their opinion of the NJEA and Rasmussen’s result that 66% of the public think the union is more interested in “protecting their members’ jobs” than in “the quality of education.”

Hmm, a union is seen as being more interested in helping their members than in helping their members’ employers. Unusual?

[Side note: Considering how many teachers will be laid off because the unions are unwilling to negotiate concessions, one has to wonder what these poll respondents were actually thinking when they said the union is protecting jobs. My guess is that the question simply taps the employee/employer dichotomy. ABC News pollster Gary Langer has a must-read post about the perils of taking question wording too literally.]

The bottom line is that there is nothing contradictory in these polls. Nothing in the Monmouth/Gannett poll contradicts Rasmussen's finding that most people see the state payroll as a burden on the budget and unionized workers should be willing to take a wage freeze. [In fact, much of our past polling supports that view.] By the same token, Rasmussen's poll does not contradict our finding that the public feels the school aid cut is unfair and the governor is more responsible for the negative tone of the debate.

As someone who has worked closely with media partners on providing public opinion data, I know that nuance in poll numbers – or indeed in any data driven reporting – is not the easiest thing to get across.

Here’s a friendly suggestion for my good friends in the fourth estate. If you are faced with seemingly conflicting polls – especially when a particular set of numbers is brought to your attention by a politician’s press secretary (!) – perhaps it would be worthwhile to contact an independent polling expert to provide context to what the different polls are saying.

Round 1 Goes to the NJEA

The latest Monmouth University/Gannett New Jersey Poll contains some mixed news for Governor Christie’s budget. The public may see it as tough, but not necessarily fair. Two-thirds of New Jerseyans say that the proposed cuts will disproportionately hurt some New Jerseyans, mainly the middle class, the poor, and … teachers.

When Governor Christie unveiled his budget, he made a specific effort to single out the NJEA as one of the primary opponents to fiscal reform. He basically dared them to oppose his cuts. The governor needed to identify an enemy in his fight to cut costs, and the NJEA was it.

The New Jersey Education Association is the state’s largest teachers’ union, with more than 200,000 members. The governor rightly assumed that the NJEA’s leadership would retrench, opposing any and all changes to their current contract provisions and benefits. However, Christie wrongly assumed that he could isolate the intransigent union leadership from their membership and win over public support.

Our latest poll finds that many state residents side with the union in this fight. Overall, more New Jerseyans blame the governor rather than the local unions or school boards for the inevitable teacher layoffs next year. This is despite the fact that only a handful of local unions agreed even to consider wage freezes as an alternative to job cuts.

The governor was warned this could happen. Here’s why.

More than one-in-five New Jersey households include either a teacher or another state worker (who have also been on the receiving end of the governor’s wrath). Another 1-in-4 Garden State households do not have public employees, but they do include kids who are taught by these teachers. That means nearly half of the New Jersey public is probably predisposed to sympathize with the teachers in this fight.

The poll results bear this out. When asked who is to blame for the impending workforce cuts at local schools, 57% of those in teacher or state worker households and 45% of parents blame Christie more than any other party in the process. Only among the other half of the New Jersey public – i.e. those who don’t have a teacher, state worker, or child in their home – does the governor (36%) share the blame with the teachers’ unions (33%).

The problem may be that the governor came out swinging just a little too hard when he decided to make an example of the NJEA. Back in January, columnist Charles Stiles described Christie as “licking his chops” at the prospect of taking on the teachers’ union. This may have made some New Jerseyans uncomfortable.

Teachers educate our children, for crying out loud. Sure, there are some long in the tooth, tenured ones who are just phoning it in. But most teachers are caring, concerned individuals who happen to have their own families to feed. Right?

I’m not saying that Christie should have avoided taking on the NJEA at all. Most objective observers would agree that the union has been less than cooperative with local school boards trying to keep job losses to a minimum. But the battle needs to be engaged more shrewdly.

The public’s predisposition to sympathize with teachers was always going to be the NJEA’s secret weapon. The union leadership knew how to play up that asset, using everything from TV and radio commercials to messages – some subliminal, some explicit – conveyed in the classroom.

The war is certainly not over, but the NJEA seems to have won the first battle in the court of public opinion.

As his term progresses, Chris Christie would do well to remember the words of Oscar Wilde: “A man can’t be too careful in the choice of his enemies.”

Public Support for Oil Drilling in the Mid-Atlantic

President Obama surprised more than a few people with his announcement yesterday to allow oil drilling in coastal waters. The northwestern boundary of the Atlantic Ocean exploration area is reportedly within 10 miles of Cape May, New Jersey. So it’s not surprising that Garden State political leaders have an opinion on this. What may be more surprising is that their opinion is nearly universally opposed – both Democrats and Republicans alike, including Governor Christie.

But how does the public feel about it? Especially among those who actually live along the coast? A study recently conducted by the Polling Institute and the Urban Coast Institute at Monmouth University provides some answers.

But, let’s take a look at the national picture first. A Pew Research Center poll in April 2009 found that 68% of Americans favor allowing more offshore drilling in U.S. waters, with just 27% opposed. This is similar to the 65% support registered by a Public Agenda poll that same month and up slightly from the 62% a national Quinnipiac poll found in August 2008. [Note: each poll used slightly different wording, but the findings are very similar.]

There are also a few state polls worth looking at. The Public Policy Institute of California has been polling on this issue for a number of years. When Californians were asked in February 2006 about allowing more drilling off their own state’s coast, only 31% were in support, compared to 64% who were opposed. That opinion shifted a little by July that same year, when support rose to 42% and opposition dropped to a bare majority of 51%. This was more in line with prior polls, where opposition was tabbed at 53% (2005), 50% (2004), and 54% (2003). PPIC’s next poll reading in July 2007 found that opinions remained stable at 41% for to 52% against.

But just one year later, that number flipped. PPIC’s July 2008 poll registered 51% majority support for drilling off the California coast, compared to 43% who opposed it. After at least five years of steady majority opposition, most Californians were in favor of drilling for oil off their own coast!

Recent polling in other coastal states reveals that most residents are in favor of drilling off their own shores. For example, a March 2009 Elon University poll of North Carolina residents found the 66% supported drilling off the Tar Heel State’s coast, compared to just 29% opposed.

In New Jersey, a Monmouth University/Gannett Poll from August 2008 found that 56% of Garden State residents supported drilling off their own coast, with 36% opposed. As a point of comparison, this result was not markedly different from the 59% of New Jerseyans who favored drilling off of Virginia’s coast and 51% who favored drilling in the Alaska wildlife refuge.

Basically, there seems to have been a shift in public opinion in favor of offshore drilling driven by the fuel price “bubble” two years ago. Moreover, there doesn’t seem to be any NIMBY effect at the state level. But is that still the case when you examine the views of those who live right on the coastline?

In 2007, Monmouth University conducted a unique survey of Mid-Atlantic coastal residents – i.e. people living in census tracts along the Atlantic Ocean from New York to Virginia. More than 8-in-10 of the study sample lived within one mile of the beach and more than 6-in-10 lived within a few blocks of the Atlantic’s waters. We repeated the study in April 2009, including a question about offshore drilling (see pages 19-20 of the report).

In the 2007 survey, we found that 33% of residents living in Mid-Atlantic coastal communities supported “drilling for oil or gas in the ocean off the Atlantic coast,” while 40% opposed it. Another 27% registered no opinion (which was an explicit choice in the survey question). By 2009, some of that “no opinion” had shifted to support – specifically, 46% of these coastal residents favored offshore drilling two years later, compared to 37% who were opposed and 16% with no opinion.

[It’s also worth noting that support for placing energy producing windmills off these residents’ coastlines also increased by about 10 points between 2007 and 2009 –57% support if the turbines are visible from the shore and 82% support if the wind farms are placed beyond the horizon.]

There are some interesting state-level differences in support for offshore drilling in the more recent poll. New York coastal residents (i.e. people who live along the south shore of Long Island) were the least supportive of offshore drilling – only 37% supported it compared to 44% opposed. Maryland residents (which were basically from the resort town of Ocean City) were most in favor, by a 65% to 22% margin. A majority of coastal residents in both Delaware (52% to 33%) and New Jersey (51% to 36%) also favored offshore drilling, while opinion was more divided in Virginia (42% to 34%).*

So, on the question of offshore drilling, it appears that the opinions of New Jersey’s political leaders are not entirely in line with the opinions of most Garden State residents, including those who live in the state’s coastal communities.

[*Note: Due to different sample sizes, the margin of sampling error for results from New York and New Jersey is +/-5.5%, for Virginia is +/-6.5%, and for Delaware and Maryland is +/-9.5%.]

Little Love for the “Representative from My District”

It’s a truism that Americans rate their own Congressional representatives more positively than the institution as a whole. An NBC News/Wall Street Journal Poll released last week bears that out.

Congress as a whole received an abysmal 17% job approval rating in the poll. Other polls released over the past few weeks showed between 14% and 26% of Americans approving of the Capitol Hill’s performance, with fully 70% to 80% registering disapproval.

Those numbers are low, but not entirely unprecedented. In the run-up to the 2008 election, Congressional approval ratings in the teens and low twenties were fairly common. And Congressional approval is rarely all that high any way. From the 1980s through the mid-1990s, positive opinion of the legislative branch hovered in the upper thirties to low forties. By the late 1990s, those numbers frequently topped the 50% mark, but went on a quick decline in the mid-2000s.

Unlike the other recent polls, though, the NBC/WSJ poll also asked respondents to “rate the congressman or woman from [their] district?” Those results were 45% approve to 41% disapprove. While that is certainly better than the 17% to 77% rating for Congress as a whole, the results still seemed low.

So I did a quick search. The “my member of Congress” question is not asked as frequently as the institutional job performance question, but the trend line indicates that the NBC/WSJ result is among the lowest recorded in national polls, since at least the mid-1970s (see chart below).

For most of the past 30 years, generic approval of the job done by “the representative from my district” lingered in the low 60s. We’ve seen that generally decline over the past five years – with one positive spike in 2007. The trend also indicates there was a great deal of volatility between 1991 and 1994, although “my member” approval rarely dipped below 50%.

Moreover, I found only two instances where the approve/disapprove gap was nearly as small as the 4 point margin in the recent NBC/WSJ poll: a Times-Mirror Poll from March 1992 (45% approve to 37% disapprove) and an ABC/Washington Post Poll from October 1994 (49% approve to 43% disapprove).

Both of these prior polls are instructive. The 1992 result was registered a few months before the election that saw Bill Clinton defeat incumbent George H.W. Bush during an economic downturn. Clinton’s victory was unusual because – unlike his modern predecessors – he had no coattails. His party actually lost a handful of seats in the House of Representatives that year.

And then we all know what happened in the next election cycle. The 1994 midterm saw the Republicans pick up a whopping 54 seats to gain control of the House.

So will 2010 be like 1994?

There are some who look at the party ratings – Republican approval ratings are worse than the Democrats – as evidence that this year won’t be like 1994. However, the Republicans were also polling lower than the Democrats in 1994. An ABC/Washington Post poll right before that election gave Congressional Democrats a net negative 14 point rating and their GOP counterparts an even worse net negative 26 point rating.

At the end of the day, voters elect individual members of Congress, not the party. A number of incumbents are consistently successful in overcoming their party’s negative ratings. But trouble is definitely brewing when the “my member” job approval number drops below 50%.

The question now is whether passage of the health care reform bill will change the outlook for this November. That depends on whether independent voters see the bill as good or bad for themselves. Up until now, their opinion has been mainly negative. But passage of landmark legislation can have a strange effect on voters.

Independents say they want bipartisan action, but at the end of the day, they’ll take any action over inaction. On Sunday, the Democrats seemed to demonstrate that they can actually get something major accomplished, and some voters may give them credit for that. Polls coming out in the next few days and week will tell us whether that is the case.

But the prospect that this deal could help Democrats is why the Senate reconciliation process will be a political football. It’s not so much that voters are concerned about the individual “fixes” in that legislation. The Republicans realize that undermining the Senate Democrats’ bargain with their House colleagues could douse any burgeoning positive opinion for the majority party generated by passage of the bill.

That may not be the best way to make policy, but it’s certainly a good way to gain political advantage.

The graph above is based mainly on CBS/New York Times and ABC/Washington Post poll results since the early 1990s. Prior results are adapted from a chart in “Public Support for Congress” by Kelly Patterson and David Magleby in Public Opinion Quarterly (v56, 1992). Note that the line does not represent every data point available. For ease of presentation in the chart, poll results were averaged for those months when multiple polls were issued.

The Hard Sell

This post originally appeared as a guest column for In The Lobby.

Governor Christie laid out his initial budget plan on Tuesday in his typical take-no-prisoners manner. The pain train is rolling down the tracks.

The soft sell has never been part of his repertoire. His message was clear and so was his intent. What is not clear is whether he will be able to sell it to the public.

Here’s why.

Yes, the public was anticipating big cuts to state services and local aid. In fact, a good number of residents were demanding it. But when you look at the bottom line, some may wonder whether they got what they expected ... and whether the pain has been fairly spread – particularly when it comes to education funding.

This is as much an accounting issue – and lack of public awareness about the budget process – as it is about any of the specific cuts.

To the extent that the public has been paying attention to the budget process over the past few years, they have been told that the last governor, Jon Corzine, reduced state spending from $34.6 billion to $29.0 billion by the time he left office this year.

Governor Christie says the state appropriations portion of his budget for the 2011 fiscal year will be $28.3 billion.

Huh? There are all these massive cuts, but state spending is going to be reduced by less than $1 billion!

But there’s more! Christie’s budget also includes another $1 billion in state spending that will be covered by anticipated federal stimulus funds. This brings his spending total to $29.3 billion.

Whoa? That’s more than Corzine spent, right? Not quite.

You see, Christie points out that Jon Corzine’s total budget spent $32.2 billion, if you include $2.3 billion in federal stimulus money that the former governor did not put on the books.

So, if we include the stimulus funds in both budgets, then Christie’s budget reduces state spending by $2.9 billion – or 9% – from the 2010 fiscal year. (This, of course, assumes there will be no supplemental appropriations made during the coming year – which would be a first!)

So, now that’s settled.
Hold on a second, you say. What about that gaping $10.7 billion dollar deficit we kept being told about? Doesn’t this budget fall short to the tune of $7.8 billion?

Well, yes and no. You see, the $10.7 billion figure was based on the assumption that the state would actually fulfill its legal obligations to fully fund the school funding formula, fully fund the property tax rebate program, and fully fund our existing pension obligations.

The truth of the matter is we haven’t done any of those things since the administration of George McClellan (I’m just guessing here). The so-called structural deficit is a bit of a canard. Just because the law says we have to fund these programs doesn’t mean that it happens.

As a side note, is anyone else scratching their heads over the continued underfunding of our current pension obligation in Christie’s “no gimmicks” budget? I know he’s pushing hard for pension reforms, but when does underfunding current annual obligations cease to be a “one shot” to balance the budget? Just curious.

That being said, it’s probably best for the governor if he drops the structural deficit issue from his rhetoric. The public is already sold on the problem and conflicting numbers only serve to confuse the situation. He’ll have enough trouble convincing the public that a $29 billion budget for FY11 actually represents a $3 billion decrease from FY10.

Most significantly, Governor Christie will be called on in the coming weeks to explain why cuts in school aid make up more than $800 million of the $3 billion reduction. This will be the line of attack used by his opponents, because it’s likely to present the best chance of undermining public support for his entire budget. The governor has to be ready for it.

And then, we’ll have to go through this entire process again next year, when – barring some economic miracle that starts filling the state coffers – Christie has to figure out how to make up for the $1 billion in federal stimulus money that will disappear.

Now that’s going to take a real hard sell.

Attitudes about gay marriage in New Jersey

I was asked to write this analysis for a Gannett New Jersey feature, including Asbury Park Press, Daily Record, Home News Tribune, and Courier News. It appeared in Sunday's papers.

What is public opinion on gay marriage? That question was certainly at the forefront of New Jersey’s recent legislative debate on the issue. But a better question may have been whether the public actually has an opinion on gay marriage.

Polls taken on same-sex unions over the past few years, both nationally and in New Jersey, have been fairly consistent in their findings. When asked about the issue, the gut-level public reaction is divided. However, as a policy issue, most people aren’t all that concerned one way or the other.

New Jersey polls over the past six years show that support for gay marriage has drifted between 41 and 50 percent. At the same time, opposition has hovered within a nearly identical 40 to 50 percent range. At different times, supporters have outnumbered opponents by a few points and at other times, it’s been the other way around. The bottom line is that neither side of the issue has been able to claim a clear majority here in the Garden State.

It’s worth noting that both state and national polls do show strong majority support for civil unions. Basically, the public is solidly behind extending marital rights to same-sex couples. It’s just that some are uneasy about using the term “marriage” to describe those unions.

Even so, many people don’t hold their views on gay marriage all that deeply. This was born out by a near universal blip in the polls last year caused by a bizarre event.

There was a flurry of polling about gay marriage last spring that suggested a big jump in support for same-sex marriage. Between the spring of 2006 and late April 2009, a Quinnipiac Poll of New Jersey voters measured an 8 point increase in gay marriage support. Nationally, the ABC News/Washington Post and CBS News/New York Times polls saw support jump by 13 and 15 points, respectively. Polling organizations that waited until May 2009 to poll on gay marriage, though, saw smaller increases (Fox News/Opinion Dynamics up 6 points) or none at all (USA Today/Gallup actually went down 2 points). [See here for national polls.]

So what momentous event occurred in April 2009 to cause this shift in opinion? It was the Miss Universe contest, when Miss California, Carrie Prejean, announced that she was personally opposed to gay marriage. The ensuing media storm was fast and furious, with the number of press articles on gay marriage doubling during those weeks. Based on the polls, public opinion initially reacted negatively to Ms. Prejean’s position, but quickly returned to its prior standing once the media attention died down.

At the end of the day, few people, especially New Jerseyans, hold deeply-rooted opinions on this issue because they do not feel that either allowing or banning same-sex marriage would affect their own lives. Every Monmouth University/Gannett New Jersey Poll conducted during last year’s campaign for governor found no more than one percent of voters reporting that gay marriage was a burning issue for the state. This was reinforced by a Rutgers-Eagleton Poll released after the election that found only two percent of the public who said that gay marriage was the most important issue facing New Jersey and just another 15 percent who said it was among a handful of issues they consider very important.

Tellingly, that same Rutgers poll asked people how they would react if the legislature had passed a gay marriage bill. The majority said they would simply live with it.

New Jersey arguably has a greater diversity of cultures and lifestyles than any other state in the union. Our state motto "Liberty and Prosperity" could probably use a rewrite. Certainly, our present fiscal predicament undermines the validity of the current slogan, but my concern is more about better reflecting the Jersey mindset.

My nominee for a new Garden State motto is "Live and Let Live." And public opinion on same-sex marriage is simply one case that illustrates that point.