Informed Opinion on Education Reform Poll?

Cross-posted at PolitickerNJ

Last month, Monmouth University and NJ Press Media released a poll on education reforms proposed by the Christie administration.  It has produced a whirlwind of blogosphere commentary from a few folks who took exception to the poll’s results.

The poll found broad, general support for the governor’s proposals, but with a few caveats.  When we asked questions pertaining to public awareness, we found a widespread lack of knowledge about these policies (especially with regard to charter schools).  We also found some concerns about implementation:  performance-based pay is a good idea, but using the current standardized tests as the metric on which to base that may be unfair.  And finally, we found that one of the arguments used by reform proponents – that it would close the achievement gap – does not necessarily hold water with the general public.

I initially chose to let the poll stand for itself.  Polling results frequently draw criticism when the results undermine a particular strategic perspective.  For example, Governor Christie was not happy with a poll we conducted early in his term that showed New Jerseyans expressing skepticism about his ability to bring about change (which was less about Christie and more about their jaded view of Trenton).  And individual election polls have been criticized at times by candidate’s campaigns – sometimes from both parties in the same cycle.  It’s understood.  Negative polling results can impact campaign contributions and undermine the storyline you are trying to put forward.

In most cases, the critics will question the poll’s methodology, say they see it differently, and move on.  That’s fair.  It’s all part of being the messenger about where the public stands on important issues of the day, a role I take very seriously. 

That’s usually the end to it.  Rarely does a critic try to misrepresent the poll or how it was conducted.  In fact, that has only happened to me twice.  This education poll is one of those times.  Unfortunately, I feel I must now respond directly to those criticisms.

To start, most of the criticism has come from people without expertise in the field of survey research.  Some has, which I will treat more seriously.  But it’s important to note that all of these critics, including some who are academic researchers, have taken very public normative positions on education policy.  Normative is one of those great social science words.  It simply means they already have a clear opinion about how things ought to be.  When normative values get applied in a research setting, they lead to bias.

The Monmouth University Polling Institute, on the other hand, has a record of measuring public opinion “as it stands” without bias.  For example, one of the charges levied against this poll centered on the question about tenure.  The criticism is that we used a colloquial definition of tenure rather than a legal one.

Well, that’s the point!  If you are trying to measure extant public opinion you need to use colloquial language.  This is especially important when it is not clear how well the public already understands the issue.  In these situations, most pollsters will look to see how other pollsters have handled it.

Our own search turned up a few poll questions on tenure, including one from the well-respected national Phi Delta Kappa survey conducted each year by the Gallup organization.  Their question defined tenure for public school teachers as “after a two- or three-year period, they receive what amounts to a lifetime contract.”  A Time magazine poll also used the word “lifetime” to describe tenure.

Based on my experience, I felt that the word lifetime could be a bit loaded in this context.  Our team spent a great deal of effort searching for something that reflected a more common definition of tenure, such as this entry in the Oxford English Dictionary: “Of an official position, usu. one in a university or school: carrying a guarantee of permanent employment until retirement.”

I decided to word our poll question as: “After working in a New Jersey public school for three years, a teacher is either given tenure or let go.  A teacher who gets tenure after this trial period is basically given a permanent job unless they engage in serious misconduct.”

I think the improved fairness of my question was borne out by the results.  In our question, 42% approved of tenure compared to 26% to 28% in the polls that defined tenure as a “lifetime” appointment.

Critics also took issue with way we described how a teacher can lose tenure since the question didn’t use statutory language regarding dismissal, i.e. for “inefficiency, incapacity, or conduct unbecoming a teaching staff member or other just cause.”  Again, the poll’s intent is not to measure the public’s opinion on the theoretical concept of tenure, but what they think of it in practice.  And considering the data available on tenure dismissal in the state (see: PolitifactNJ), only a handful of teachers have ever been dismissed for “inefficiency” – far fewer then are probably dismissed for this reason in any other profession (a good empirical question in itself).

As such, I stand by the question wording as an accurate measurement of public opinion on current tenure practices, to the extent the public is aware of them.  Even still, I am confident that using the legal language to describe dismissal conditions would have had little to no effect on the end result.

Our poll included a follow-up question, asking people if they would support a change to “limited tenure” which requires periodic evaluation and potential loss of tenure.  Even though this is technically not “tenure” by the dictionary definition, it is a common term used in public discussions of this proposal.

There is widespread support – 77% in fact – for changes to the tenure system that would make it easier to dismiss underperforming teachers.  Here are some interesting facts about that statistic.  These changes to tenure are supported by a whopping 71% of teacher households and 73% of those who actually approve of the current tenure system.

In our press release I wrote, “It appears that New Jerseyans want some type of job protection for teachers, but broadly support modification to the current system.”  I don’t know how you argue with that considering that teachers themselves support these changes.

Critics of this poll have focused on minor wording issues without considering the larger context within which this opinion is formed.  During an extended economic downturn with persistently high unemployment, 4-in-10 New Jerseyans feel that teachers should benefit from an extraordinary level of job protection – and I use extraordinary in the sense that this is something that no other profession enjoys.  And fully 3-in-4 New Jerseyans feel that teachers should have at least better job protection than most other workers.  That should be somewhat surprising, and heartening to these critics, given the current economic climate.

There are many ways to ask about tenure, and I strove to provide a definition that was fairer than other polls I have seen.  I am open to discussing other ways to approach this issue.  And if this was the nature and tone of all the critiques, I would have welcomed the debate.

Unfortunately, the poll was also subject to a number of other attacks that were ill-informed and downright malicious.  Since those attacks have gone unabated, I feel it is important to respond on behalf of the reputation Monmouth University’s Polling Institute has earned in New Jersey.

In some cases, critics oddly misinterpret questions that actually support their normative view.  For example, standardized tests would be a major component in determining tenure and merit pay under current proposals.  One critic saw our poll results as saying “people think standardized tests are reasonably accurate at measuring student abilities.”  The results show quite the opposite, which actually bolsters the argument against merit pay!  Just 38% give a positive response of excellent or good to the accuracy of the tests, compared to 59% who give a negative response of only fair or poor.  [By the way, the “only fair” or “just fair” construction is textbook polling procedure to delineate between two positive responses and two negative responses in a balanced response set].

Furthermore, the critic takes issue with the same question as it relates to how these tests reflect teacher competence.  He writes: “Implied within that question is that student achievement and good teaching are a 1 to 1 ratio.”  Well, I’m not sure what he was reading, but the question we posed pretty clearly asks if people think there is a direct correlation between student test scores and teacher ability – and a clear majority (62%) do not!  So, I’m left scratching my head at the charge.

Critics have also charged that the headline of our press release was misleading.  It stated that the public “supports” proposed education reforms.  The data show that the public does support these ideas.  And anyone who has followed opinion in New Jersey knows that the public has become increasingly supportive of all measures that promote greater accountability and choice.  It would certainly have been misleading if we wrote that the public “demands” or “calls for” these reforms.  Instead, we accurately reflected that the public expresses “support” for these proposals as they are generally understood by the public.  No more, no less.

The real problem is when critics lower themselves to base accusations that we conducted a “push poll,” which shows a clear misunderstanding of that term.  Or try to plant rumors with the media that nefarious forces were behind the poll questions.  That’s where the criticism steps over the line.

As I mentioned before, this is one of only two times that displeasure with a poll I conducted reached a level where critics actively tried to misrepresent the poll.  The other time in question, the criticism came from members of the Tea Party.  This was in response to a poll that showed their candidate not doing as well as they believed she was.  That poll turned out to be right on the mark, by the way.

The criticism aimed at this poll is more disappointing because these advocates are doing so on behalf of our teachers.  I know they don’t represent all teachers, including the NJEA members who teach my own child.  However, because their actions reflect on the teaching profession, one hopes that they would engage in a more productive dialogue regarding public opinion on these reforms, and indeed on the reform proposals themselves.

As the poll results indicate, public opinion on education policy is not always well informed and at times is misinformed.  But it is the public’s present opinion on this issue; the opinion that policymakers listen to.

So here’s my advice to critics who disagree with the poll results.  Spend more time working to change public opinion rather than disparage the poll that measures it.