What Does Cheap Mean to You?

How’s your survey research IQ? Let’s try one simple hypothetical question. Suppose there is a state representative district in a southern state, with a incumbent Democrat, a Republican challenger with no significant name identification, and a 20 point advantage for Democrats in party registration. Black voters represent 29% of those registered.

What do you think the ballot gap is today, before the contest really starts? According to a published report, Survey USA says it is Republican by 8%. And, mind you, that survey reports a sample of 58% women and 42% men and only 18% undecided.

We conducted a live call study in the same district with the same number of interviews a week later and came up with an even contest. We were within the margin of error on blacks, partisan registrants and gender. There were gross differences within the cross tabulations between the two methodologies. For instance, there was a 21% difference in how men were voting. Which one do you want to base your campaign strategy on?

The differences between live interviews and robo calls is pronounced, various comparison ‘studies’ of the two notwithstanding. We see these differences almost daily. The same robo-call firm, two to three days out from a mayoral Primary election in Buffalo, New York was 18% off of the actual election results in a Mayor’s race. Another firm [PPP] had the losing candidate in a recent NY special congressional election up 17% three or four days out. He lost by 2%.

Of course, a robo-survey or IVR is cheap compared to a live interview. So you make the decision: What does cheap mean to you?