Books Columns Blog forums Recent Events Crusades Links About Arianna

Arianna Online contains Arianna Huffington's columns and blog posts up until early 2005. This site is an archive and is not actively maintained. For frequent updates from Arianna, please visit The Huffington Post.

Columns Recent Columns

Investigating The Pollsters

October 12, 1998 [ Printer-friendly version ]

When the history books are written, the Clinton crisis will be the first political crisis to be so entirely driven and shaped by polls. It was, according to Dick Morris' grand jury testimony, a poll that he secretly conducted for the president when the Lewinsky scandal broke that set him on his 8-month-long course of deceiving the public. "You can't tell them about it, they'll kill you," Morris told the president. "They're just not ready for it." And the man who has lived by polls throughout his political career concluded, "Well, we just have to win, then."

So the few hundred people who answered Morris' poll determined a critical presidential decision. And now, nine months later, the president's high approval ratings remain his only protection. If the polls are going to be the instrument by which we will judge the fate of this president, it becomes all the more important to answer the key question: Who is talking to pollsters and who isn't?

In the 20 months before Richard Nixon resigned in August 1974, 128 polls were conducted asking people whether the president should leave office. In the nine months of the Lewinsky scandal, 325 polls asked that question. It's no wonder that the mushrooming number of opinion polls coupled with the outrageous growth of telemarketing calls have led to a soaring refuse-to-answer rate among people polled.

This is not good news for pollsters. The key to polling's accuracy is the principle of "equal probability of selection." But if larger and larger numbers among those randomly selected refuse to participate, this principle no longer applies.

It turns out that polling companies will talk about anything except the response and refusal rates of their last poll. Here's a sampling of a nonscientific poll of pollsters that my office conducted between Oct. 1 and Oct. 9, and that illustrated the nonscientific nature of polling.

Ours was a short poll: Can you please give us the response and refusal rates for your most recent national poll? ABC News pollster Jeff Alderman's first response was to say that he didn't understand the question. When it was repeated to him, with minor refinements, he growled: "That's proprietary information. ... I've got another call. Goodbye." In polling lingo, that was a refusal -- but a very revealing one. After all, we were not asking if the pollster wanted to change telephone services or presidents. And we were not calling at home during dinner time.

Tom Riehle of Peter Hart Associates also used the "proprietary information" defense. He called their methods "our secret recipe" and explained usefully: "That's not your business." Our little poll was batting 0 for 2 a 100 percent refusal rate. CBS' Kathy Frankovic was reluctant to release CBS response and refusal data without knowing the information her competitors were giving out. She added that it was a complicated issue. But then hiding behind complexity has been a staple of the polling profession.

Mike Kagay of the New York Times, Frankovic's partner in the CBS/New York Times polls, did release a response rate for an actual poll, though not the most recent one: 43 percent for the Sept. 12-15 poll.

At Gallup, senior methodologist Rajesh Srinivasan promised to fax us response rate data right away. And indeed, we did receive reams of data right away -- on everything except response rates.

A representative for Roper-Starch-Worldwide, who did not want his name used, explained that "that information is not available. Caddell, who conducted the first major presidential polls for the Carter campaign, is now appalled by the monster he helped unleash. "The dodging of such basic questions is alarming. When the polling industry is talking to itself, they express their worries about the progressive decline in response rates. But when they talk to the public, they clam up. It's ludicrous to suggest that response and refusal rates are any more proprietary than the size of the sample or the date of the interviews."

It's time to ask polling companies to make their response rates public for every poll. And if they refuse, perhaps it's time for the media to stop just quoting and start investigating the polling industry to get to the truth behind all the smoke it's been blowing.

[ Printer-friendly version ]


Search Arianna's weekly columns

Discuss this column in the Arianna Online Forums

Archived Columns

2005 Archived Columns
2004 Archived Columns
2003 Archived Columns
2002 Archived Columns
2001 Archived Columns
2000 Archived Columns
1999 Archived Columns
1998 Archived Columns
1997 Archived Columns
1996 Archived Columns

© 2004 Christabella, Inc. All rights reserved. Web site designed by Ottenhoff Consulting.