Analyzing Polls: Interpretive Analysis-Assessing Quality
|
|
|
 |
||
"Are you a good poll or a bad poll?" (Apologies to The Wizard of Oz) Who conducted the survey? Reputable organizations won't risk conducting poor or unprofessional surveys . These organizations have experience in the process of conducting surveys—from sampling to questionnaire design to having trained interviewers to reporting results. Reputable survey organizations deposit their data with an archive like the Roper Center that will provide broad access for other researchers to learn from the data and scrutinize the results. Responsible polling groups will also adhere to professional standards and practices of disclosure, thereby making all of their data publicly available. However, this obviously does not apply to private polling organizations who are dealing with proprietary information.
If the sponsor of the poll has a vested interest in the outcome, this does not automatically translate to poor survey quality or reporting. After all, individuals and groups with special interests seek to know the truth on issues that are important to them. This is not to suggest that the motives of the sponsor be ignored; but rather, to alert researchers to the possible biases that may occur during the survey process. That way, they can be extra diligent in critiquing the methods used to gather and report results.
The description and size of the sample along with the exact interview dates should be reported in the methodology section of the report, press release, or other source of the data. The report should also include a description of the methods and modes of interviewing. Most US national public opinion polls today are conducted over the telephone, a method that has been very successful in securing a random sample or a probability sample. Other common acceptable modes of interviewing are "in person" surveys, mail surveys, and self-administered surveys (Exit polls). Sound research avoids sample surveys were the respondents are self-selected. Call-in polls to vote for your favorite song on the local radio station are unscientific, not random and only represent those who chose to make the call, got through, and voted-at least once! The latest and growing mode of interviewing is using the Internet. However, this technique is still being assessed because not everyone in the national adult population has online access and the validity of generalizing the results of a national sample has been questioned. Suffice to say, caution should be taken when analyzing any survey that requires respondents to have a particular mode of communication that is not generally available to everyone in the universe it purports to represent.
Principles of Disclosure The policing agency for the industry, the National Council on Public Polls has issued a statement regarding standards of disclosure for organizations who release survey results to the public. The organization also provides information on how to assess the quality of a poll. According to the NCPP, a report or press release stating that a survey organization is in compliance with the Principles of Disclosure of the NCPP will have reported the following information:
|
|
