Analyzing Polls: Interpretive Analysis-Questionnaire Design
|
|
|
 |
||
There are different styles of questions and as with any research, it's probably best to look at the results from multiple angles to better understand an issue. This section addresses some of the different types of questions and a few tips on what to watch for.
Those who work with data frequently value the ease with which questions having pre-coded response categories can be manipulated. The question, "Do you think things are going on the right track or wrong direction?" seems pretty straightforward-good vs. bad-basic responses and easy to report. Consider how enhanced an analysis of this question might be with a follow-up open-ended question, "Why do you say that?" The results to the first question provide the general mood; the second set of data substantiates that mood. Another important use of open-ended questions frequently found in surveys is the "MIP" question. Essentially, the question is: "What is the Most Important Problem facing the country today?" The results offer a plethora of responses. Most importantly, this item can provide a sense of how salient an issue is in the public mind. When conducting intense research on any topic, it's easy to miscalculate its importance among the public. The following types of questions should offer some insight into the process.
Could the order of the questions that were asked have impacted the results? It is possible that the process of the interview changed the respondent in some way, perhaps sensitizing or desensitizing participants to respond a particular way. Consider the "MIP" open-ended question, "What is the most important problem facing the country today? Let's say you're looking at a survey focusing on the condition of the US education system. Also included, are a variety of questions tapping public opinion on the quality of schools, state and federal funding, educational programming and testing, etc. In this case, the placement of the MIP is important. We expect to see different results, depending on whether the MIP question is asked first or twenty-first, after specific questions on education have been asked.
Intensity measures can be very useful in determining how firmly held a position is. These questions seek to determine not only how passionate the respondent feels about an issue, but also how ambivalent they may feel. See an example of question intensity measures.
Questions that lead the respondent to an intended answer are called loaded questions. There are blatant examples, like the one below that should be interpreted with caution. For the most part, professionally and scientifically constructed surveys will avoid using loaded questions. That's not to say, however, that they don't creep into even the best data collections on occasion. Questions that lead with long detailed descriptions of a position or policy should signal careful reading, making sure that a fair and balanced presentation of the positions has been made. See an example of a loaded question.
Phrasing questions isn't as easy as it appears-a good reason most public opinion survey organizations access (and archive with) the Roper Center collections before designing new surveys! Even the best survey organizations have had their share of confusing question wordings. Here are examples of double-barreled and confusing questions.
|
|
