Survey Design:  Developing Questions and Questionnaies

I.                    Introduction to survey research.

A.     Survey research methods collect self-reported data from individuals and, occasionally, small groups.  They are most often used to measure the characteristics, attitudes, and behaviors of individuals, but may also be used to collect data about small groups or larger organizations. By collecting data from large, representative samples of individuals, surveys can even be used to draw conclusions about institutions, cities, communities, or entire societies.

B.     The popularity of surveys.

C.     Types of surveys.

                                                1.      Self-administering questionnaires.

a.       Designed so that respondents can read and complete it on their own with little or no involvement from the researcher.

b.      Typically they are mailed to respondents, delivered by hand to individuals or groups, or made available over the internet.

                                                2.      Interviews.

a.       Requires personal contact between researcher and respondent.

b.      Interviews can be conducted by going to people's homes or places of business, by telephone, or even online.


II.                 General guidelines for constructing questions in survey research.

A.     Keep questions simple and clear.

                                                1.      Avoid long and wordy questions.

                                                2.      Avoid jargon and slang.

                                                3.      Avoid confusing and vague words.

                                                4.      Avoid negative items.

                                                5.      Avoid double-barreled questions.

B.     Make sure you ask questions your subjects can answer.

                                                1.      Avoid using words your subjects won't understand.

                                                2.      Avoid asking questions your subjects will not be able to answer or that don't apply to your subjects.

                                                3.      Avoid asking questions your subjects have no interest in.

                                                4.      Avoid false premises, hypothetical situations, and speculative questions.

C.     Avoid biased or leading items.

                                                1.      Leading questions.

                                                2.      Biased wording.

                                                3.      Social desirability.


III.               Types of questions.

A.     Closed-ended/fixed choice/forced choice questions.

                                                1.      Strengths.

a.       Predictable and reliable.

b.      Easily coded and quantified.

c.       Useful in composite measures.

d.      Easy to answer.

                                                2.      Weaknesses.

a.       May lack validity.

b.      Limits the kind of information you get.

c.       Restricts respondents' answers.

                                                3.      Answer categories must be exhaustive and mutually exclusive.

                                                4.      Types of closed ended questions.

a.       Yes/no questions.

b.      Likert-type items.

c.       Semantic differentials.

d.      Multiple-choice questions.

e.       Check lists.

f.        Rank order questions.

B.     Open-ended questions

                                                1.      Strengths.

a.       Increases variability.

b.      Increases validity.

                                                2.      Weaknesses.

a.       Less reliable.

b.      Difficult to code and quantify.

c.       Often left unanswered.

                                                3.      Following closed-ended questions with open-ended questions.

                                                4.      Coding open-ended questions.

                                                5.      As a general rule, limit open-ended questions.

C.     Contingency questions.

                                                1.      Simple contingency questions.

                                                2.      Complex contingency questions.

                                                3.      Problems with contingency questions.

D.     Matrix questions.

                                                1.      Types of matrix questions.

                                                2.      Matrix questions and composite measures.

                                                3.      Response set.

                                                4.      The benefits of matrix questions.


IV.              Developing a self-administering questionnaire.

A.     Questionnaires should look attractive, professional, and interesting.

                                                1.      You want respondents to be interested enough to complete the questionnaire.

                                                2.      You want them to take it seriously.

B.     Make sure it is clear and legible.

                                                1.      Make sure everything is spelled right and your grammar is correct.

                                                2.      Don't crowd things together.  Make it easy to read, but don't waste space.  You don't want it to appear to long.

                                                3.      It should be typed on a quality typewriter or printed on a quality printer.

                                                4.      All copies should be good quality.

C.     Question order is important.

                                                1.      You want your questionnaire to be logical, connected, and consistent.  It should "make sense" to those completing it.

a.       Put related questions together.

b.      Use logical groupings with headings.

                                                2.      Try to put interesting questions first, but if some questions are particularly troubling or threatening, you should put them in the middle.

                                                3.      Generally put demographic questions near the end.

                                                4.      Question order, interpretive context, and validity.

D.     Give clear instructions.

                                                1.      General and specific instructions.

a.       The cover letter/instruction page.

b.      Instructions in the body of the questionnaire.

                                                2.      Make sure you tell respondents how and where to answer questions.

a.       Make it as easy as possible.

b.      Be as consistent as possible.

c.       Using a separate answer sheet.

                                                3.      Pay careful attention to changes in the type of questions or the way answers should be marked.

                                                4.      Pay careful attention to contingency and matrix questions.

                                                5.      Put instructions for returning the questionnaire on the instruction page and at the end of the questionnaire.