Expert Review

Expert review is an early step in questionnaire design that usually occurs before cognitive interviewing or other testing techiques are implemented, but after a first draft of the questionnaire has been developed. There are three types of experts that should be involved in the review of a questionnaire at this stage:
  • Subject-matter Experts: Even if the questionnaire designer is an expert in the subject on which the questionnaire has been written, he/she should consult with other experts on whether the questions, as designed, will capture the desired information accurately. For example, a subject-matter expert may find that a closed-ended question does not contain enough or appropriate answer categories, or that an important aspect of the subject is not covered adequately by the questionnaire.

  • Cultural Experts: These types of experts are especially important for a survey that is to be administered to respondents from multiple cultures, or from a culture with which the questionnaire designer is not intimately familiar. Cultural experts can pick up on wording that may have a different meaning in one culture than the next, point out topics that are taboo or sensitive, and discuss access to certain subpopulations in the culture, such as women.

  • Questionnaire-design Experts: Even experts in questionnaire design consult with other experts in questionnaire design during the development of a survey instrument. Such experts are trained to look for vague wording, double-barreled questions, problems with skip patterns through the survey, misleading questions, and questions with multiple interpretations.
Once a questionnaire has been modified on the basis of the expert comments, it will be ready for the formal testing phase of questionnaire design.