Before you begin designing your questionnaire, and throughout the design process, it is important to give careful consideration to the following issues outlined by Cohen, Manion, and Morrison (2018) and Gehlbach and Brinkworth (2011):

Things to consider

  • Clearly define the concept you will measure, known as a construct (e.g. student self-efficacy, or student sense of belonging) – this will focus your questionnaire.
  • Intended population/sample – If you are administering the questionnaire to your students to evaluate an aspect of your module, you will want to word the items in a way that is accessible to all learners. You will also want to consider when and how to administer the questionnaire (e.g. on paper or electronically, after a class meeting or by email, etc.).
  • Intended method of data analysis – to ensure that questions are framed appropriately, you should consider how you plan to analyse the data you collect from the questionnaire.
  • Type of questionnaire – of the different types of questionnaire (e.g. structured/closed, semi-structured or “unstructured”), what is most appropriate to help you evaluate the specific topic or aspect of your module?

Items

  • Look to the literature to determine that a new questionnaire scale is needed to evaluate a certain topic. A scale is a group of items that capture different aspects of the same theme (also known as a construct). The use of multiple items, each assessing a different aspect of the construct, enhances construct validity, the degree to which the scale measures what it sets out to measure (Eisinga, te Grotenhuis, & Pelzer, 2013; Emons, Sijtsma, & Meijer, 2007). Learn how a scale is developed.
  • Item/response types – there are many different question/response types (e.g. dichotomous questions, multiple choice, Likert/rating scales, constant sum, rank ordering, open ended), and we will share some on this page. For example, if you are looking at sense of belonging, you may want to include both structured questionnaires and an unstructured, open-ended questionnaire so students may give some more information and explanation of their sense of belonging.
  • Wording of items – just as you will use accessible language in the wording of your items, you should ensure that the items are worded clearly and do not risk leading the students to select certain responses.

Final checks

  • Make sure you follow best practices – consult this checklist by Panorama Education to ensure you have followed best practices in questionnaire design.
  • Pilot and revise questionnaire – do your best to make time for a pilot test of your questionnaire. This will begin to establish validity for your scale and include analyses that can cull problematic items help you to revise the questionnaire if necessary. For example, if items are not clear to students, you may clarify and revise them.

References

Cohen, L., Manion, K., & Morrison, K. (2018). Questionnaires. In L. Cohen, L. Manion, & K. Morrison (Eds.), Research Methods in Education (8th ed.). Abingdon, England: Routledge.

Eisinga, R., te Grotenhuis, M., & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? International Journal of Public Health, 58(4), 637-642.

Emons, W. H., Sijtsma, K., & Meijer, R. R. (2007). On the consistency of individual classification using short scales. Psychological Methods, 12, 105-120.

Gehlbach, H., & Brinkworth, M. E. (2011). Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15(4), 380-387. Retrieved from https://dash.harvard.edu/bitstream/handle/1/8138346/Gehlbach%20-%20Measure%20twice%208-31-11.pdf?sequence=1&isAllowed=y