Before you begin amending or designing your questionnaire, and throughout the design process, it is important to give careful consideration to best practice. Below, we address FAQs in line with the latest scholarship on survey design and the psychology of survey response in order to maximise the reliability and validity of scales. For more details about best practices in questionnaire design, please see the Best practice in questionnaire design page. 

What is a scale?

A scale is a group of items that capture different aspects of the same theme (also known as a construct). The use of multiple items, each assessing a different aspect of the construct, enhances construct validity, the degree to which the scale measures what it sets out to measure (Eisinga, te Grotenhuis, & Pelzer, 2013; Emons, Sijtsma, & Meijer, 2007). If you are interested in learning how a scale is developed, click here [link].

When should I adapt a scale?

  • I want to compare my data to that of another study, but that other study used a scale that doesn't follow best practice. You will produce more reliable data if you can adapt that scale to more closely align with best practice. If this is not possible, we recommend that you make yourself aware of the limitations of that scale before using it for your study.
  • The existing scale doesn't cover everything I am interested in. If you want to create a new item, continue reading through these FAQs and consult the best practice page of this toolkit.
  • My intended population is diverse or differs from the population for which the scale was originally intended. If you think that the wording or content of items of the scale might not be accessible or relevant to all of your participants, it might be a good idea to amend the scale according to best practices.

I want to word the items as agree-disagree statements

This is generally inadvisable. Agree-disagree statements can be taxing on the respondent, therefore reducing respondent effort and increasing error. It is best to word the items as questions.

For example you can turn the statement, “I am confident that I will achieve most of the goals I have set for myself” into the question “How confident are you that you will be able to achieve most of the goals you have set for yourself?”

Can I use different labels in the response options?

It is advisable to use verbal labels that tie back to the question you are asking in your item. This grounds the responses in exactly what you are asking about (e.g. confidence) and helps to focus the attention of the respondent on the question you are asking.

For example:

How confident are you that you will be able to achieve most of the goals you have set for yourself?

confidence on a scale of 1-5

I want to gain more information. Can I combine two points in one question?

Add another, separate item to address the second point. Do not try and capture two points in one item. This may lead to respondents only answering one part of the item and complicates analysis.

For example, instead of "How confident are you that you will successfully overcome any challenges and still perform well when things get tough?", split this into two items:

  1. How confident are you that you will successfully overcome any challenges?
  2. When things are tough, how confident are you that you can still perform well?

Can I ask questions in a negative way? (e.g. can I ask about failure, or feeling left out?)

Negative words are more difficult to process cognitively, leading to increased response times and respondent error. Instead, keep the items phrased with positive language so that you can maximise respondent effort and understanding exactly what you are asking.

For example, instead of asking instead of asking “How confident are you that you will not accomplish/fail at difficult tasks?”, ask “How confident are you that you will accomplish difficult tasks?”

I want to change the number of response options

Research recommends 4-7 options. This is because when given many options, respondents tend to cluster their responses in 4 options. More than 7 options tends to overwhelm respondents and they tend to end up choosing between 4 anyway. We recommend the 5 response options that accompany the items in each of the questionnaires you will find for download on this page.

Some have used a ‘sliding’ or ‘0-100 point’ scale (Bandura, 2006). While this technically allows more fine-grained statistical analysis, issues to consider are:

  • Whether you have sufficient responses to make more finely-granulated variations meaningful – this may not be the case if you are only administering the survey to a class or small cohort;
  • Whether you have sufficient grounds for believing that the more finely-granulated variations you may capture are genuinely caused by the intervention or change being evaluated – this may be challenging if you are evaluating things which are complex and have multiple causes, such as one’s self-efficacy, academic attainment, or ‘sense of belonging’.  

I want to remove some of the items from a scale that I am planning on using

The way that scales are designed is such that each item addresses a piece of the construct that is to be measured. We recommend using all of the items as they all work together to help us make sense of the topic of interest.