How to write a survey questionnaire

The questionnaire is a researcher’s most important tool! You need to ask the right questions, in the right way to get the insight you need. This guide explains what makes a good survey, be it online or telephone, and what pitfalls to avoid.

Starting the questionnaire

Before starting to write your survey, you should revisit the original project objectives to keep your line of questioning on-track.

It’s a good idea to start with a ‘wish list’ – noting down all the questions that you and your stakeholders would like to include. It’s helpful to have everything in contention written down so you can prioritise which are the most important questions. It’s also helpful to start grouping these questions together by broad topic area to have an overview of what you’re covering and help get the order right.

Structure

The key things to consider when structuring a questionnaire are:

• Introduction and screening: In a few sentences, explain what the research is about, how you will use the answers and how long it will take to complete. Be honest, if you say it will take 5 minutes to complete but actually it’s going to take 10 minutes, people are likely to drop out halfway through.

You might also need to ask some screening questions which collect basic profiling characteristics to make sure you are talking to the right type or enough of a certain type of student (e.g. subject area or home / overseas) or business (e.g. location or sector).

• Create a natural flow: It’s useful to think about the order in which people would normally talk about a topic. For example, if you are running a survey to measure student satisfaction during the pandemic, you might want to consider structuring your questionnaire chronologically, starting with arrival in Freshers’ Week through to current views on course delivery.

Linked to this, if you need to tackle harder or more sensitive topics, ‘warm up’ respondents by asking easier or more general questions first. Likewise, if you need to collect more personal information, then place these questions at the end, unless the information is needed to screen at the beginning of the survey. And remember to include a ‘Prefer not to say’ option.

• Vary question types: Try and add different types of question to maintain interest, for example, when capturing how satisfied students are with various aspects of their learning experience, you could combine agree-disagree statements with questions that ask them to rank different elements of learning by level of satisfaction.

Types of question

How you ask your questions will determine the sort of data you collect and the type of analysis you can conduct. Think about how you will use your survey data at the end and what this means for how you ask your questions. Some common question types include:

• Scales with labels or numbers, for example ‘very good, fairly good, neither good nor poor, fairly poor, very poor’ or a scale of 1 to 5 with 1 being ‘very good’ and 5 being ‘very poor’. Scales should always be balanced with the same number of ‘positive’ options as ‘negative’ options and the two ends of the scales should be genuine polar opposites (e.g. ‘very good’ and ‘very poor’ rather than ‘excellent’ at one end and ‘very poor’ at the other end).
• Open questions, where a respondent answers in their own words and are best used when you don’t have a good idea of what the answer might be or if you want to collect quotes for example, ‘What have you enjoyed most about your time at university?’
• Closed questions, where an answer is selected from a pre-determined list. Be careful not to introduce any response bias by rotating the order in which response options appear so the same answer doesn’t always appear at the top of the list. Make sure important answers aren’t missing from the list – add an ‘Other (specify) as a safety net – or that two or more contradictory answers can’t both be selected.
• Ranking used to find order of preference for items on a list. This type of question is most useful to differentiate between items when everything is obviously either a ‘good thing’ or a ‘bad thing’. The list should be limited to 7 or 8 items – for longer lists, asking for 1^st, 2^nd and 3^rd preference would be a better option.

Question wording

The way a question is worded is critical in how it is interpreted. Badly worded questions can lead to careless, inaccurate or dishonest answers. Keep in mind the following:

1. Simplicity – keep language simple, but tailor it accordingly, how you ask questions to students will be different to how you ask them to employers.
2. Length – use short sentences, but say enough to be specific (e.g. rather than ‘Where do you live?’ use ‘In what town or city do you live?’)
3. Ambiguity – be aware of common words that are ambiguous and use definitions wherever necessary (for example, if asking ‘Do you regularly travel to the university campus by bus?’ include ‘By regularly, we mean at least twice a week).
4. Assumption – keep in mind not everyone will be able to answer the question or know something (for example if asking an employer, ‘How often do you take on apprentices?’ this should be split into ‘Have you ever taken on an apprentice?’ and then if yes ‘In the last three years, how many apprentices have you taken on?’).
5. Leading wording – Steer clear of phrasing that guides the respondent to an answer or implies certain responses are more ‘normal’ or expected.
6. Avoid asking about more than one thing in the same question e.g. How would you rate your accommodation and the on-site support services? You won’t know if a student is rating the overall accommodation, the on-site support services or giving an average across the two.
7. Avoid double-negatives, for example ‘Do you disagree with the university’s decision not to run face to face lectures this term? – Yes or No’ Such questions are difficult to understand and are likely to cause confusion and yield inaccurate answers.
8. Memory limitations – err on the side of using shorter reference periods and avoid ambiguity by stating precise timeframes you are interested in.

A few final pointers on format, accessibility and length

As part of planning the project, you will have already decided how you are going to run your survey –online, telephone, or indeed a mix of these.

Remember, how you run your survey can influence how questions are answered. For example, you want to ask an applicant a closed question – Why did you shortlist this university in your application?  When you display this question on an online survey, the applicant will see a list of response options to choose from. However, unless you read out these options in the equivalent telephone survey, they will not ‘have sight’ of them. Their exposure to the list of response options is different depending on how they are surveyed which may result in different answers being given.

Make sure you cater for all needs, for example large font or high contrast colours when completing an online survey for those with visual impairments or advance sight of the questions for those completing a telephone survey but are hard of hearing.

Typically, the shorter the questionnaire, the better. The longer a questionnaire is, the higher the dropout rate. As rule of thumb it is good practice to keep an online questionnaire to no more than 10 minutes, and a telephone questionnaire to 15 minutes.