Do you want to find out what your clients/employees/residents are thinking, without having to ask them yourself?  Do you want their honest responses so that you can make educated decisions and improve things for the better?  At the end of the day, do you want to be able to make comparisons and predictions based on a pool of accurate, reliable data?

It sounds like you want to build a kick-ass survey!  But survey design is not a “build it, and they will come” project.  In fact, without the right training, it can often be a “build it, and they will run away screaming” enterprise.  You don’t want to irritate or scare away your respondents, so here are eight simple tips and tricks you can use to build better, more productive surveys.

1. Define your purpose.

The best way to figure out where to start is to ask where you want to end.  

• What is your intention?  What do you hope to get out of this survey?
• What kind of comparisons do you want to make?
• What are you trying to prove or disprove, confirm or refute?
• How do you want to apply your results?
• At the end of the day, what do you hope to get out of your data?

Getting a grip on these questions will help you narrow the scope of the survey and focus your questions.  A good rule of thumb is to try to focus on one or two main goals per survey.

2. Go ugly first.

No one said it was going to be pretty.  A good start is to develop a few key topics that you want to explore in the survey, and then jot down every question - no matter how choppy or ugly - that you can think of that falls under each topic, even if only loosely.  For example, if one key topic is employee engagement, brainstorm about what employee engagement is and how you could measure it.  This is the beginning of the “operational” phase (don’t worry, it’s not like the game Operation - nothing will “buzz” if you get it wrong).  You’re operationalizing your key variables, i.e. making them tangible and measurable.At this point, anything goes.  Big questions, like “How satisfied are employees with their work?” are just as good as the little guys, like “How much time do employees want for their lunch breaks?”  Put it all on the table.  It’s much easier to start big than to focus too quickly and lose track of your goals.

As you brainstorm, you’ll want to think about not only the meaty questions (e.g. employee engagement or attitudes about the community), but also the side dishes.  What kind of demographic data do you want to gather?  What other tidbits of information might be useful?  How much information do you want about each respondent (e.g. gender, age, race, income, household situation, marital status, sexual orientation, shoe size)?  More is not always better, but at this point, casting a wide net will help you explore all possibilities.

3. Check with the authorities.   

If you’ve done things correctly, you’ve got a big mess on your hands now, with tons of questions and little direction.  Don’t worry - many people have gone before you, and you can learn a lot from their triumphs as well as their mistakes.  Do a little research on your key topics and see what you can find.  What kind of questions have been asked before?  What kind of data did they use?  What kind of results did they get?  There are peer-reviewed articles in reputable journals on almost every topic you can think of.  Chances are, they’ve asked some of the questions you want to ask, which can help you focus.    You can also research survey design by simply taking surveys.  Sign up for regular surveys from Harris Polls.  Check out online polls and resources from Nielsen or Gallup.  Model your demographic questions after the Census Bureau.  Review nationally representative surveys - and their detailed codebooks—like the General Social Survey.  All of these websites provide valuable resources and examples to help steer you in the right direction.

4. Know your audience.

Take off your survey guru hat, and put on your empathy hat.  Try to put yourself in the position of your respondents, and do your best to approach the survey from their point of view.  How willing are they going to be to divulge information about themselves?  How can you ease any concerns they might have?  What kind of language and tone will speak to them as unique individuals?

A light-hearted tone with some humor might be appropriate for a survey of high-school seniors about their favorite places to shop, but would not be appropriate for a survey of community members about their health practices.  If you’re having trouble walking in your respondents’ shoes, it might be helpful to start with a focus group with the type of respondents you’re targeting.  Find out what speaks to them, what they’re curious about, and how to approach them.  

5. Respect their privacy.

Your survey respondents trust you with their information, and it’s important to respect that trust and keep their information confidential.  If you want to collect demographic information, it’s best to collect it at the end of the survey, and allow respondents some way to opt out if they don’t want to provide it (e.g. add a “decline to answer” option).  If you’re surveying a smaller group (e.g. your employees), or if you’re surveying about a sensitive topic (e.g. health, benefits, income), it’s extremely important not to ask for too much detail about their identity.  If you ask for gender, job title, age, and race at a workplace that’s mostly white and male, your female and minority employees may be skeptical of how “anonymous” the survey really is.

You can ease some of their concerns by being up-front.  The first page of your survey should tell your respondents whether or not the survey is confidential, whom the results will be shared with, what the survey is about, approximately how long it will take, and what the pay-off is.  If you can’t offer a monetary reward or incentive, you can at least offer a sense of importance, e.g. “The results of this survey will be shared with your employer to help make it a better place to work.” 
   
6. Keep it simple.

Theoretically, do you think these survey guidelines are representing the essential truths of survey design?  You probably stumbled while reading that question.  A survey is no place to have theoretical discussions.  Instead, survey questions should be simple, easy to understand, and rooted in experience.  People have a much easier time reflecting on their experiences than theorizing about what could be.  Ask, “In your experience, how satisfied are you with XYZ?” rather than, “Why do you think people are/are not satisfied with XYZ?”    

Keeping it simple also means keeping it short.  A survey that takes 15-20 minutes is a harder pill to swallow than a 5-minute survey.  Let your respondents know how much of the survey they’ve completed by inserting a progress bar or page numbers (e.g. 4 of 6 pages).  Remove clutter by splitting questions up to no more than 5 per page.  

7. Mix it up.

There are many types of survey questions, and each is suited to different goals.  Chances are, one type of question will not suit all of your needs, so don’t be afraid to mix it up a little - this keeps it interesting for respondents, too!

a. Open-ended questions.  These questions have no pre-set answer, and may be a good way to break the ice or explore a new issue.  You can still control the focus of the question by honing in on a specific idea, e.g. “What is ONE wish you have for your organization?” is a more focused question than, “What additional comments do you have?”  A con to these questions is that they can be difficult to analyze and categorize on the back-end.  In other words, be prepared for a data dump!

b. Likert-scale.  A likert-scale is one of the most widely used scales in survey research.  Respondents rate their level of agreement (or level of importance, satisfication, etc.) with a series of statements.  Most Likert items use a 5-point scale, e.g. Strongly Agree (5), Somewhat Agree (4), Neutral (3), Somewhat Disagree (2), Strongly Disagree (1).  Likert scales are a great way to gauge respondents’ feelings or attitudes about a group of related items, e.g. satisfaction with current employee benefits or perception of a community’s “green” initiatives.  The drawback to Likert scales is that they are more difficult to design than open-ended questions.  It’s important to be consistent, smart, and simple in the wording of each statement.

c. Multiple choice.  If you simply want respondents to choose one (or more) responses from a group of choices, then multiple-choice is the way to go.  A single response multiple-chocie question will force respondents to choose only one answer, e.g. the one thing they value most about their organization, or the one resource they use most.  A multiple-response question allows respondents to choose more than one response, e.g. the top three reasons they chose to live in the region, or all the classes they’ve taken in a particular area.  Like Likert-scale items, multiple-choice questions are more difficult to craft than open-ended questions, and require survey designers to imagine all (or most) possible responses ahead of time.
   
d. Ordinal/Ranking.  If you have a series of items that you’d like respondents to rank, then a ranking question should suit your needs.  For example: “Please rank each of the following items from 1 to 5, where “1” is most important and “5” is least important.  Please use each number only once.”  The pro of ranking questions is that you force respondents to prioritize - they can’t rank every item #1.  The con is that you force respondents to prioritize (which some respondents will dread), and not all respondents view rankings the same way.  Joe’s #1 and #2 choices might be neck-and-neck, while Peg’s #1 and #2 choices could be miles apart.

e. Numerical.  Numerical questions are like cherries on the survey guru’s sundae.  They are straight-forward and provide a lot of information for analysis.  Unfortunately, they only apply to questions that have a real numerical response, like age, number of children, money spent on groceries, etc.

8. Test, test, and re-test.

If survey design is like cooking a delicious meal, then you’ll want to taste the sauce before you serve it.  Take the survey yourself and pass it along to trusted friends, peers, coworkers - anyone willing to taste it and give honest feedback.  Your buddy with a penchant for correcting grammatical errors can find all those comma splices you missed.  Your no-nonsense mother-in-law might help you cut down on wordiness.  Your saleswoman coworker might help you jazz it up a little.  Send the survey to a diverse group, and see what comments they have. 

Then, send it to a small sample of your target population, and gather as much feedback as possible.  Then re-test it with another sample.  If the survey has only been seen by 2-3 pairs of eyes, or only 2-3 times, it’s not ready.  Don’t push it in if it hasn’t learned to swim yet.  Test it until your eyes hurt - the pain will be worth it in the end.

Now what?  So you’ve launched it, and the survey gods are rejoicing.  What are you going to do with all that data? Stay tuned for our Survey Analysis guide, “How to Analyze Your Survey Results (without getting in the weeds).”
 

Comments