Noboa™

Bias exist in survey design. It is a “systematic error introduced into sampling or testing by selecting or encoou

Sampling bias

If you are looking to draw conclusions about a certain group, you'll need to ensure that everyone in that group has an equal chance of receiving an invite to respond to your survey.

For example, if you are looking to draw conclusions about all of your customers, you will need to share your survey in such a way that all people have a chance of responding to the survey. If you only share a link to your survey via social media, it's very likely you are biasing your results. Chances are, not all of your customers have social media accounts. Further, only a fraction of those that do, will have liked or followed your company page.

How can I reduce sampling bias?

The easiest way to remedy this trouble is to share your survey via various methods. Send out an email campaign, share it on your website, post QR codes in your physical locations. You might also consider hiring telephone or in-person interviewers to recruit respondents who don't have access to internet in their home or do not otherwise spend much time on the web. You can also mail out paper surveys to segments of your population who are not particularly tech savvy.

Nonresponse Bias

Even if you get the sampling just right, you then need to fight the bias introduced by who responds. There will be some people who are either unwilling or unable to respond, those who are often systematically different than those who do respond.

Increasing response rate will increase the chances that everyone in your population is represented. You can increase your response rates by sending a pre-notification email with information about the survey, a personalized invite, or a reminder.

Response Bias

Humans are complex creatures. Getting people to respond to your survey is only the first step. You also need think about getting accurate answer. Both subconscious and conscious cognitive factors can result in less-than-truthful responses.

Research shows that people want to be agreeable. Often, survey respondents tell you what you want to hear. If your survey includes agree/disagree type questions they’re more likely to agree. On the hand other, respondents tend to only select neutral responses when presented with a 5-point scale. People with the opposite of this bias would select the most extreme options.

To reduce response bias, you should try asking neutrally worded questions and making sure your answer options are not leading.

Order bias

The order of both questions and answer options in your survey matters! Questions that come early in your survey might influence how respondents respond to questions later in your survey.

To eliminate order bias, you can randomize the options shown to each respondents. For example, let’s say you ask “Which social media platform do you use the most?” and ask them to select from a list of LinkedIn, Facebook, Instagram, Twitter, or Snapchat. Based on market research, LinkedIn will be overstated a bit because it shows up as the first in the list. By randomizing these choices it eliminates this bias and shows the 5 social media platforms in a random sequence each time to each new respondent answering the online survey.