UX Research 101: Guide to Surveys

Survey is an important tool in UX research that helps you understand user experiences and preferences through a set of structured questions. It plays a crucial role for product teams, being one of the most widely used form of research that is applicable at various stages of the product development cycle, ranging from early discovery phase to post-production phase.

Needless to say, survey should be a fundamental component of your research toolkit, as it offers valuable user insights at a relatively low cost and time compared to other methodologies like in-depth moderated interviews and usability testing. To maximize its benefits, you should be deliberate in survey design and how you process the survey results.

This guide presents some of the best practices and tips on how to build impactful surveys and drive actionable insights.

Launch smarter surveys with Hubble

From surveys to in-product intercepts, collect contextual feedback that drive action

Schedule a demo

What are UX surveys?

Surveys are structured data collection instruments used in product research to gather feedback, and preferences from a sample of users. Usually, survey respondents are presented with a series of questions designed to collect specific responses related to their experiences with a product or service.

There are various survey modes including online forms, emails, in-app and product prompts, or paper-based questionnaires. One of the most powerful advantages of UX surveys is that they are easy to scale, offering a quick way to collect large quantitative measures to complement qualitative research methods, such as user interviews and usability testing.

Some of the common objectives for running UX surveys are:

• Exploring the problem space: Because they are relatively cost effective, surveys serve as efficient tools that can be used to initially explore a problem space.
• Uncovering the key factors: Surveys help uncover the "what" and the "why" by collecting data and insights to a particular topic or problem space.
• Understanding user needs: Surveys serve as a key research tool to gain insights on user preferences, pain points, and expectations.
• Gathering feedback: Polls and simple questionnaires provide a platform for users to express their opinions and provide feedback on products or features.
• Validation: Surveys can be used to validate hypotheses, test assumptions, and gauge user reactions to proposed ideas.
• Quantitative analysis: By collecting numerical data, you can draw conclusions based on significance tests.

What are the advantages of surveys?

UX Surveys are a indispensable tool in user research. Some of the pros of using surveys include:

• Versatility: Surveys are versatile, meaning that they can be used during various stages of product development, from initial discovery to gathering customer feedback on new features or designs.
• Cost-effectiveness: They are very cost effective relative to other research methodologies such as moderated user interviews, focus groups or usability testing.
• Scalability: They can be administered to a large number of participants, making it an effective option when gathering insights from a diverse audience and driving statistically significant results.

What are the disadvantages of surveys?

• Limited depth: Often times, UX surveys don't provide as much depth as other qualitative research methods like user interviews or observation, because they rely on a set of predefined questions within limited time.‍
• Response bias: There is always an inherent risk of response bias, as research participants may not accurately represent the broader user population, leading to skewed results.
• Lack of context: Questionnaires may fail to effectively gather contextual information that can be otherwise obtained through direct interaction with users.
• Incomplete data: Study results often include incomplete responses and miscellaneous errors, which require an additional step of data cleaning before analysis.
• Potential for misinterpretation: Without proper validation or follow-up questions, participants' qualitative or textual response may be limited or open for misinterpretation.

Despite some of the disadvantages listed above, survey is a fundamental toolkit for researchers that can be utilized at various stages of design research process.

How are surveys different from polls and questionnaires?

Surveys, polls, and questionnaires are often used interchangeably, but they serve different purposes.

1. Surveys: are comprehensive research tools used to collect user feedback, typically involving a series of structured questions designed to gather detailed feedback on various topics. Surveys are often used in UX research to gather insights on user preferences, behaviors, and experiences.
2. Polls: are typically shorter and more focused than surveys. They usually consist of one or multiple questions and are designed to quickly gather opinions or preferences from a large audience. Polls are commonly used to gauge public opinion on specific topics or to gather feedback on a particular issue.
3. Questionnaires: are similar to surveys but are typically shorter and less comprehensive. They often consist of a set of predefined questions that respondents answer in writing. Questionnaires are commonly used to gather specific information or opinions from a large group of people.

‍

Different Types of Surveys

Surveys can be classified into different categories depending on their structure and type of data they are designed to collect.

Quantitative Surveys

Quantitative studies are designed to collect numerical data to quantify user behaviors, attitudes, or opinions within a population mostly through closed-ended questions and predefined response options. By quantifying the study results, researchers measure and analyze trends, correlations, and statistical significance at scale.

This approach commonly uses closed-ended questions, such as ranking, Likert scales, or multiple-choice questions, making the data easy to quantify and statistically analyze. They are ideal for validating hypotheses, identifying patterns, and supporting decision-making with measurable insights.

Common applications include market research, customer satisfaction (e.g., NPS), usability feedback, and product feature prioritization. Their standardized format ensures consistency across responses, allowing for comparisons and generalizations across a target audience.

Qualitative Surveys

On the other hand, qualitative ones use open-ended questions and free-form responses to gather rich, descriptive data about participants' experiences, perceptions, and attitudes. Qualitative approach is often used to explore motivations, uncover pain points, and gain a deeper understanding of user needs and behaviors.

While the method provides valuable context and nuanced insights, it requires more time for analysis due to the unstructured nature of the data. It typically involves smaller sample sizes and utilize qualitative coding and thematic analysis to process open-ended survey questions.

Mixed-methods surveys

Mixed-method surveys combine elements of both quantitative and qualitative studies to get a comprehensive understanding of the users. Mixed-methods usually involve the sequential or concurrent collection of both quantitative and qualitative data.

Mixed-methods offer a comprehensive understanding of research questions by triangulating data sources, validating findings, and providing a more holistic view of user experiences.

Because of these benefits, mixed-method approach is often used to corroborate quantitative findings with qualitative insights, provide context to numerical data, or explore unexpected findings in more depth.

Cross-sectional surveys

Cross-sectional studies collect samples from participants at a single point in time. This form of questionnaires lets you examine relationships between variables and make inferences about the population at that specific moment.

Cross-sectional studies often involve administering questions to participants from different demographic groups or segments to capture a broad range of perspectives.

Longitudinal surveys

Longitudinal studies collect feedback from the same participants over an extended period, allowing you to track changes, trends, or developments over time.

This type of study enables to observe individual trajectories and identify potential causal relationships. Participants are surveyed multiple times at regular intervals, ranging from weeks to years, to assess changes or stability over time.

In-product surveys

In-product intercepts are integrated directly into a digital product or website, typically presented to users while they are actively using the product. These surveys aim to collect contextual feedback, opinions, or user experiences related to specific features, functionalities, or interactions within the product.

In-product intercepts are often triggered based on user behavior, such as completing a specific task, visiting a specific URL page, or spending a certain amount of time on the platform. In-product surveys can be presented through various formats, including pop-up dialogs, slide-out panels, or embedded forms within the user interface.

This approach is commonly used by product teams to gather feedback on new features, user satisfaction, usability issues, or product-market fit. It becomes an effective channel for active users to provide feedback on the product.

Transactional surveys

Transactional studies are sent to users immediately after completing a specific transaction or interaction, such as making a purchase, completing a support ticket, or signing up for a service. These surveys aim to collect feedback on the user's experience particular to the transactional process.

Transactional ones are typically triggered automatically based on predefined events or actions, such as order completion, service cancellation, or customer service interaction. They are often delivered via email or in-app notifications and contain questions related to the specific transactional experience.

These studies are widely used in e-commerce, customer support, and service industries to measure customer satisfaction, gather feedback on service quality, or identify opportunities for upselling or cross-selling. In addition, transactional surveys are also used in subscription-based models to assess user retention and churn factors.

When should I run a survey?

The study should be strategically timed and aligned with the goals of your user research and product development efforts. A few examples include:

Product discovery phase

During the initial product discovery phase, surveying serve as a valuable tool to explore initial problem spaces, understanding user needs and preferences before starting the actual product development process. Surveying in the discovery phase helps gather early signals to inform product ideation and feature prioritization.

Iterative development

UX surveys can be conducted throughout the product development lifecycle to gather feedback at various stages of iteration. For example, you can run a quick poll after launching a new feature or product update to gauge user satisfaction, identify usability issues, or gather suggestions for improvement.

Onboarding experience

During an onboarding process, ask questions to assess how users are understanding the product or its features. By collecting feedback from newly onboarded users, you can identify areas where additional guidance or support may be needed to improve the onboarding experience.

Data triangulation

Because survey is relatively easier to implement and scale than other research methods, you can utilize it to complement other methods. For example, having a follow-up questionnaire after a usability testing session can provide opportunities for additional feedback from users.

Post-interaction feedback

Surveys can be triggered after specific user interactions or transactions within the product, such as completing a purchase, submitting a support ticket, or attending a webinar.

Collecting feedback immediately after these interactions provides contextual feedback on the user experience and allows for timely improvements.

Regular user feedback

Establishing a fixed cadence (i.e. monthly or quarterly) for running studies can help maintain ongoing communication with users and track changes in user needs and behaviors over time.

Regular practice also demonstrates a commitment to listening to user feedback and continuously improving the product.

‍

Best practices for an effective survey design

Below are some checklist and best practices to consider when devising survey questions for a successful questionnaire design:

• Every question in the survey should be intentional and purposeful. If the study is too lengthy, revisit each question and evaluate why its being asked.
  • Demographic questions often provide little insight into user behavior. If you're asking about factors like education level or marital status, be sure to consider the purpose and relevance of the question.
• Keep the study concise and relevant to avoid fatigue.
• Avoid asking double barreled questions that ask for more than one information at a time. Break down the questions as necessary. An example of a double barreled question is: "How satisfied are you with the check-out experience and customer service?"
• Avoid binary yes-no questions; instead, provide more specific options to learn about user behaviors.
• Proof read to check that the type of question (single select, multi-select, numerical scale, etc.) and answer choices are the most appropriate options for each question.
• Double check question wording so that the questions are clear. Use simple terms and avoid double negatives.
  • Avoid asking leading questions or giving away important keywords.
• Use skip logic as needed to facilitate the survey's flow.

‍

How to Launch a Survey

1. Outline your research objectives

First, start by clearly defining the study objectives. What design questions need to be answered, and what insights would help? Establishing clear objectives for your study helps you design and launch a good survey with high quality and insights.

2. Define your audience

Determine the target population based on your study objectives. Who are the key stakeholders or user segments you want to gather feedback from? Consider user behavior patterns and demographic factors such as age, gender, location, and occupation that may influence the responses.

3. Choose the right user research tool

Select a tool or platform that aligns with your study needs. There are many options available, ranging from simple free builders like Google Forms to more robust platforms like SurveyMonkey or Qualtrics. Choose a tool that offers the features and functionality you require, such as customizable question types, respondent targeting options, and data analysis capabilities.

At Hubble, you can launch full-scale survey research or in-product surveys to quickly gauge user feedback in realtime. In addition, you can recruit participants and distribute studies within a single app, complete the study with usability testing or collect quantitative NPS and CSAT scores.

4. Devise your survey questions

Create a clear and concise list of questions that effectively captures the information you need to collect from your respondents. Start by drafting a list of questions based on your research objectives, keeping them focused and relevant to the topic at hand. Use a mix of question types, including multiple-choice, rating scales, and open-ended questions to gather relevant information.

5. Run a pilot study

Before distributing the study, run a pilot study internally or with a small group of participants to identify any potential issues or ambiguities in the questionnaire. Review their feedback and make necessary revisions to improve clarity in answer choices, wording of survey questions, or flow of the study.

Make sure the introduction is clear and visible with information containing what to expect, approximate time it takes, and link to privacy statements as needed.

6. Set up distribution channels

Determine the most appropriate channels for distributing your study to reach the desired audience. This could include email invitations, social media posts, website pop-ups, or in-app notifications. Consider the preferences and behaviors of your audience when selecting distribution channels to maximize response rates. Use survey participant pools like Hubble's participant pool to find specific personas.

In order to encourage participation, clearly communicate its purpose and value to potential respondents. Highlight any incentives or rewards for participation, such as discounts, prizes, or access to exclusive content. Use persuasive messaging to motivate respondents to complete the survey and emphasize the importance of their feedback.

7. Launch your survey

Once everything is set up, launch your survey to your target audience. The time it takes to reach the desired sample size depends on the participant profile. Keep track of response rates and monitor the progress of your survey when you can. If response rates are lower than expected, consider sending reminders or follow-up communications to encourage participation.

8. Analyze and interpret survey results

Analyze the data to extract meaningful insights and actionable findings. Use statistical analysis tools or qualitative coding techniques to search for trends, patterns, and themes. Interpret the results in the context of your research objectives and use them to inform decision-making and product improvements.

‍

Key Steps to Driving Actionable Insights

Analyzing study results involves several key steps to derive meaningful insights.

Revisit research objectives

Revisit the original study plan and objectives, and skim through the data. Clearly outline what you aim to achieve with your analysis. Determine the key questions you want to answer and the insights you hope to uncover.

Data preparation and cleaning

Before diving into analysis, the data needs to be clean and organized. Depending on the distribution channel, the quality of the data may vary and require more manual inspection for cleaning, which mainly includes removing incomplete responses, redundant submissions, and formatting textual data.

Explore correlations and identify patterns

Explore relationships between different study variables. Use cross-tabulations to examine how results vary across demographic groups or other relevant segments. For more details on quantitative analysis, see below. Additionally, look for patterns and trends in the data. Use visualizations such as charts, and graphs to identify patterns and trends visually.

Quantitative Survey Analysis

Quantitative analysis provides valuable insights through numerical data and statistical significance. Depending on the type of data you have, analysis approach will vary. Here are some common methods from basics to more advanced:

Descriptive statistics

Begin by conducting descriptive analysis to summarize the main characteristics. Calculate basic descriptive statistics such as frequencies, percentages, means, medians, and standard deviations to summarize the distribution.

Correlation analysis

Conduct correlation analysis to identify relationships between continuous variables. Determine whether changes in one variable are associated with changes in another variable and the strength of these relationships.

T-test

Perform t-tests to compare means between two groups and determine whether there are statistically significant differences. This method is useful for analyzing results with a binary or categorical independent variable.

Chi-square test

Conduct chi-square tests to examine the association between categorical variables. This technique is commonly used to assess whether observed frequencies differ significantly from expected frequencies.

Regression analysis

Use regression to explore the relationship between one or more independent variables and a dependent variable. This technique can help identify predictors of specific outcomes and quantify their effects.

Analysis of Variance (ANOVA)

Use ANOVA to compare means across two or more groups and determine whether there are statistically significant differences. This method is particularly useful for analyzing survey data with categorical independent variables.

Time-Series Analysis

If your survey data includes repeated measurements over time, use time-series analysis techniques to explore trends, patterns, and changes over time.

Qualitative Survey Analysis

Analyzing qualitative data involves a more systematic approach to examining and interpreting text-based responses to open-ended questions or comments. Unlike quantitative data, which can be easily summarized through numbers, qual data requires a more in-depth and subjective approach to analysis. To learn more in depth about analyzing qual data in user research, refer to our comprehensive guide on qualitative data analysis.

Get familiarizing with the data

Begin by familiarizing yourself with the data by reading through all the survey responses to get a holistic understanding, and potential patterns.

Coding

Organize the notes by coding, which involves systematically labeling or categorizing responses based on their content or themes. Coding helps identify recurring topics, ideas, or sentiments.‍

Thematic analysis

Conduct thematic analysis to identify overarching themes and patterns that emerge. Look for commonalities, differences, and relationships between codes to develop a deeper understanding of the respondents' experiences, opinions, and behaviors.

Data triangulation

Once you have identified the themes, compare the qualitative findings with quantitative data. Validate the qualitative findings by comparing them with other data sources or perspectives.

Interpret the qualitative findings by synthesizing the themes, patterns, and insights uncovered during the analysis. Explore the implications of the findings in relation to the original design questions, and consider how they can inform decision-making, problem-solving, and further research efforts.

In this guide, we explored how to use surveys for UX research. Survey is an instrumental research toolset because of its versatility and relatively low cost of execution to gauge user behavior and feedback.

Driving survey success with Hubble

In this guide, we explored how to use surveys for UX research. Surveys are important instruments that cannot be excluded from any UX research tool set. Because of their versatility and relatively low cost of execution, they can be used widely across the product development process to gather insights from users and maximize quantitative and qualitative data points on user behavior and satisfaction.

Launch smarter surveys with Hubble

From surveys to in-product intercepts, collect contextual feedback that drive action

Schedule a demo

If you are interested in exploring more on UX research and related topics, we recommend some of the articles below:

• To see our curated list of survey tools, see the best survey tools for research in 2024.
• To explore more on AI-powered research tools, see our top 11 AI-powered research tools.
• To get started with Hubble, see our comprehensive guide to Hubble.