Usability Testing 101: A Guide on Unmoderated Usability Tests, Prototype Tests and Concept Tests

As product people, we have all had moments such as the following: For weeks, your team has hustled to dedicate numerous hours brainstorming and creating your product, ensuring it fits all the specs and all the core features are nailed down. However, in the back of your mind you keep thinking: "Will our users understand how to use this?”

Usability testing is a commonly used technique to bring your product to the users, have them engage, think-out-loud, and provide feedback. It's ideal for uncovering key issues with your product’s usability, intuitiveness, and overall experience to your target customers.

What may seem obvious to your product team that live and breathe in the product space, might not be as clear to your customers. By validating assumptions and integrating usability testing throughout the product development process, you can quickly identify and address problems, ensuring a seamless and continuously optimized user experience.

Usability testing is a powerful method for evaluating your product with your users. In this guide, we'll cover everything you need to know to conduct effective usability studies that deliver actionable insights to enhance your product's user experience.

Elevate your continuous UX Research

Validate assumptions with usability study, and prototype testing with Hubble

Try for free

What is usability testing and how is it useful?

Usability testing allows product teams to validate their hypotheses about customers and the product by testing assumptions with real users, even at the concept or low-fidelity prototype stage. The study results provide actionable insights, highlighting potential issues to address. This user testing process allows users to get the most value from the product experience without encountering usability challenges.

Through well-designed usability tests, product teams can build impactful products that not only solve users' problems but also more effectively. Usability testing is just one of many user testing methods aimed to evaluate the usability of a product. There are other types of user research:

• In-depth Interviews: Conduct one-on-one or group interviews with users to explore their experiences, opinions, and pain points.
• Focus Groups: Bring together a small group of users to discuss specific topics or product concepts, facilitating group discussions and idea generation.
• Surveys: Collect quantitative data from a large number of users to gather insights on their preferences, demographics, and attitudes.
• A/B Testing: Compares two or more design variations to determine which performs better based on qualitative feedback or quantitative metrics like task success or time on task.
• Card Sorting, Tree Testing: Help evaluate the effectiveness of information architecture by understanding how users categorize, navigate, and label content.
• First Click Testing: Involves whether users click the right area or button first when trying to complete a task usually visualized through click/heatmap.
• Five-Second Testing: Evaluates initial impressions by showing a design for five seconds and asking users to recall what stood out.

Different Types of Usability Testing

Usability testing takes many forms, and choosing the right approach starts with aligning your team and defining clear goals. By understanding your objectives, you can select the method that provides the most valuable insights.

Conducting a prototype test with the appropriate audience and well-defined objectives is essential for assessing usability and customer satisfaction. At a high-level, there are three parameters to take into account when designing for an effective study:

1. Moderated vs Unmoderated
2. Remote vs. In-Person
3. Quantitative vs. Qualitative

Moderated vs. Unmoderated

Moderated and unmoderated usability tests are two common approaches to conducting usability testing, each with its own advantages and considerations.

In moderated usability tests, a facilitator (usually a researcher or usability expert) is present during the testing session. The facilitator runs and engages directly with participants, observe their actions, and ask questions in real time.

• Moderated tests are effective for collecting rich qualitative testing as it allows for relevant follow-up questions to fill-in the gaps, but involves more time and resource investments.

In unmoderated usability tests, participants complete tasks independently without a facilitator. They follow predefined instructions and tasks provided by the researcher using unmoderated usability testing tools to record their actions and behaviors.

• Unmoderated tests are resourceful for collecting user feedback quickly as participants can complete them at their own time and pace. A well-crafted unmoderated test can be just as effective as a moderated test.

Remote vs. In-Person

The COVID-19 pandemic has transformed research practices, with video conferencing becoming a standard practice of the work environment. As a result, product teams have ways to utilize remote studies more effectively rather than relying on in-person sessions in usability labs (while hardware testing still heavily requires in-person sessions).

Remote testing involves conducting tests with participants remotely, often through video conferencing software or online usability tools. This method is convenient and cost-effective, as it eliminates the need for travel and can be conducted with participants from all over the world. Remote usability testing allows you to gather feedback from a diverse user base, providing insights into how different demographics interact with your product. However, it can be more challenging to establish a rapport with participants and may not provide the same level of depth and nuance as in-person tests.

In-person testing, on the other hand, involves conducting tests with participants in person, often in a usability lab or other controlled environment. This method provides a more immersive and engaging experience for participants, allowing for more in-depth and nuanced feedback.

In-person testing enables real-time observation and interaction, which can be invaluable for understanding user behavior and gathering qualitative usability testing focuses. However, it can be more expensive and logistically challenging than remote testing, as it requires travel and coordination of schedules.

Quantitative vs. Qualitative

Quantitative Usability Testing focuses on measurable metrics to evaluate user performance, such as task completion rates, time on task, error rates, and satisfaction scores. It provides statistical insights to assess usability trends and compare design options.

Qualitative Testing explores user behavior, qualitative feedback, and emotions through methods like think-aloud protocols, interviews, and observations. It identifies underlying issues, uncovers pain points, and provides rich, detailed feedback for improvement.

Both approaches complement each other: quantitative testing highlights what’s happening, while qualitative insights explain why it’s happening, and often a study comprises of both qualitative testing and quantitative approach (Mixed methods) for a fuller study.

‍

What are the benefits of usability tests?

If you have not conducted any usability studies before, you may be asking, why are they important and what benefits do they provide? Usability study is a crucial component of the user-centered design process and offers numerous benefits for product development and improvement.

1. Identifies Usability Issues

“Pay attention to what users do, not what they say.”

_{Jacob Nielsen}

First and foremost, usability test helps uncover specific usability problems and challenges that users encounter when interacting with your product.

This includes issues related to navigation, layout, content, functionality, and more - you can actually observe how users engage with the product to analyze their interactions.

2. Improves User Satisfaction

By addressing the usability issues identified during testing, you can enhance the overall user experience. A more user-friendly product leads to increased satisfaction, trust, and loyalty.

Even if we think we understand our users, talking to users and testing your assumptions helps you continuously understand their needs and improve the product experience.

3. Optimizes Design and Functionality

Usability test is a valuable tool for optimizing the design and functionality of a product or interface. By systematically gathering customer feedback and observing their interactions, usability study helps identify areas for improvement and informs evidence-based design decisions.

Design optimization is a continuous effort, and usability study can help your team identify key issues that need attention.

"Teams need to make a commitment to continuous improvement, and that means not simply refactoring code and addressing technical debt but also reworking and improving user interfaces. Teams must embrace the concept of UX debt and make a commitment to continuous improvement of the user experience.”
‍
_{Jeff Gothelf}

4. Saves Time and Resources

Conducting usability studies may seem costly upfront when it involves deliberate planning and consistent stakeholder buy-ins. It is important to establish them as part of a product lifecycle ritual.

Detecting usability issues and addressing them will save you time and money in the long-run. The value of user research lies in failing fast so that you make iterative improvements during early stages.

5. Supports Data-Driven Decisions

Usability study provides both quantitative and qualitative data about user behavior and preferences. Audio and video recordings can help unlock qualitative feedback while heatmaps, time-on-task and completion rates can provide quantitative insights.

6. Enhances User Engagement

Usability study helps you identify potential usability issues and make improvements before releasing a new product or feature. This can be hugely helpful in building delightful user experience and increase user engagement.

Elevate your continuous UX Research

Validate assumptions with usability study, and prototype testing with Hubble

Try for free

When should I conduct a Usability Test?

By now, you will have a good sense of how important prototype testing and usability research are. Usability studies can be conducted throughout the product development cycle. Testing interactive prototypes is crucial for gathering valuable feedback, which drives iterative improvements and aligns the design more closely with user needs and business objectives.

If you need step by step instructions on how to start building a Hubble usability test, please check out this guide from our help center and this blog post.

1. Early Concept and Ideation Phase

Through concept and prototype testing, you can pinpoint user requirements, expectations, and possible areas of frustration in early stages. With low fidelity prototypes or even paper prototypes that involve storyboards and mock-ups, the goal is to deliver the value of the product and focus less on user interaction.

Don't be afraid to show incomplete designs. Product teams often use low fidelity prototypes such as paper prototypes to convey the concept of a product before building anything. Successful prototype testing early on is critical aspect of product development, ensuring early feedback and validating new features before implementation to avoid costly revisions.

Using this information as a baseline, you can initiate the design process with a strong basis for crafting a user-focused product.

“Testing with one user early in the project is better than testing with 50 near the end.”  
‍
_{Steve Krug}

2. Mid-Design and Prototyping Phase

Usability and prototype testing can be valuable when you have created high fidelity or interactive prototypes. Testing at this stage allows you to identify and address usability issues before committing to full development.

Test your prototype throughout the design process to uncover design flaws and gather customer feedback. Early and continuous prototype testing with basic wireframes ensures that the prototype meets user needs and allows for effective iterative process.

3. Pre-Launch or Beta Testing

At this stage, you already have a near-final or beta version of your final product. In this time, you can use usability testing to evaluate its overall effectiveness. This ensures that you can fix mistakes and make critical adjustments before the product is launched into the hands of users.

4. Post-Launch and Continuous Improvement

Even after launching and having a live final product, it's important to continue to run usability studies to prioritize your product roadmap. As the product continues to mature, you can run multiple tests with your own users and with external target audience.

‍

How to create and conduct a unmoderated usability study

1. Define a goal and target audience

Clearly outline the goals and objectives of the research. What specific aspects of the product's design or functionality are you looking to evaluate or improve?

As a next step, determine the target audience of the product or test. A good set of questions to answer would be the following:

• What's your targeted demographic (e.g. seniority, professional skillsets, job titles)?
• What do you want your testers to accomplish with your product? I.e., What tasks or jobs does your product help your customers achieve? (e.g. purchase an item, book a home, add a new integration)

With answers to the questions above, draft a problem statement that you are trying to find an answer to through the study. Below are a few examples of a task:

• "Search for and locate today’s weather forecast for San Francisco."
• "Schedule a haircut appointment for next Friday at 10:00 AM."
• "Order a coffee maker and ensure it will be delivered within three days."
• "Pay your monthly internet bill through the app and download the receipt."

2. Choose a user research tool

Choose a prototype testing tool that your product team can use to build, design, and conduct tests across the product development cycle.

Hubble allows for continuous testing throughout the development cycles including the early ideation, prototyping, beta testing and post launch, supporting various prototype testing methods.

To see our curated list of usability/prototype testing tools, refer to the article usability testing tools.

Features to look out for:

Prototype Testing: Some prototype testing tools explicitly feature prototype testing with integrations with prototype tools such as Figma, AdobeXD, InVision, and more. Depending on the type of tool that your team uses, you might consider what integrations each research tool supports.

Participant recruiting is also another key element for streamlining your research. Research tools often offer recruitment panel that you can directly source from. At Hubble, we've partnered with two leading recruitment platforms, User Interviews and Respondent, to bring customers millions of high quality participants in our platform.

Various user testing methods including card sorting or in-product surveying to complement your usability/prototype testing with IA testing and live feedback.

To learn more in depth about evaluating new usability/prototype testing tools, we recommend the section on choosing the right usability testing tool.

3. Establish your evaluation criteria

Before launching the study, determine what you would like to measure. There are several metrics that can be collected through a tool like Hubble including:

Depending on the goal of your test, consider evaluating key usability metrics like:

• Direct and indirect success
• Customer satisfaction score
• Task completion rate
• Time spent on task
• Misclicks

It's also a good idea to establish benchmark test results using earlier prototypes or competitor products, so you can track user performance as your product evolves over time.

4. Outline a study script

Every good usability test starts with a solid script that outlines the tasks. The script serves as the main guideline for test participants and provides an opportunity to identify any potential problems or areas of improvement.

To learn in-depth about writing effective scripts, we recommend our article on writing effective usability test scripts with examples.

When designing the study, make sure that your tests are concise. Factors, such as the complexity of the product, the scope of the testing goals, and the level of participant engagement, will influence the length of the study.

However, as a general rule, a typical unmoderated test lasts around 15–20 minutes and involve 5–10 tasks. This timeframe strikes a balance, keeping participants engaged while allowing your team to collect valuable feedback.

5. Run pilot tests

To ensure that your test is ready to launch, get some of your team members to participate in the test. To remove cognitive bias, try to involve team members that are not directly involved with your project because ideally they should approach the usability test without having an idea of what it's trying to validate.

The pilot test will reveal any potential issues related with your script, the logic flows, the prototype loading time and other factors that may influence the outcome of your test.

6. Recruit test participants

After building the usability test, you must recruit participants that pertain to the audience you are looking to target. Here is how you can find participants for your usability test:

Find testers from your user base

For starters, you can have your own product users as participants. There are multiple different ways to find the user participants, including in-product recruitment surveys, emails, and even phone calls.

Testing with your own users does involve pros and cons; on the positive side, it can help you collect real, tangible feedback from people that are using your product every day. On the other hand, many of your users will have biases because they are already familiar with the product have cognitive bias regards to how the workflow should look.

Using Hubble’s in-product SDK, you can opt in users to start building an internal participant group.

Recruit from a panel

You can also find participants from external pools using Hubble. Make sure to set up effective screener questions to ensure high quality participants in the demography that you are looking for.

Hubble's external participant pool includes 3 million participants from more than 150 countries. You can find both B2B and B2C participants. For more information on recruiting participants effectively, please refer to our article on finding participants.

7. Analyze data and create a narrative

Analyzing data from a usability study involves examining both quantitative metrics and qualitative insights to uncover patterns and trends in user behavior. Start by organizing the data—categorize user feedback, task success rates, time on task, error occurrences, and any observed frustrations or confusion.

Identify recurring issues across participants, such as points where users struggled to complete tasks or misunderstood the interface. Quantitative data, like task completion rates or usability scores, provides a measurable overview of performance, while qualitative data from think-aloud protocols or post-test interviews offers context to explain user challenges and successes.

Creating a narrative from this analysis means synthesizing the findings into a cohesive story that conveys the user experience. Begin by highlighting the key takeaways: what worked well, what didn’t, and why. Use participant quotes, screenshots, or examples to illustrate specific problems or successes.

Link these observations to actionable recommendations, prioritizing fixes for critical usability issues. Presenting your findings as a narrative helps stakeholders and development team understand the real-world implications of the test results, driving alignment on next steps to improve the final product. A compelling narrative not only reports data but also brings the user’s journey to life, fostering empathy and encouraging user-centered decision-making.

8. Iterate the process

Iterating the testing process involves refining your methods, objectives, and final product based on user insights gathered from initial tests. After analyzing the results of a usability test, identify unresolved issues, areas for improvement, and new questions that emerged during the evaluation. Update the design to address the identified pain points and prioritize changes that enhance the user experience.

Simultaneously, refine the testing process itself by evaluating what worked well and what could be improved, such as revising test tasks, recruiting a more representative user group, or adopting new tools to gather deeper insights.

This iterative process ensures continuous learning and improvement, making the product progressively more user-friendly with each product lifecycle. By testing revised designs with users, you validate whether the changes effectively resolve prior issues and uncover new challenges that may arise.

Each iteration not only improves the product but also fine-tunes the research methodology, creating a more efficient and impactful research protocol. Research iteration will also foster a culture of user-centered design to ensure that the product evolves in alignment with user needs and expectations.

‍

Elevate your product's user experience by using usability testing

Usability study is an amazing tool for any product team that can help to elevate a product's user experience. Through usability research, teams can understand their users better and unearth hidden insights that can provide guides to help prioritize the right initiatives to improve the product for its users.

Following the guides and tip above, usability studies will allow let you:

• Discover UX pain points and issues
• Prioritize key features on your product roadmap based on impact
• Understand how users interact with your product
• Gather both quantitative and qualitative user feedback on product features

If you need any help in implementing usability tests, or need any help in evaluating various tools, please try Hubble to help conduct continuous research across the product development cycle. Using Hubble, world class product teams have been conducting various types of research including building prototype usability tests, recruiting participants and launching in-product surveys, throughout different stages of product ideation and development.

Elevate your continuous UX Research

Validate assumptions with usability study, and prototype testing with Hubble

Try for free