The Tools of Validation — Confirming Whether the Design Actually Works
From usability testing to A/B testing and quantitative scales, we look at five validation tools that check, in front of real users, whether a design actually works.
These are the tools you reach for when you need to confirm whether a design actually works.
If the tools of implementation gave a hypothesis its form, the tools of validation put that form in front of users and ask whether the hypothesis holds. This is the moment we come face to face with something this series has emphasized from the very beginning — users do not behave the way designers predict.
Looking back, the journey of this series traces a single loop. We began by observing user behavior to infer cognition, and validation, too, closes the loop by observing user behavior. The difference is that our first observations were aimed at “what experience is the user having right now,” while the observations in validation are aimed at “what experience does the user have in front of the design we’ve proposed.” The fact that both the starting point and the destination are user behavior is a reminder, once again, of what this work is fundamentally about.
The tools of validation, too, belong to no single stage. Validation doesn’t wait for a finished product — it can begin the moment you’re sitting across from someone with a single sheet of paper prototype. What matters isn’t the stage, but the recognition that “this hypothesis hasn’t been validated yet,” and the willingness to go check.
Usability Testing
“Can users achieve their goals with this design?”
Usability testing is a method of directly observing users as they actually use a design. The typical approach is to give users a specific task and watch them carry it out. You observe where they pause, where they get lost, and where they succeed.
There’s a principle that has to be established first in usability testing: what’s being tested is the design, not the user. When a user fails to complete a task, that is a failure of the design, not a failure of the user. The moment this perspective gets inverted, the facilitator starts steering the user toward the “correct” answer, and the test loses its meaning.
For this reason, facilitators must not help users at the moment they run into difficulty. That difficulty is itself the most honest signal pointing to where the design needs to improve. Questions that probe intent, like “What were you trying to do?” are fine, but statements that steer behavior, like “Try clicking this button,” contaminate the test.
You don’t need a large number of users. According to research by Jakob Nielsen, around five users are enough to uncover the majority of major usability problems. Small, iterative rounds of testing are more efficient than one large panel.
Caution. The more artificially controlled the test environment is, the wider the gap grows between it and real-world usage. Performing a task in a quiet, focused lab setting is a completely different experience from using a phone one-handed in a noisy café. Where possible, test in an environment that resembles the actual context of use, or interpret the results with the environment’s limitations in mind.
Think-aloud Protocol
“What’s going on inside the user’s head?”
The think-aloud protocol asks users to say out loud whatever comes to mind while they use a design. It has the user narrate their own experience — “What are you looking at right now?” “Why did you think that?”
If behavioral observation shows “what” someone does, think-aloud reveals “why” they do it. The fact that a user stares at a button for a long time is something behavioral observation can capture, but whether that’s because they’re anticipating its function or because they don’t understand what the label means and are hesitating — that’s something only think-aloud can surface. It’s the most direct way to hear exactly where the gap lies between the mental model and the design.
There are two variants of the think-aloud protocol. Concurrent think-aloud has users speak while performing the task, and retrospective think-aloud has them look back on their actions and narrate after the task is done. The former captures thoughts in real time but raises cognitive load; the latter doesn’t interfere with performance but risks reconstructed memory.
Caution. People don’t say exactly what they’re thinking. When a researcher is nearby, users unconsciously try to construct explanations that sound more rational and coherent. Rather than taking the content of think-aloud at face value, pay attention to the gap between what’s said and what’s done. When someone says “I clicked this button — of course it does that,” but they actually clicked it three times — that gap is the insight.
Heuristic Evaluation
“When you need to spot a design’s potential problems quickly, without users”
Heuristic evaluation is a method in which UX experts inspect a design against a predefined set of principles — heuristics. It differs from other validation tools in that no users are involved. It’s a form of expert review, using expert knowledge as the yardstick.
The foundation of this method is Jakob Nielsen’s ten usability heuristics. These include visibility of system status, error prevention and easy recovery, consistency and standards, and minimizing cognitive load. Experts use these principles as criteria to systematically document which principle each element of the design violates.
Recruiting real users and preparing sessions takes time. Heuristic evaluation, by contrast, can be carried out quickly by a small number of experts. It’s useful early in a project, or when budget and time are limited, for quickly filtering out obvious problems. It can also serve as a pre-screening step that makes later usability testing more efficient.
Caution. Expert predictions can’t substitute for actual user behavior. A problem flagged by heuristic evaluation might not be a problem for real users at all, and conversely, users sometimes catch problems experts miss entirely. This tool should be used to complement usability testing, not replace it. Also, a single evaluator introduces bias, so where possible, it’s recommended to have three to five experts evaluate independently and then pool the results.
A/B Testing
“When you need data to confirm which of two designs produces better results”
A/B testing exposes two versions of a design to real users at the same time and measures which one performs better. Users are randomly split into groups, each experiencing version A or version B, and quantitative metrics like click-through rate, conversion rate, and bounce rate are compared.
The strength of this tool is that it measures the effect of a single variable while holding everything else constant. “Does the blue button get more clicks than the green one?” “Does the version without an image convert to purchase better than the one with an image?” — A/B testing answers these questions with data instead of impressions or intuition.
A/B testing proves most useful on live services. Because it requires enough traffic to reach statistically meaningful results, it suits the incremental improvement of a product that already has users, more than the early stages of a project.
Caution. A/B testing tells you “what” is better, but not “why” it’s better. Whether the blue button gets more clicks because of its color, its position, or some other factor is something this method alone cannot tell you. It’s also worth noting that a short-term metric like click-through rate doesn’t necessarily align with the long-term quality of the user experience — satisfaction, trust. Optimizing a metric and improving an experience are not always the same thing.
Quantitative Scales (SUS, CSUQ, and others)
“When you want to measure usability as a number and compare it across versions or products”
Quantitative scales are survey-style tools that turn usability experience into a number. The most widely used are the System Usability Scale (SUS) and the Computer System Usability Questionnaire (CSUQ).
SUS consists of ten items, and responses on a 1-to-5 scale for each item are converted through a formula into a score out of 100. A score of 70 or above is generally interpreted as acceptable, and 85 or above as excellent. Because it’s simple to administer and standardized, it’s useful for tracking change over time or comparing different products.
What quantitative scales offer is an objective baseline. A number like “SUS score of 62” conveys the severity of a problem far more convincingly than an impression like “users struggled with it,” and a change to “SUS score of 78” after improvements makes the impact of those improvements unmistakably clear.
Caution. Don’t lose sight of the context a score carries. The same SUS score of 72 carries different weight for a medical device than it does for a social media app. Quantitative scales also measure how users rate their experience, not what they actually went through. To understand the reasons behind a score, you need to pair it with qualitative methods like usability testing or interviews. A number is the start of a question, not the end of one.
These tools ask the same question from different angles: is our design actually improving the user’s experience? Usability testing and the think-aloud protocol observe behavior and thought directly; heuristic evaluation filters issues in advance through an expert’s eye; A/B testing compares outcomes with data; and quantitative scales express results as numbers. None of them is complete on its own — it’s best to combine them according to the nature of the question at hand.
And once more: validation is not an ending, but a link in the loop. What’s discovered in validation feeds back into observation and analysis, giving rise to new definitions, expressions, and implementations. The fact that this loop never stops is exactly why UX design is not a single project, but an attitude.
Now it’s time to bring this series’ journey to a close. The next post looks back over every loop we’ve traveled through and returns to the question we started with — what is UX design?