---
title: "Measuring UX: How to Use Success Rate, Time on Task and SUS Properly"
url: "https://www.insideuxdesign.com/en/blog/measuring-ux-how-to-use-success-rate-time-on-task-and-sus-properly"
language: "en"
description: "Measure UX properly: calculate and interpret success rate, time on task, errors, SUS, SEQ and HEART. With worked examples and a checklist to apply."
published: "2026-09-02T09:00:00+02:00"
updated: "2026-10-15T00:09:59+02:00"
---

# Measuring UX: How to Use Success Rate, Time on Task and SUS Properly

More page views are not a success in themselves. Which metrics really measure the quality of a user experience, how to calculate them properly with small samples and which thinking errors to avoid.

Picture a product team that has redesigned the travel expenses module of an HR software product. Employees use it to photograph their receipts, assign them to a trip and submit the expense report to the accounting team. Three weeks after launch, the team proudly presents its first number: page views in the module are up 30 percent. Shortly afterwards, support gets in touch. Tickets about travel expenses have gone up too, and accounting is sending more reports back for corrections than before.

So was the redesign a success? Nobody in the room can say for sure. More page views could mean people enjoy using the module. They could just as well mean people are getting lost and clicking back and forth. The team measured something, just not what it actually wanted to know.

This article shows you which metrics actually capture the quality of a user experience, how to calculate them properly and where the typical thinking errors lie. The expenses module serves as a running example throughout. It is made up, but typical.

## Why UX metrics?

Qualitative [usability tests](https://www.insideuxdesign.com/en/glossar/usability-testing) show you where and why people struggle. What they do not tell you is whether a change paid off or whether the product is getting better over time. For that, you need numbers.

The ISO 9241-11 standard provides a framework. It describes [usability](https://www.insideuxdesign.com/en/glossar/usability) as the extent to which specified users can use a system to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. The classic metrics follow directly from these three terms: you measure effectiveness with success rate and errors, efficiency with time on task, and satisfaction with questionnaires such as the SUS or SEQ.

Metrics do not stand on their own, though. The Google team behind the HEART framework also stresses that they should be triangulated with findings from usability studies and field studies, and that they do not replace early [user research](https://www.insideuxdesign.com/en/glossar/user-research). Numbers tell you that something is happening. Why it is happening, you find out by observing and asking.

## Behavior and attitude: two kinds of data

Christian Rohrer of the Nielsen Norman Group classifies research methods along several dimensions, one of which is whether they capture behavior or attitudes. He sums up the difference neatly as “what people say” versus “what people do.” The two can be very different.

- **Behavioral metrics** are observed: Did someone complete the task? How long did it take? How many errors happened? How often does someone come back?
- **Attitudinal metrics** are asked: How easy did someone find the task? How do they rate the system overall? Would they recommend it?

You usually need both. The two perspectives are related, but they are not the same. Jeff Sauro, who has been collecting UX metric benchmarks with his company MeasuringU for many years, finds only a moderate correlation of around r = .5 between how difficult people rate a task and their time or success. If you only ask, you miss problems people did not even notice. If you only observe, you do not know how the experience felt.

## Success rate: the simplest and most important number

The success rate (also called task completion rate) tells you how many participants completed a task successfully. The math is simple: success counts as 1, failure as 0, and the average is the rate. If people cannot complete a task, almost every other metric is secondary.

The real work happens in preparation. Before the test, define what counts as success, concretely enough that two people would independently reach the same verdict. In the expenses module, the task reads: “Submit the hotel invoice for your business trip to Hamburg.” Success means the receipt is assigned to the right trip, the amount is correct and the report has been submitted. A receipt stuck in draft is not a success, even if the person believes they are done.

As a rough point of reference, Jeff Sauro analyzed data from 1,189 tasks from usability tests in 2011. The average success rate was 78 percent. Sauro also places this value at the 50th percentile: roughly half the tasks scored higher, half lower. Whether 78 percent is good for your task depends heavily on context, though. For a bank transfer in online banking, it is far too low.

### Confidence intervals for small samples

The team tests the launched version with twelve employees. 7 out of 12 complete the task, which is 58 percent. After a targeted rework, a second test shows 11 out of 12, or 92 percent. That sounds like clear progress. With samples this small, however, the observed value is only an estimate. The confidence interval shows how precise it is.

For success rates from small samples, Sauro and James R. Lewis recommend the adjusted Wald interval. It fixes the weakness of the simple formula, which becomes unreliable with small samples and, for example, produces an interval with no width at all when 10 out of 10 succeed. Free calculators are available online. For the example, at 95 percent confidence, you get:

- **Before:** 7 out of 12, interval of roughly 32 to 81 percent.
- **After:** 11 out of 12, interval of roughly 62 to 100 percent.

The intervals overlap. That does not automatically mean there is no difference, but it shows how uncertain the exact size of the effect is. So rather than “the success rate rose from 58 to 92 percent,” a more honest statement is: “The new version performs clearly better, and the true success rate is probably somewhere above 60 percent.”

## Time on task: why the average misleads

Time on task measures how long someone needs for a task. There is a trap in analyzing it, though. Task times are almost always positively skewed: most people take a similar amount of time, while a few take much longer, often because they ran into more problems. These outliers pull the arithmetic mean upward, so it no longer reflects the typical time.

Many people therefore switch to the median. At the 2010 CHI conference, Sauro and Lewis investigated whether that is the best choice for small samples. Using simulations, they repeatedly drew samples from 61 tasks with large datasets and compared different averages. The result: the median of small samples systematically overestimates the true median. The geometric mean had, on average, 13 percent less error and 22 percent less bias than the median. Their recommendation: for samples under 25 people, report the geometric mean. For larger samples, the median also estimates the center well.

The geometric mean sounds more complicated than it is. You take the logarithm of every time, average those values and transform the result back with the exponential function. In Excel or Google Sheets, you can use LN and EXP, or GEOMEAN directly. You also calculate the confidence interval on the log values and then transform the bounds back.

In the expenses example, the seven successful participants on the first version needed between 225 and 560 seconds. The arithmetic mean is 327 seconds, the median 290, and the geometric mean 313 seconds, with a confidence interval of roughly 235 to 416 seconds. After the rework, the geometric mean of the eleven successful participants is 162 seconds, with an interval of roughly 123 to 214 seconds. This time, the intervals do not overlap. Submitting is very likely genuinely faster.

Two more practical tips. First, decide which times you include. The most common approach is to report only the times of successful attempts, as in the example. Second, when you measure time, do not interrupt participants with questions in the middle of a task. According to Sauro, probing mid-task distorts the time, while asking afterwards yields more stable values.

## Error rate: exactly where things go wrong

Errors are the third classic behavioral metric and the most diagnostic one. A low success rate tells you a task is hard. The errors tell you where.

Sauro distinguishes two kinds: slips, where the goal is right but the execution is not, such as a typo in the amount. And mistakes, where the goal itself is wrong, for example when someone files the hotel invoice under “meals” instead of “accommodation.” As with success rate, define in advance what counts as an error.

The simplest way is to count in binary terms whether a person made at least one error. More precise is the average number of errors per task. If you relate errors to the number of opportunities, such as the number of input fields, you get an error rate as a percentage.

Error-free tasks are the exception, by the way. In an analysis of 719 tasks, mostly from consumer and business software, Sauro found an average of 0.7 errors per task. About two in three participants made at least one error, and only 10 percent of tasks were completely error-free. So if your test shows errors, that is normal. What matters is where they cluster. The article [Forms Nobody Abandons](https://www.insideuxdesign.com/en/blog/forms-nobody-abandons-12-rules-for-better-form-ux) shows how forms can help prevent errors.

## System Usability Scale: the classic questionnaire

The [System Usability Scale](https://www.insideuxdesign.com/en/glossar/system-usability-scale) (SUS) is probably the best-known questionnaire for perceived usability. John Brooke developed it in 1986 at Digital Equipment Corporation in the UK and published it in 1996 as a chapter titled “SUS: A ‘quick and dirty’ usability scale.” The name says it all: the SUS was meant to be quick to fill in and easy to score.

It consists of ten statements that alternate between positive and negative wording, for example whether you would like to use the system frequently or whether you found it unnecessarily complex. Participants rate each statement on a scale from 1 (“strongly disagree”) to 5 (“strongly agree”). The SUS is typically administered at the end of a test session, not after each individual task.

### How to calculate the SUS score

1. For the odd-numbered statements (1, 3, 5, 7, 9), subtract 1 from the response.
2. For the even-numbered statements (2, 4, 6, 8, 10), subtract the response from 5.
3. Add up the ten values and multiply the sum by 2.5.

The result ranges from 0 to 100. An example: someone answers the ten statements in order with 4, 2, 4, 1, 4, 2, 5, 2, 4 and 2. The odd statements give 3, 3, 3, 4 and 3, which is 16. The even ones give 3, 4, 3, 3 and 3, also 16. The total of 32 times 2.5 gives a SUS score of 80.

### How to read the SUS score

The most common SUS mistake is reading the score as a percentage. A SUS score of 70 does not mean “70 percent good,” nor does it mean a product beats 70 percent of all others. Sauro analyzed around 500 studies. The average was 68, which corresponds to the 50th percentile. A score of 74 sits at roughly the 70th percentile, and an “A” on Sauro’s grading scale takes more than 80.3 points. That is why it pays to translate SUS scores into percentiles for stakeholders.

In the expenses example, the first version averages 58 points, with a confidence interval of roughly 53 to 63. The reworked version reaches 74 points, with an interval of roughly 70 to 78. So the first version was clearly below average, and the reworked one is above it. According to Sauro, the SUS is reliable even with small samples. The estimate of the true score becomes imprecise, though, which is why a confidence interval belongs here as well.

## Single Ease Question: one question after every task

While the SUS captures the overall verdict, the Single Ease Question (SEQ) measures the difficulty of individual tasks. After each task, participants answer exactly one question, in Sauro and Lewis’s current version: “How easy or difficult was it to complete this task?” The scale has seven points, from 1 (“very difficult”) to 7 (“very easy”).

That a single question is enough has been well studied. At CHI 2009, Sauro and Joseph Dumas compared a seven-point version with two more elaborate methods. The simple question held its own against the more complex SMEQ scale and outperformed what is known as Usability Magnitude Estimation.

Across more than 400 tasks and 10,000 participants, Sauro reports an average of roughly 5.3 to 5.6, well above the scale midpoint of 4. An SEQ score of 4.5 is therefore rather weak, even if it sounds decent at first glance.

Sauro’s most useful practical tip: ask briefly why when someone gives a rating below 5. The reasoning immediately points you to the problem. In the expenses module, the SEQ score for the receipt task rises from 4.2 to 5.9. The explanations for the first version almost all revolve around the same spot: nobody can find how to assign a receipt to an existing trip.

If you are specifically interested in mental workload, the NASA Task Load Index (NASA-TLX) is a more detailed questionnaire. How it works and when it is worth using is covered in the article [Cognitive Load: Why Less Is Often More in Interface Design](https://www.insideuxdesign.com/en/blog/cognitive-load-why-less-is-often-more-in-interface-design).

## From the test to the product: the HEART framework

Success rate, time, SUS and SEQ come from usability testing. Many teams, however, also want to know how a product performs in day-to-day use. For that, Kerry Rodden, Hilary Hutchinson and Xin Fu from Google presented the HEART framework at CHI 2010.

The starting point was a critique of common product metrics, which the team called PULSE: Page views, Uptime, Latency, Seven-day active users and Earnings. These numbers matter, but they measure user experience only indirectly. The team gives an example that fits the expenses module perfectly: if page views for a feature go up, that may mean the feature is popular. But it may also mean a confusing interface has people wandering around lost.

HEART stands for five categories:

- **Happiness:** attitudinal metrics such as satisfaction, perceived ease of use or likelihood to recommend, usually collected through surveys.
- **Engagement:** how intensively people use a product, such as visits per person per week. The team recommends reporting such values per person rather than as a total.
- **Adoption:** how many people start using a product or feature within a given period.
- **Retention:** how many people from a given period are still using it later.
- **Task success:** the classic behavioral metrics, meaning effectiveness, efficiency and error rate, also at scale.

Not every category fits every product. The Google team explicitly mentions enterprise software as an example where engagement says little, because people are expected to use it as part of their work. That is exactly the case for the expenses module. Nobody is supposed to spend as much time as possible in it.

### Goals, signals, metrics

Perhaps the more valuable part of the paper is a simple three-step process. First, you define the goals: what should the redesign achieve? Then the signals: how would you recognize in behavior or attitudes that the goal has been met? Only at the end do you formulate concrete metrics. Two of the team’s tips are especially useful: failure is often easier to spot than success, for example through abandoned tasks. And raw counts rise as your user base grows, so ratios and per-person values are usually more meaningful.

For the expenses module, it looks like this:

- **Task success:** The goal is for expense reports to be correct on the first attempt. Signal: accounting has to send fewer back. Metric: share of reports per month that need no follow-up.
- **Adoption:** The goal is for employees to capture receipts directly with their phones. Signal: the camera feature gets used. Metric: share of active employees who upload at least one receipt photo per month.
- **Retention:** The goal is for photographing receipts to become a habit. Metric: share of those who are still capturing receipts by photo three months later.
- **Happiness:** The goal is for the expense process to feel easy. Metric: average SEQ score from a short question right after submission.

Page views no longer appear on this list. Support tickets, on the other hand, become a useful signal for task success.

## Small samples, honest numbers

How many participants do you need for reliable numbers? The Nielsen Norman Group recommends 40 participants for quantitative studies in most cases. That puts the margin of error for a success rate at about 15 percentage points at a 95 percent confidence level. If you can live with 90 percent confidence, NN/g says you can get by with fewer people. For time measurements at the same precision, NN/g estimates more like 47 participants.

Measurements with twelve people are still not worthless. They reveal large differences, like the time on task in the example, but not small ones. What matters is reporting the uncertainty openly. Giving confidence intervals keeps a jump from 68 to 71 SUS points with ten people from being celebrated as a breakthrough. Sauro and Lewis describe suitable tests for comparing two versions in their book “Quantifying the User Experience.” For how to get usable insights from tests even without statistics, see [How to Extract Valuable Feedback from Usability Tests](https://www.insideuxdesign.com/en/blog/how-to-extract-valuable-feedback-from-usability-tests).

## Common pitfalls

**Vanity metrics.** In a widely cited 2009 guest post, entrepreneur Eric Ries described so-called “vanity metrics”: numbers that make you feel good but do not offer clear guidance for what to do next. His example was total hits on a website. Total sign-ups or downloads are typical candidates too. They often rise simply because a product is growing. A good metric, by contrast, changes a decision when it changes.

**Comparing apples and oranges.** Metrics are only comparable within the same task. If the receipt task reaches an SEQ score of 5.9 and the task “claim mileage for a trip in your own car” only 4.8, that does not mean the second one is designed worse. It may simply be harder because it asks for more information. The same goes for times from different test formats: Sauro points out that lab times are idealized and remote times are affected by distractions. Comparisons made with the same method are the most meaningful.

**Changing the measurement midway.** If you change the task wording, the success criterion or the questionnaire version between two test rounds, you end up comparing measurements rather than designs. Keep tasks, criteria and questionnaires stable across rounds and document them.

**Measuring without a goal.** A dashboard with 40 metrics helps no one. It is no accident that the HEART process starts with goals. If you cannot say which decision depends on a number, you probably do not need it.

## Conclusion

UX can be measured, and with manageable effort. Success rate tells you whether people reach their goal. Time on task, reported as the geometric mean for small samples, shows how efficiently they do it. Errors show where things go wrong. SUS and SEQ add people’s own view to the picture. And HEART helps you derive metrics for the live product, starting from goals rather than from whatever data happens to be available.

The team behind the expenses module can now answer its question. The rise in page views was not a success but a warning sign: many people were searching the first version for a way to assign receipts to trips. After the rework, the success rate is clearly higher, submitting takes just under three minutes instead of just over five on average, and the SUS score is above average. Whether accounting now sends back fewer reports is tracked by a metric on the dashboard. Without the right metrics, the team would have celebrated the wrong number.

## Checklist

- For every measurement, it is clear which decision depends on it.
- Behavioral and attitudinal metrics are combined.
- Tasks and success criteria are written down before the test.
- Success rates are reported with a confidence interval, using the adjusted Wald interval for small samples.
- With fewer than 25 participants, task times are reported as the geometric mean, not the arithmetic mean.
- It is defined whether only successful attempts count toward times.
- What counts as an error is defined, and errors are mapped to their place in the flow.
- The SUS is collected at the end of the session, scored correctly and explained as a percentile, not a percentage.
- The SEQ follows every task directly, and ratings below 5 prompt a question about why.
- Comparisons are made only within the same task and with the same method.
- Product metrics are derived via goals, signals and metrics, not copied from existing dashboards.
- Raw totals are replaced with ratios or per-person values.

## Sources

- [Christian Rohrer, Nielsen Norman Group (2022): When to Use Which User-Experience Research Methods](https://www.nngroup.com/articles/which-ux-research-methods/)
- [Jeff Sauro, MeasuringU (2011): What Is A Good Task-Completion Rate?](https://measuringu.com/task-completion/)
- [Jeff Sauro, MeasuringU: Confidence Interval Calculator for a Completion Rate](https://measuringu.com/wald/)
- [Jeff Sauro and James R. Lewis (2010): Average Task Times in Usability Tests: What to Report? Proceedings of CHI 2010, pp. 2347 to 2350](https://doi.org/10.1145/1753326.1753679)
- [Jeff Sauro, MeasuringU (2011): 10 Things To Know About Task Times](https://measuringu.com/task-times/)
- [Jeff Sauro, MeasuringU (2012): Measuring Errors in the User Experience](https://measuringu.com/errors-ux/)
- [Jeff Sauro, MeasuringU (2012): 10 Benchmarks for User Experience Metrics](https://measuringu.com/ux-benchmarks/)
- [John Brooke (1996): SUS: A “quick and dirty” usability scale. In: P. W. Jordan et al. (eds.): Usability Evaluation in Industry. Taylor & Francis](https://doi.org/10.1201/9781498710411-35)
- [Jeff Sauro, MeasuringU (2011): Measuring Usability with the System Usability Scale (SUS)](https://measuringu.com/sus/)
- [Jeff Sauro, MeasuringU (2012): 10 Things To Know About The Single Ease Question (SEQ)](https://measuringu.com/seq10/)
- [Jeff Sauro and Jim Lewis, MeasuringU (2024): The Evolution of the Single Ease Question (SEQ)](https://measuringu.com/evolution-of-seq)
- [Jeff Sauro and Joseph S. Dumas (2009): Comparison of Three One-Question, Post-Task Usability Questionnaires. Proceedings of CHI 2009, pp. 1599 to 1608](https://doi.org/10.1145/1518701.1518946)
- [Kerry Rodden, Hilary Hutchinson and Xin Fu (2010): Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications. Proceedings of CHI 2010, pp. 2395 to 2398](https://doi.org/10.1145/1753326.1753687)
- [Raluca Budiu and Kate Moran, Nielsen Norman Group (2021): How Many Participants for Quantitative Usability Studies: A Summary of Sample-Size Recommendations](https://www.nngroup.com/articles/summary-quant-sample-sizes/)
- [Eric Ries (2009): Vanity Metrics vs. Actionable Metrics. Guest post on Tim Ferriss’s blog](https://tim.blog/2009/05/19/vanity-metrics-vs-actionable-metrics/)
- [Jeff Sauro and James R. Lewis (2016): Standardized Usability Questionnaires. In: Quantifying the User Experience, 2nd edition. Morgan Kaufmann (Elsevier), pp. 185 to 248](https://doi.org/10.1016/B978-0-12-802308-2.00008-4)