Synthetic Users: Can AI Replace Real User Research?

“User research. Without the users.” That was the promise the start-up Synthetic Users made on its website. Research without recruiting, without scheduling, without incentives: you describe your target group, and an AI delivers finished interviews within seconds. Sounds too good to be true? That is exactly what this article is about.

Synthetic users are one of the most controversial AI topics in UX design. Some see them as the shortcut everyone has been waiting for, others as the beginning of the end of real user research. Time for a sober look: what are synthetic users, what do comparative studies show, and where can they still be useful?

What are synthetic users?

A synthetic user is an AI-generated profile meant to imitate a specific user group. The Nielsen Norman Group (NN/g) describes it as an artificial profile that expresses simulated thoughts, needs and experiences without a single real person ever being asked. You can “interview” it, give it surveys or show it a concept, and it answers as if it were a person with a story of its own.

Under the hood is a large language model (LLM) such as ChatGPT or Claude. These models were trained on huge amounts of text and can therefore convincingly phrase how a nurse, a tax adviser or a shop owner probably thinks. A simple prompt along the lines of “Imagine you are …” is enough. There are also specialised tools that automatically generate entire interview series including a summary.

At first glance, synthetic users resemble classic personas. The crucial difference: a good persona summarises insights from real research. A synthetic user is based on probabilities, meaning on which answer statistically fits the question best.

Why the idea is so tempting

Anyone who has ever planned user research knows the hurdles: finding participants, scheduling sessions, paying incentives, analysing conversations. That costs time and money, two things that are scarce in small teams and early project phases.

Synthetic users promise to remove exactly these hurdles. They are always available, never get tired and answer every question at length. And that is exactly where the problem starts.

What the research says

There are now several studies that directly compare synthetic and real participants. Three of them show particularly well where the limits lie.

Nielsen Norman Group: too shallow, too positive

NN/g, one of the best-known UX consultancies in the world, generated synthetic users for three interview studies it had already completed, once with ChatGPT and once with the tool Synthetic Users. The result was sobering.

In a study on online courses, real participants openly admitted that they had abandoned many courses, for example because a new job left them no time. The synthetic users, by contrast, claimed to have completed every course. Discussion forums that real learners found contrived and unhelpful were rated very positively by the synthetic users. And when the team presented a rather adventurous idea, delivering product samples by drone, it received enthusiastic approval.

MeasuringU: too good to be true

Jeff Sauro’s team had ChatGPT complete a tree test. This method checks whether people can find content in a navigation structure. In the experiment, it was the menu structure of the US tax authority. ChatGPT found almost every target, while the 33 human participants solved only about half of the tasks on average. What sounds like a good result is worthless for research: if the test participant solves every task, the test reveals no weaknesses in the information architecture.

One side finding is interesting: ChatGPT was fairly good at estimating how difficult people would find a task. As a substitute for real behaviour, however, the model was still not suitable.

First clicks: newer approaches barely help

A 2026 study by a research team led by Eduard Kuric compared twelve first-click tests from real projects with a total of 3,431 participants against simulated results from GPT. In 53 percent of the tasks, the AI’s click distribution differed significantly from the real one. The reasoning the model gave for its clicks did not match the thought patterns of real participants either.

Particularly revealing: neither personas nor step-by-step reasoning by the model (chain of thought) nor different settings improved the results noticeably. They mainly made the answers more believable, but not more accurate.

The four core problems of synthetic users

1. They want to please

Language models are trained to be helpful and agreeable. Experts call this tendency sycophancy. For a chatbot, that is often convenient. For user research, it is fatal. Real people tell you when an idea makes no sense. Synthetic users like almost everything.

2. Everything matters to them

Ask a synthetic user what matters to them about a product, and you get a long, tidy list. In the NN/g study, a synthetic user listed seven factors when asked what makes a good online course. Real people, on the other hand, have clear priorities: one thing annoys them enormously, three others they don’t care about. A team needs exactly this weighting to decide what to work on next.

3. They have experienced nothing

A synthetic user has never used your product, never waited on hold, never abandoned a form in frustration. It can only describe what such experiences typically sound like. Behaviour cannot be observed this way, and behaviour often reveals more than what people say about themselves.

4. They only know what is already known

At its core, a language model reproduces the average of what has been written about a group. For broad target groups, that often sounds plausible. For niches, such as a specialised industry or a regional market, it quickly becomes patchy. And the most valuable insights from user research are usually the ones nobody expected. Those are exactly what a synthetic user cannot deliver.

Where synthetic users can still help

Synthetic users are not completely useless. Used correctly, they make it easier to prepare real research. NN/g sees sensible use cases here, too:

Getting into a new topic: If you design for an unfamiliar industry, you quickly get a first overview of typical processes and terminology, similar to desk research.
Testing an interview guide: Before the first real conversation, you can check whether the questions are understandable and the conversation flows sensibly.
Collecting hypotheses: Synthetic answers provide ideas for topics to raise in real interviews. As hypotheses, not as results.
Proto-personas: As a first draft that gets revised after real research, AI-generated profiles can offer a starting point.

In short: synthetic users help you ask better questions. The answers come from real people.

How to use synthetic users responsibly

Label the source: Never present synthetic results as real research. Every insight states where it came from.
Treat everything as a hypothesis: Every statement from a synthetic user is checked with real people before it influences a decision.
Stay away from niches: The more specialised the target group, the less reliable the answers.
No concept tests: Whether an idea will land is not something an AI that likes almost everything can tell you.
Do the honesty test: Before every use, ask: would the team act on these answers without checking them? If so, it is replacing research, and that is exactly what it should not do.

Better than no research at all?

The most interesting question is not really whether synthetic users can replace real ones. They can’t. More interesting is whether they are better than no research at all. At first glance, yes: learning something about your target group sounds better than nothing.

The problem: part of what you learn is wrong, and without real research you won’t notice which part. NN/g also warns of a habituation effect. Once a team has experienced how fast and cheap “research” can be, it becomes hard to justify a budget for real interviews. Synthetic users feel like research without being research.

Real user research doesn’t have to be elaborate. A few short conversations with people from the target group often deliver more than a hundred synthetic interviews. That pragmatic attitude is what Lean UX is about: learn fast, but from real people. How that works without a dedicated research department is shown in the book Lean UX für kleine Teams (in German).

Conclusion

AI is a powerful tool in the UX process: for researching, structuring, summarising and preparing. It does not replace real user research. Synthetic users are too positive, too superficial and too predictable to show what people really need. Use them to ask better questions. You’ll find the answers with real people. What this work involves is described in the article What Does a UX Researcher Do?