AI has changed the economics of UX analysis. What used to take a research team a week (screening screens against heuristics, combing through session recordings, benchmarking competitors) can now be drafted in an afternoon. Drafted, not finished. That distinction is the whole point of this guide.

At Things, a design and engineering studio founded in 2018 with roots in Istanbul and Elazığ, we've shipped more than 50 products and run UX research on most of them. We use AI in nearly every audit we deliver today. We also know exactly where it breaks. Here's how to run AI-assisted UX and UI analysis properly, and how to avoid the traps.

What Is AI-Assisted UX Analysis?

AI-assisted UX analysis is the practice of using large language models and vision models to evaluate interfaces, flows, and behavioral data against established usability principles. The AI acts as a fast, tireless first reviewer: it flags issues, categorizes them, and drafts findings. A human expert then validates, prioritizes, and contextualizes.

It is not a replacement for research. It is a multiplier on top of it.

In practice, AI-assisted analysis covers five workstreams:

  1. Heuristic evaluation: screening UI against principles like Nielsen's ten heuristics.
  2. Accessibility review: checking contrast, structure, labels, and target sizes against WCAG.
  3. Analytics and session analysis: summarizing funnels, drop-offs, and behavioral patterns.
  4. Competitive UI analysis: comparing your flows against competitors, screen by screen.
  5. Copy and content review: evaluating microcopy, error messages, and information scent.

The output of all five is the same thing a classic UX audit produces: a prioritized list of usability issues with evidence and recommendations. AI just gets you to a credible first draft dramatically faster, which means the expensive human hours go into judgment, not clerical work.

How Do You Run an AI-Assisted Heuristic Evaluation?

You run it by feeding screenshots of key flows to a vision-capable LLM, prompting it to evaluate each screen against Nielsen's ten usability heuristics, and requiring structured output: heuristic violated, severity, evidence, recommendation. Then an experienced reviewer verifies every finding before it enters the report.

Here's the workflow we use at Things:

1. Scope the flows, not the whole product. Pick the three to five journeys that matter commercially: onboarding, checkout, core task completion. Capture every state: empty, loading, error, success. Most heuristic violations hide in edge states, and most teams forget to screenshot them.

2. Give the model the heuristics explicitly. Don't assume the model will apply Nielsen rigorously on its own. Paste the ten heuristics (visibility of system status, match between system and real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, help users recognize and recover from errors, help and documentation) and instruct it to cite the specific heuristic for every issue.

3. Provide context. Tell the model who the user is, what the task is, and what happened on the previous screen. A screenshot without context invites confident nonsense. "This is step 3 of a B2B invoicing flow; the user has just selected a client" produces far better findings than a naked image.

4. Demand structured, evidence-based output. Require a severity rating (we use a 0–4 scale, cosmetic to catastrophic), the exact UI element in question, and a concrete fix. Ban vague findings like "improve clarity."

5. Run it more than once. LLMs are non-deterministic. Two or three passes surface different issues, much like multiple human evaluators do in classic heuristic evaluation. Merge and deduplicate.

6. Verify everything. This is non-negotiable. An expert reviewer confirms each finding against the live product, kills the false positives, adjusts severities, and adds the issues the model missed. In our experience the model is a strong breadth tool and a weak depth tool: it catches many surface violations quickly, and misses the systemic ones (inconsistent mental models across a flow, misaligned information architecture) almost every time.

Can AI Handle Accessibility Audits?

AI can accelerate accessibility checks but cannot certify accessibility. Automated tooling and LLMs together catch a meaningful share of WCAG issues (contrast failures, missing labels, tiny touch targets, ambiguous link text) while keyboard traps, screen-reader logic, and cognitive accessibility still demand human testing.

A sensible AI-augmented accessibility pass looks like this:

  • Automated scanners first. Run axe, Lighthouse, or equivalent on live code. These are deterministic and cheap.
  • Vision-model review second. LLMs are surprisingly good at flagging low-contrast text in screenshots, icon-only buttons without visible labels, and dense tap targets on mobile layouts.
  • Code-level LLM review third. Feed component markup to the model and ask it to check semantic structure, ARIA usage, focus order, and form labeling.
  • Human assistive-technology testing last. Navigate the product with a keyboard only. Use a screen reader. No model replaces this.

Treat AI as the tool that makes sure your human accessibility hours are spent on the hard problems, not on counting contrast ratios.

How Do You Analyze Analytics and Session Data with AI?

You export funnel data, event streams, or session summaries and ask the model to find patterns, anomalies, and hypotheses. Then you validate those hypotheses against recordings and, ideally, real users. AI turns behavioral data from something you report into something you interrogate.

Concretely:

  • Funnel narration. Give the model step-by-step conversion data and ask where the story breaks. It's good at spotting the step whose drop-off is out of proportion and articulating candidate explanations.
  • Session pattern mining. Summaries of rage clicks, dead clicks, back-and-forth loops, and abandonment points can be clustered and labeled by an LLM far faster than by hand.
  • Segment comparison. Ask the model to contrast behavior across devices, cohorts, or acquisition channels and flag divergences worth investigating.
  • Hypothesis generation. The most valuable output isn't a conclusion. It's a ranked list of testable hypotheses: "Users may be abandoning at the address form because of X; validate with a five-user test."

One rule: AI explains what might be happening. Only research explains why. Every analytics insight is a hypothesis until a user confirms it.

How Does Competitive UI Analysis Work with AI?

You capture equivalent flows from three to five competitors, feed the screenshots to a vision model alongside your own, and ask for a structured comparison: patterns, conventions, differentiators, and gaps. What used to be a week of manual benchmarking becomes a day of capture plus a few hours of synthesis.

Useful prompts to structure the comparison:

  • Where does our flow require more steps or more decisions than the category norm?
  • Which conventions do all competitors share that we violate, and is the violation deliberate?
  • Where is every competitor identical, leaving room to differentiate?
  • How do onboarding, pricing pages, and empty states compare in information density and hierarchy?

The trap here is convergence. AI will happily recommend that you look like everyone else, because "consistency and standards" is baked into its training. Sometimes the right move is to break the pattern deliberately. That's a strategy call, and strategy is a human job.

What Are the Limits of AI UX Analysis?

AI analysis has three structural weaknesses: context blindness, false confidence, and the absence of real users. Any audit that ignores these limits will look rigorous and be wrong.

Context blindness. The model sees the screen, not the business. It doesn't know that the "redundant" confirmation step exists for regulatory reasons, that your users are field technicians wearing gloves, or that the awkward flow is a deliberate trade-off shipped under a deadline. Without context, it optimizes for a generic user who doesn't exist.

False confidence. LLMs deliver wrong findings in the same fluent, authoritative tone as right ones. They hallucinate UI elements that aren't in the screenshot. They cite heuristics that don't apply. A team without evaluation experience cannot tell the difference, which is precisely why AI audits are most dangerous in the hands of people who need them most.

No real users. This is the hard limit. AI predicts plausible behavior; it does not observe actual behavior. It cannot be surprised. It cannot hesitate, misread a label, or tell you the feature solves a problem nobody has. Every genuinely transformative insight we've found in eight years of research came from watching a real person do something no one predicted.

Why Does Expert Review Still Matter?

Because heuristic evaluation was never a checklist. It's judgment applied through a checklist. Nielsen's own research showed that evaluator expertise dramatically changes what gets found. AI raises the floor of an audit; expertise raises the ceiling.

An expert reviewer does four things a model can't:

  • Prioritizes against business reality. Severity in a vacuum is meaningless. Severity against your roadmap, your segment, and your unit economics is a plan.
  • Sees systems, not screens. Design-system inconsistencies, mental-model breaks across a journey, information architecture debt: these live between screenshots.
  • Knows what's missing. A model critiques what it sees. An expert notices the feature, state, or safeguard that should exist and doesn't.
  • Takes responsibility. A report someone stakes their reputation on is a different artifact than a generated document.

This is why we call Things a studio, not an agency. Agencies scale by adding process. A studio scales by deepening craft, and AI, used well, is a craft tool. Our founders' shelves hold Red Dot, Cannes Lions, and Effie awards; none of that came from accepting a first draft, human or machine.

How Does Things Combine AI Analysis with Real Usability Testing?

We run AI-assisted analysis as the first movement of a larger research process (Understand, Shape, Build, Evolve), and we validate every machine finding with expert review and real users before anyone acts on it.

In practice, an audit at Things looks like this:

  1. Understand. AI-assisted heuristic evaluation, accessibility scan, and analytics review produce a broad findings draft fast. Expert reviewers verify, kill false positives, and add the systemic issues models miss.
  2. Validate. Moderated usability tests and interviews with real users confirm which findings actually hurt, and reveal the ones no model predicted.
  3. Shape and Build. Findings become prioritized design decisions, then shipped interfaces, because we design and engineer under one roof.
  4. Evolve. Post-launch data closes the loop and feeds the next cycle.

AI makes the audit faster. Research makes it true. Craft makes it matter.

FAQ

Can AI fully replace a UX audit? No. AI accelerates the evaluation phase and broadens coverage, but findings require expert verification, and behavioral truth requires real users. AI-only audits produce confident reports with unvetted findings.

Which AI tools work for heuristic evaluation? Any vision-capable large language model can evaluate screenshots against Nielsen's heuristics, and models like Claude or GPT-4-class systems handle it well. The tool matters less than the method: explicit heuristics, real context, structured output, multiple passes, human verification.

How many screenshots should I include in an AI heuristic review? Cover three to five critical flows end to end, including empty, loading, and error states: typically 20 to 60 screens. Full-product screen dumps dilute attention; scoped flows produce sharper findings.

Is AI-generated accessibility analysis WCAG-compliant proof? No. AI and automated scanners catch a subset of WCAG issues. Compliance claims require human testing with keyboards and screen readers, and ideally testing with disabled users.

How long does an AI-assisted UX audit take? Typically one to two weeks including expert verification, versus three to six for a fully manual audit, with usability testing added when findings need behavioral validation.

Ready for an Audit That's Fast and True?

If your product needs a UX audit that combines AI speed with research rigor (heuristic evaluation, accessibility review, usability testing, and the engineering to ship the fixes), talk to us. Design. Code. Mastery.

Write to hello@things.ist and tell us what you're building.

Things, a studio, not an agency. Istanbul & Elazığ, since 2018.