What can an automated UX audit really catch?

Most UX audit tools tell you what they found. Very few tell you what they couldn't check. That gap matters, because a clean report from a tool that skipped half the problems can be worse than no report at all.

This guide maps exactly where automation is strong, where it needs a real browser, and where it still needs a person. It draws on Deque's study of nearly 300,000 accessibility issues, Baymard's tests of AI auditors, and the WebAIM Million 2026 report. We then show our own coverage map, rule by rule.

The evidence at a glance

How much can automation actually catch?

It depends on what you count. There are two honest answers, and they differ by more than double.

Counting rules: 20–30%. For years, the accepted figure was that automated testing covers only 20–30% of accessibility requirements. That figure counts WCAG success criteria: of all the rules in the standard, only a minority can be fully tested by software.

Counting real issues: 57%. In March 2021, Deque analysed over 2,000 audits covering more than 13,000 first-time page assessments and nearly 300,000 issues. Their automated testing fully covered 57% of the issues found, on average (Deque, 2021).

Both numbers are true. The gap exists because some problems are far more common than others. A missing label or a low-contrast button appears dozens of times on a page, and software catches every one. A confusing checkout flow appears once, and software misses it.

The WebAIM Million 2026 shows how lopsided this is. Across one million home pages, 96% of all detected errors fell into just six categories (WebAIM, 2026):

Every one of these is machine-detectable. That is why automation covers so much of what actually goes wrong, even though it covers a minority of the rules.

A second large dataset agrees. The 2025 Web Almanac, which runs Lighthouse (powered by Deque's axe-core) across millions of sites, found only about 30% of sites pass minimum colour contrast, and that 53% of desktop and 55% of mobile inputs rely only on placeholder text for their label (HTTP Archive, 2025). Both problems are cheap to detect and cheap to fix, which is exactly where automation earns its keep.

What this means for you: When a tool quotes a coverage figure, ask which one it means. "57% of issues" and "30% of rules" describe the same tool.

There's also a ceiling per tool. The Web Almanac's authors cite a GOV.UK comparison of popular automated accessibility tools: every one detected fewer than 50% of known errors. They add that a perfect Lighthouse score doesn't guarantee full accessibility and should be treated as a starting point (HTTP Archive, 2025).

What do automated audits catch reliably?

Automation is strongest where a rule has a clear, measurable pass or fail. If you can write the test as a number or a yes/no check, software can run it on every element of every page, faster and more consistently than a person.

These checks have two things in common. The answer doesn't depend on your audience or brand, and the same input always gives the same result. That is what makes them safe to automate.

One caution: "automatable" does not mean every tool does it well. Deque notes that its rules library puts heavy emphasis on not reporting false positives (Deque, 2021). A tool that floods you with false alarms is technically automated but practically useless.

What can only be caught in a real browser?

Many UX problems only exist when someone uses the page. A screenshot or a code scan cannot see them.

Baymard tested this directly in 2023. They gave GPT-4 screenshots of 12 ecommerce pages and compared its findings with six trained UX benchmarkers who used the live sites. GPT-4 found 26% of the issues visible in the screenshot, but only 14% of the issues actually present on the live page (Baymard, 2023). Baymard's explanation: a large share of UX issues are interactive, and can't be judged from an image.

Here is what a real browser can test that a screenshot cannot:

• Hover and focus states. Does a button change when you hover or tab to it? Is the keyboard focus ring visible at all?
• Focus hidden behind sticky elements. WCAG 2.2 added criterion 2.4.11 (Focus Not Obscured). It fails when a sticky header or cookie banner covers the element you've tabbed to. You can only find this by tabbing through the page.
• Layout shift. A banner or font that loads late and pushes content down is invisible in a finished screenshot. It only shows up while the page is loading.
• Interaction speed. INP measures how long the page takes to respond after a tap or click. It needs a real interaction to measure.
• Mobile-only problems. Menus that only open on hover fail on touch screens. Elements that overflow at a 390px width don't appear in a desktop capture.
• Keyboard traps. A modal or menu that you can tab into but not out of.

These aren't edge cases. In 2025, 67% of sites removed the browser's default focus outline, and only about 25% used :focus-visible (HTTP Archive, 2025). On mobile, 62% of pages had at least one image without set dimensions, a leading cause of layout shift, and 40% ran non-composited animations (HTTP Archive, 2025). Whether these actually hurt users can only be confirmed by loading and using the page.

Why this matters for tool choice: If a tool works from a screenshot or a static code scan, it can't test any of the items above. Ask any audit tool whether it loads and interacts with your live page.

What still needs a human?

Some UX questions have no fixed right answer. They depend on who your users are, what your brand stands for, and what you want people to do. These still need a person.

• Brand fit. Does the design feel like your company? Software can measure contrast, not personality.
• Copy tone and clarity. Is the headline clear to your audience? Does the microcopy sound like you? Reading level can be scored, but meaning can't.
• Task flows. Can a first-time user sign up, find pricing, or complete checkout without getting lost? This depends on goals that span several pages.
• Content accuracy. Are prices, claims and product details correct?
• Context-dependent choices. Is infinite scroll right for this product list? Should this form be one step or three? The answer depends on the use case.

Why not just ask AI to judge these?

Because the evidence says it isn't reliable yet. In Baymard's 2023 test, GPT-4 had a 19.9% accuracy rate and an 80.1% false-positive error rate. On average per page, it correctly found 2.9 issues, overlooked 18.5, made 1.3 suggestions likely to harm UX, and 10.6 that were a waste of time (Baymard, 2023).

Tools have improved since then, but not enough. In March 2025, two Microsoft UX researchers tested AI tools on heuristic evaluations and found accuracy rates of 50%, 62%, 67% and 75%. The 75% tool only reached that score by reporting far fewer issues. It missed 13 of the 16 problems the human expert found (Baymard, 2026).

The real risk is that the wrong suggestions look just as confident as the right ones. Baymard gives an example: GPT-4 suggested Overstock replace pagination with infinite scrolling, a pattern their research shows hurts usability (Baymard, 2023). If you can't tell which suggestions are wrong, you can't safely act on any of them.

The lesson: Use automation for checks with clear right answers. Flag judgement calls for a human instead of letting a model guess.

The ProCurveUX coverage map

ProCurveUX checks every site against 120+ rules drawn from UX laws, Nielsen's 10 usability heuristics, visual design principles and WCAG 2.2 AA. About two-thirds of those rules are tested automatically. The rest are flagged for manual review, and every report says which is which.

We split the work into three layers:

Layers 1 and 2 together make up roughly two-thirds of our rules. Layer 3 is the remaining third.

Three design choices follow from this map:

1. We don't guess at Layer 3. We would rather mark a rule "needs manual review" than produce a confident answer that might be wrong.
2. Every finding shows its evidence. Each issue comes with a screenshot, the CSS selector and the pages it appears on. You can verify it in seconds instead of trusting it blindly.
3. Findings are ranked by your goal. You choose sign-ups, demo bookings, checkout or support deflection, and each finding is weighted by how much it blocks that goal.

We publish this split because a report is only as useful as your understanding of what it left out.

What should a site owner do?

A simple routine gets you most of the value of a full audit at a fraction of the cost.

1. Run an automated audit on your key pages. Start with the homepage, pricing and sign-up or checkout. Fix the measured issues first; they are cheap, certain and often widespread.
2. Check it in a real browser on mobile. Use a tool that tests at a phone width and tabs through the page, or do it yourself: put your phone in your hand and try to complete your main task.
3. Do the 20-minute human review. Go through brand fit, copy clarity and your main task flow. Better still, watch one or two real users try it.
4. Fix by impact, not by count. One weak call-to-action on your pricing page costs more than ten minor contrast issues in the footer. Rank fixes by the goal you care about.
5. Re-audit after every release. New features bring new problems. A quick re-run tells you what you fixed, what broke and what's new.
6. Bring in an expert for high-stakes changes. For a redesign, a checkout rebuild or a compliance review, pair automated results with an experienced designer or accessibility specialist.

Key takeaways

• Automated coverage depends on what you count. Deque found automation covers about 57% of real accessibility issues, but only 20–30% of WCAG rules.
• Six error types, all machine-detectable, account for 96% of detected accessibility errors on the web's top million home pages (WebAIM, 2026).
• Screenshots miss interaction problems. GPT-4 found 26% of issues in a screenshot but only 14% of those on the live page (Baymard, 2023).
• AI is not yet reliable for judgement calls. Published tests show accuracy of 20% (GPT-4, 2023) to 50–75% (AI tools, 2025).
• ProCurveUX automates about two-thirds of its 120+ rules in a real browser and flags the remaining third for human review.
• The safest audit combines automated checks, real-browser interaction testing and a short human review.

Frequently asked questions