Accessibility automated website scans and how much can they catch

Automated accessibility testing tools have gained traction due to their ability to quickly analyze web pages for compliance with accessibility standards, such as the Web Content Accessibility Guidelines (WCAG). These tools can scan a website’s code and identify potential issues, making them an attractive option for developers and organizations looking to improve accessibility.

Introduction

Automated website scans offer a fast and efficient way to assess web accessibility, making them ideal for integration into CI/CD pipelines. These tools provide consistent results by validating against a predefined set of rules, ensuring that no accessibility cases are overlooked.

However, they may generate false positives due to the complexity of web code and content, and they struggle with nuanced judgments, such as evaluating the relevance of alt text descriptions. In practice, automated tests cannot cover every WCAG requirement because of current technological limitations, leaving some failure patterns that commonly appear in lawsuits undetected.

While automated testing is a valuable component of a comprehensive accessibility strategy, it should be complemented by manual testing and real-user testing, including input from individuals with disabilities, to ensure a truly inclusive digital experience across devices and assistive technologies.

What percentage of accessibility issues can automated tools catch?

WCAG 2.2 Level AAA defines 87 Success Criteria (SC), but only a subset can be translated into deterministic, machine-testable rules suitable for automation.

Research that maps WCAG 2.2 SC to automated rules puts the theoretical ceiling at 20%-40% of all possible SC failures. That figure is not fixed and it shrinks or grows depending on how many of the 87 Success Criteria (SC) are actually present on the page under test.

For instance, a static marketing site with no multimedia or complex forms might expose only 30 relevant SCs, while a transactional SaaS dashboard could surface 60 or more.

Because automation can only validate criteria that lend themselves to binary pass/fail logic, such as color contrast, missing alt attributes, or duplicate IDs, the real-world catch rate is always a fraction of the applicable Success Criteria, not a universal percentage. In other words, the practical catch rate depends on the scope of features present in the tested instance, not the full WCAG catalog.

What automated tools can catch

Think of an automated accessibility checker as a spell-checker for code: it scans for clear-cut mistakes it can quantify with confidence.

• Missing alternative text on images.
• Text that doesn’t meet the minimum color contrast ratio but only when foreground and background colors can be reliably identified. If the text sits over a background image, gradient, semi-transparent overlay, or sticky/fixed layer, the tool may report contrast can’t be determined.
• Form fields lacking associated labels.
• Improper heading structure (e.g., an h3 directly following an h1).
• Duplicate element IDs, or interactive elements like buttons and links that are empty or unlabeled.

If an issue can be expressed as a binary yes/no rule, the tool can flag it instantly and across every page.

The major advantage: the automated audit never gets tired. It tests every rule, on every element, every time, so nothing that can be detected is ever skipped. This makes results predictable, repeatable, and easy to integrate into a build pipeline.

Why manual testing is non-negotiable

Automated scans end where human experience begins. Code can flag a missing <label>, but it can’t determine whether the visible text beside a control is intelligible, or whether a keyboard trap appears only when the viewport is zoomed to 200%. The following three mini-case studies illustrate the gap in action.

1. Perfect contrast that fails in sunlight.

   A fintech dashboard scored 100% on an automated color-contrast audit. In user testing, two low-vision participants could not read error messages on a glossy tablet outdoors because the automated test only sampled the CSS colour values, not the semi-transparent red overlay that JavaScript injected on submit. A five-minute manual check caught the issue and the fix was one line of Sass.

2. Keyboard order that logic can’t see

   A travel-booking flow passed the automated test for the tab order rule. However, manual keyboard navigation revealed a calendar widget that sent focus to a hidden close button in the DOM, stranding keyboard users three modal layers away from the main content. No rule in the engine recognized that the widget was visually hidden but still focusable.

3. Alt text that is present and meaningless

   An e-commerce gallery included alt attributes on every product image: img-1, img-2, img-3, etc. The automated report passed, but screen reader users heard only meaningless filenames. A quick manual review rewrote the strings to describe product type, color, and key feature – cutting customer support calls significantly in the next release.

These patterns-dynamic state, intent, real-world context-recur in every audit. Until tools can ask why is this here? and does it make sense to a person?, expert and disabled person testing remains the only safety net that prevents compliance metrics from becoming a vanity score.

WCAG 2.2 quick-reference: which Success Criteria can be scripted, which can’t, and why

Below is a table outlining all WCAG Success Criteria, along with corresponding test cases, an indication of whether they can be automated (YES/NO), and a justification column explaining why certain criteria cannot be automated or the challenges involved.

Note: the data in the table will change dynamically because the cases and tests will be subject to continuous verification.