Do design systems actually deliver on accessibility?

Pick a design system and you get accessibility for free. I have heard some version of that line in pitch decks, conference talks, and README files for years. The components ship with sensible semantics, the contrast decisions are already made, and focus management is somebody else's solved problem. It sounds plausible. As far as I could find, nobody had checked it against real production websites at any serious scale.

So I checked. 12,252 US federal websites, then 8,136 UK public sector sites I scanned myself. This post is the plain English version of what I found. The full treatment, with the pre-registration and every regression table, lives on my research site, and I will point you there at the end.

What I measured

The US government runs a programme called GSA Site Scanning. It visits every known live federal website daily and records, among other things, two values I care about: a graded score for how much of the US Web Design System (USWDS) each page uses, detected from CSS classes, asset paths, fonts, and version strings, and the number of accessibility violations that axe-core finds on the page.

Axe-core is the engine behind Lighthouse's accessibility audit, so a violation here means the machine-checkable kind: missing form labels, broken contrast, bad ARIA, images without alt text. No scanner can tell you whether your focus order makes sense to an actual person. Keep that in mind for everything below. This is the floor of accessibility, not the ceiling.

I froze a snapshot from June 2026, kept every live primary site with a completed accessibility scan, and grouped sites into five adoption bands, from no USWDS signal at all to definitely built on it.

The raw gradient is hard to miss. Sites with no design system signal average 6.3 detected violations on the homepage; likely adopters average 2.4. Another way to see it: about a quarter of no-signal homepages scan completely clean, and that share climbs to roughly half once a site is a likely or definite adopter.

About half, once you compare like with like

Raw averages are where the sceptic in you should speak up, because agencies differ in everything: budgets, team size, how modern the stack is. So the headline number comes from a model that only compares sites within the same agency, and adjusts for general digital maturity signals like proper HTTPS, sitemaps, and modern hosting.

After all of that, strong adopters still show about half the violations of non-adopters. The precise version: an incidence-rate ratio of 0.50, with a 95% confidence interval from 0.38 to 0.65. The effect is also graded. A few traces of USWDS buy a little, partial adoption buys more, and the big step arrives at heavy adoption. That is the shape you would hope to see if the components themselves are doing the work.

Could it just be better teams?

Nobody assigned design systems to agencies at random. The teams that adopt one might be the same teams that care about accessibility in the first place. Observational data cannot fully rule that out, and the paper says so plainly. But I pre-registered three checks designed to break the result, and it survived all three.

• The within-agency estimate barely moved. If team quality explained everything, the association should collapse once you stop comparing across agencies and only compare each agency's sites with each other. It shrank by about 20% and then held.
• Placebo outcomes stayed flat. Teams that are simply better at engineering should also be visibly better at performance. They are not. The adoption gradient on accessibility is steep (-0.20 standard deviations), while the same model pointed at layout shift and load speed finds roughly nothing (+0.09 and -0.07).
• Selection bounds. There is a standard method, Oster's, that asks how strong an unobserved team-quality effect would need to be to wipe the result out. The answer is: stronger than everything I did control for, combined. Even then the estimate stays clearly negative.

Then I tested it on a country the model had never seen

Find a pattern in one dataset and you should worry you have quietly tuned your analysis to it. So before touching any UK data, I wrote the entire UK analysis down: the detector, the model, the hypotheses, and a pass-or-fail rule saying the UK estimate had to land within 0.20 of the US one. I locked all of it in a public pre-registration. Only then did I scan 8,136 UK public sector websites, central government, local councils, parish councils, and the NHS, in one afternoon. The detector looks for govuk-frontend, the design system behind GOV.UK, scored as a component-for-component mirror of how GSA scores USWDS.

One shot, no peeking. 6,295 sites survived deduplication and quality filters, and the UK answer came back: 44% fewer violations for strong adopters, an incidence-rate ratio of 0.56 against the US value of 0.50. It landed 0.07 away, well inside the window I had committed to.

What didn't go my way

Three predictions missed, and they are the most useful part of the study.

• USWDS v3 is not measurably better than v2. I expected the newer major version to show fewer violations. The point estimate goes the wrong way (1.50) with a confidence interval wide enough (0.95 to 2.37) to call it a wash. On this evidence the win is adopting a system at all, not chasing version upgrades.
• The UK partial-adoption band is a mess. Sites with a medium detection score showed about 4.0 times the violations of non-adopters. On inspection the band holds only 39 sites, mostly CMS templates that sprinkle govuk-frontend class names onto markup that is not really the design system. The detector reads partial adoption; reality is a site wearing the costume.
• My mechanism prediction failed. I predicted the violation categories that components handle directly, contrast, ARIA, labels, would improve more than page-level ones like landmarks and document structure. No meaningful difference. The data declined to confirm my mental model of why the effect exists, which is worth knowing too.

What I'd take from it as an engineer

1. If you are weighing bespoke UI against a mature design system, accessibility is now a measurable argument rather than a bullet point of faith. Roughly half the automated violation count, twice, in two countries, on over 18,000 production sites.
2. Partial adoption is where the trouble hides. A handful of design system classes on a CMS theme can look like adoption to tooling and deliver none of the benefit. If you adopt, adopt properly: the components, not just the class names.
3. The floor is cheap to measure. My scan of 8,136 sites took one afternoon on one laptop with open tooling, and your estate is smaller than a country's. Point axe-core at it. Just remember that passing the scanner is the start of accessibility work, not the end of it.

Dig deeper

Everything above is the conversational version of a pre-registered study whose data, code, and locked hypotheses are public. If you want the regression tables, the diagnostics, or you would rather interrogate the data yourself, start here.

• The pre-registration, locked before any estimation ran.
• Explore the data in your browser: US federal sites and UK public sector sites. Queries run locally with DuckDB, and the raw Parquet files are downloadable from the paper page.