Skip to content
Sentinela

Method

How we test

What our automated checks cover, how the score works, and what a person adds in an audit.

Automated testing

Monitoring and the free scan use automated testing only: axe-core rules mapped to WCAG 2.2 level A and AA criteria, plus, in monitoring, our own keyboard, accessibility-tree and purchase-flow checks. Monitoring renders each page at 1366 and 320 pixels wide; the free scan at 1366 only. Pages are sampled by type, such as the home page, product pages and the cart, up to the plan's page limit, and we respect robots.txt.

Websites: the keyboard form path

On a website that isn't a store, pages are sampled the same way: the home page, then contact, booking, menu, services, about and sign-up. The purchase-flow check becomes a form path: on your contact, booking or sign-up page we reach every field with the keyboard alone, type test text, and check that a wrong email shows an error people can perceive. The form is never sent. A booking service embedded from another company, such as OpenTable, Calendly or Mindbody, is third party: the keyboard form path stops at it. Automated checks still list issues inside its frame, apart from yours, as third-party content: only that company can fix them, and they don't count in your score. The free scan doesn't test the form.

Sample pagesHome, collections, products, cart, by type
Render twiceAt 1366 and 320 px wide
Run checksaxe, keyboard, accessibility tree, purchase flow
Score and compareAgainst the last scan of the same site

Automated testing detects only some accessibility issues. Its results are not a full audit and do not determine legal conformance.

The score

The score summarizes automated findings from 0 to 100 so you can follow a site over time. It is not a percentage of WCAG conformance. When we change how it is calculated, as when third-party content stopped counting, past scores are not recalculated, and a comparison across the change says so.

  1. 1Each page starts at 100.
  2. 2Each distinct issue on the page subtracts its severity weight multiplied by our confidence in the finding.
  3. 3A page never goes below 0, and the site score is the average of its pages.
  4. 4Issues rejected in review, or marked as false positives with a reason, do not count.
  5. 5AI-assisted suggestions are shown separately and do not change the score until a person confirms them.
  6. 6Issues inside another company's frame, such as a booking widget, a video player or a payment field, are listed apart as third-party content and do not count.
Severity weights
SeverityWeight
Critical10
Serious5
Moderate2
Minor1
Example pageaxe findings, confidence 0.95

Comparing scans

Each scan is compared with the previous one of the same site:

New
Found now, not found last time.
Persisting
Found both times.
Fixed
Found last time and not found now, on a page we scanned again. An issue on a page that was not tested again is not counted as fixed.

How reports word results

Issues found
An automated check or the AI review reported an issue.
No issues detected by automated checks
The checks found nothing, which does not show that the criterion is met.
Needs human review
No automated check covers the criterion, or the result was unclear.
Meets or does not meet the criterion
Only in audit reports, from a reviewer's manual check.

AI findings are shown as suggestions pending review until a person confirms them. Audit reports include only confirmed findings.

What a person checks in an audit

A reviewer confirms or rejects every automated and AI finding, adds their own, records a verdict for every WCAG 2.2 AA criterion, and completes this checklist before the audit is issued. Evidence is kept for 3 years.

Scope, tools and sampling

  • Scope: the pages agreed when the audit is ordered, up to the page count of its size (small, medium or large). The automated scan covers all of them; the manual checks run on the home page, a collection, a product, the cart and the start of checkout, plus the login, contact and newsletter forms where the store has them.
  • Tools: a keyboard alone; NVDA with Firefox or Chrome on desktop; VoiceOver with Safari on iOS; the browser at 200% zoom and at a 320 px wide viewport; text-spacing overrides; a contrast checker for spot checks.
  • Target: WCAG 2.2 at level AA (or 2.1 AA when a customer asks for it). The report states the pages, the dates and the target, and marks each criterion as meets, does not meet, or not applicable, with the evidence behind the verdict.

See the method on your store

A free scan runs the automated checks on your home page and two key pages.

Get a free scanBook a call