Quality audit

@alea-design/audit runs your app in a real browser once for every combination of width, theme, text size, language, pointer and motion setting, and checks every route on each. It is the common core of the audit scripts Alea, Heliodore, Spinelle and Grenat each wrote on their own.

Install

The package does not bundle a browser. It drives the Google Chrome on the machine (or the one you point CHROME_PATH at) through playwright-core, and reads axe-core from your project. Both are optional peers: the audit says what to install when one is missing.

npm i -D @alea-design/audit playwright-core axe-core
npx alea-audit init      # writes alea-audit.config.mjs and a GitHub Actions workflow (one job, weekly)
npx alea-audit           # npx alea-audit list shows the checks

With the Alea CLI installed, alea audit does the same and tells you what to install when the package is missing.

Config

One file declares the app, the routes, the matrix and the checks. Logic stays in the package; the file is data plus a few hooks.

// alea-audit.config.mjs
import { defineConfig } from '@alea-design/audit';

export default defineConfig({
  server: { command: 'npm run build && npx vite preview --port {port} --strictPort', url: 'http://127.0.0.1:{port}' },
  routes: ['/', '/pricing', { path: '/settings', setup: async ({ page }) => { await page.getByRole('button', { name: 'Load sample data' }).click(); } }],
  viewports: [320, 375, 768, 1024, 1280, 1920],
  themes: ['light', 'dark'],
  theme: { attribute: 'data-theme', storageKey: 'my-app-theme' },
  textSizes: [100, 150, 200],
  locales: ['en', 'es'],
  checks: {
    overflow: { scrollRegions: ['.alea-table-wrap'] },
    'layout-parity': true,
    guardrails: true,
  },
  guardrails: { files: ['src/**'], ignoreFiles: ['**/tokens.css'], rules: { 'raw-px': true, 'raw-hex': true, 'long-dash': true, 'alea-internals': true } },
  ignore: [
    { check: 'axe', rule: 'color-contrast', route: '/legal', reason: 'The third-party widget draws this text; reported upstream.' },
  ],
});

The matrix

The default strategy, cover, uses every value of every axis at least once and always meets the narrowest width with the largest text, which is a few dozen sessions. Use matrix: { strategy: "cross" } for the full product (coarse pointers are only paired with widths up to 1024).

Before any check measures, the page has to be still: no loading indicator, fonts ready, and two passes in a row with no finite animation running. One quiet pass is not enough, because data that arrives late starts new animations, and an exiting row stays in the document until its animation ends. Measuring earlier gives contrast ratios of 1.01 at half opacity and 43 px buttons at a scale of 0.98.

Checks

Custom checks go in customChecks: an object with a name, a description and a run function that receives the page and returns findings.

Screen-reader simulation

The simulation reads Chrome’s accessibility tree and computes what a reader would announce: name, role, state, position, table rows with their headers, the regions entered and left, and live regions captured while you act. It is available on its own for end-to-end tests.

import { ScreenReader, verifyErrorAnnounced } from '@alea-design/audit/reader';

const reader = await new ScreenReader(page).attach();
await reader.readScreen();                       // what the whole screen says, in order
await reader.press('Tab');                       // { text: 'Save, button', live: [...] }
const problems = await verifyErrorAnnounced(page, reader, {
  field: '#email',
  trigger: () => page.keyboard.press('Enter'),   // focus is already in the field
});
console.log(reader.toText(), reader.issues, problems);

Known limits, not findings: date input segments follow the browser language; there is no per-reader punctuation or acronym behaviour; non-breaking hyphens and negative balances written with a minus sign need a real reader. It does not replace VoiceOver or NVDA.

Continuous integration

Results are cached by route, combination, build and check settings, so an unchanged run finishes in seconds. A browser that closes under load is restarted for that combination and is never a finding. AUDIT_JOBS or --jobs sets how many combinations run at once; by default it follows the CPU load.

name: Quality audit
on:
  workflow_dispatch:
  schedule:
    - cron: '0 5 * * 1'
  # pull_request:
  #   paths: ['src/**', 'public/**', 'package-lock.json']
concurrency:
  group: audit-${{ github.ref }}
  cancel-in-progress: true
jobs:
  audit:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci --no-audit --no-fund
      - uses: actions/cache@v4
        with:
          path: .alea-audit/cache
          key: alea-audit-${{ hashFiles('package-lock.json', 'src/**') }}
          restore-keys: alea-audit-
      - run: npx alea-audit run --jobs 2 --json audit.json
      - uses: actions/upload-artifact@v4
        if: failure()
        with: { name: audit, path: '.alea-audit/artifacts' }

Private repositories pay for every minute of every job, and each job installs the project again. So the workflow that init writes is one job that runs on demand and once a week, not on every push; the pull_request trigger is there as a commented opt-in for the paths that change the UI. A newer run cancels an older one, a run stops after 30 minutes, and the npm install and the results are cached. The header of the file estimates the billed minutes.

Run npx alea-audit run --plan to count the matrix and estimate the time before you add a trigger. init --shards 3 splits the matrix over three jobs: it finishes sooner but bills about two more minutes for every shard. The audit uses the Chrome that GitHub Ubuntu runners already have, and running it on your machine costs nothing.

The exit code is 0 when nothing at or above failOn was found, 1 when something was, and 2 when the audit itself could not run. --json writes the full report, and screenshots of failing screens go to .alea-audit/artifacts.