Skip to content
quest'n'click

ContributeHow-toFor contributorsApplies from 4.2.3

The coverage ratchet

Cette page en françaisThis page on the GitHub wiki

The coverage floors of vite.config.ts only rise. Two checks hold them: Vitest fails when the tests reach less than a floor, and the ratchet fails when a floor lags three points or more behind what the tests reach. This page says what to do when either is red.

What you get

A coverage job green again: every floor met, and every floor within three points of what the tests measure.

Context and versions

The engine repository. The floors date from 4.1.0 "Clarity" (the level measured then); the ratchet, tools/coverage-ratchet.ts, from 4.1.3; its GitHub annotations from 4.1.9.

What is measured: V8 coverage of src/**/*.ts (not *.d.ts), bridge/src/**/*.ts and studio-server/src/**/*.ts, by the unit suite and the asset tests (npm run test:coverage). The floors as of 4.4.0:

ScopeFloor
All measured fileslines 75, statements 74, functions 71, branches 70
src/engine/core/{cond,diff,migrate,save}.tsbranches 100
src/engine/core/session-runtime.ts, tools/replay.ts, core/content-ids.tsbranches 94, 99, 96
src/engine/core/reality-runtime.ts, reality/protocol.ts, reality/client.ts, reality/http-port.tslines and branches, 90 to 100
23 files of the Studio Server (the journal, the gate, OIDC, leases, previews, the worker's jobs…)lines and branches, set two points under what was measured when each arrived
6 files of the Bridge (bridge.ts, store.ts, server.ts, policy.ts, lock.ts, cli.ts)lines and branches

The exact values are in vite.config.ts, test.coverage.thresholds, with the reason for each floor below 100 written beside it.

Prerequisites

A prepared repository with Python and its image modules (Developer setup): the asset tests run under coverage too.

Where to run the commands

In the engine repository, on your machine. CI runs the same in its coverage job.

Steps

  1. Measure.

    # in the engine repository
    npm run test:coverage

    It runs the unit suite under coverage and writes .cache/coverage/ (a text summary, coverage-summary.json, an HTML report in .cache/coverage/index.html). Vitest fails here when a floor is not met, naming the file and the metric.

  1. A floor not met (the tests reach less than it): your change removed tests or added code no test runs. Add the tests; open .cache/coverage/index.html to see the lines and branches not run. A floor is never lowered to let a change through.
  1. Read the ratchet.

    # in the engine repository, after test:coverage
    npx tsx tools/coverage-ratchet.ts            # warns
    npx tsx tools/coverage-ratchet.ts --strict   # fails, as on main

    Each line ⚠ <scope> <metric>: floor <n>, measured <m> is a floor three points or more under what the tests reach. The end says ✔ every coverage floor is within three points of what the tests reach, or the count to raise.

  1. Raise the floors it names in vite.config.ts (test.coverage.thresholds), to a value at or a little under what was measured, and commit them with the change that earned them. vite.config.ts is a file the maintainer reviews (CODEOWNERS).

Expected result, and how to check it

npm run test:coverage passes and npx tsx tools/coverage-ratchet.ts --strict exits 0. In CI, the coverage job is green with no "coverage ratchet" annotation.

Common errors

SymptomCauseFix
Vitest: a threshold for lines, branches, functions or statements not metFewer lines run than the floorAdd tests (step 2)
⚠ coverage ratchet annotation on a pull requestA floor three points or more under the measureNot blocking on a pull request: raise it (step 4), or leave it to the next change
✖ n floor(s) at least 3 points below… on main, a tag or release-checkThe same, under --strictRaise the floors, in a pull request
✖ …: no coverage summary (npm run test:coverage first)The ratchet ran without a measureRun npm run test:coverage first (exit 2 otherwise)

Files or data changed

vite.config.ts (the floors); the measure writes only .cache/coverage/.

Recovery or going back

A floor raised too high fails the next run: lower it back to what the tests reach (with the reason in the pull request), since a floor above the measure is a red build, not a promise.

References