# The CI Matrix

Serena's CI has a problem most projects never meet: sixty-odd languages, each with its own
toolchain, and no runner that could hold them all at once. The answer is a matrix with
opinions — and the opinions are written down, in the header of
`.github/workflows/pytest.yml`. This page walks through them; when CI surprises you, that
header is the primary source.

## Five batches, three operating systems

Tests run as parallel matrix jobs over (OS × batch). The five batches are marker-based —
`jvm`, `native`, `other-langs`, `niche`, and a `catch-all` for everything unmarked — and
the batching is deliberately by pytest *marker*, not pytest-xdist: tests share fixtures,
language-server processes and on-disk resources, so in-process parallelism would race them.

Two properties keep the partition honest: each batch installs only the toolchains its
languages need, and `catch-all` is defined as the negation of all the named groups — every
test lands in exactly one batch, and adding a language cannot desync the split.

## Where the matrix thins out

The default is the full cross-product, with two kinds of exception:

- **`niche` runs on ubuntu only.** Its toolchains (lean4, ocaml, nix, julia, R, perl) are
  heavy and slow — not worth the Windows and macOS cost.
- **An OS-specific language inside an all-OS batch** (swift on macOS, haskell on Linux) is
  skipped one level down, by the central guard in `test/conftest.py` — never by the matrix.

That split matters when you add a language: OS gating belongs in the conftest, not in the
workflow file.

## The ceilings

GitHub's free tier caps runs at 20 concurrent jobs and 5 concurrent macOS jobs. The current
matrix is 13 jobs, 4 of them macOS — inside both limits, and anything that adds batches or
operating systems has to stay inside them too, or jobs queue. Superseded runs on the same
ref are cancelled outright to free capacity, and every job carries a 60-minute timeout: a
healthy warm-cache run peaks around 16 minutes, and the ceiling exists because a hung job
once burned the full six hours.

## Caching

Language servers are cached per (OS × batch); the uv venv is keyed on `uv.lock`; several
individual toolchains and the web fixtures' npm state have caches of their own. Most of the
niche toolchains restore from cache, which is why that batch stays fast despite its size.

## What this means for your pull request

- `poe lint` and `poe type-check` run once per OS, in the catch-all batch — the format gate
  is the first thing to fail, in the first minute.
- Your language's tests run in whichever batch its marker belongs to; nothing to configure.
- A red run is usually one red *job*. Read that job's log before concluding anything about
  the other twelve.
- The docs build (Sphinx, warnings as errors) and a spell check run alongside the tests.