Understanding quality gates in CI/CD

A quality gate is the point in a pipeline where code quality stops being a dashboard and becomes a decision: this change merges, or it doesn't. Getting that decision right is mostly about what you ask, and when. This article covers the ideas, then shows them with Sonda, where every output below comes from a real run.

What a gate is

A gate is a set of conditions over metrics: no new bugs, coverage on new code of at least 80%. The pipeline measures the change, checks each condition, and passes or fails. In CI, failing usually means a non-zero exit code, so the job fails and the pull request can't merge.

Judge the change, not the history

A gate over the whole codebase ("fewer than 50 code smells") has two failure modes. On an old codebase it fails every build until someone loosens the threshold into meaninglessness. On a new one, it passes until the day it doesn't, and then blocks whoever happens to be merging.

A gate over new code asks every change the same, fair question: did you make it worse? Old issues get fixed as code is touched, which is where people are already working. That's why Sonda's default gate only uses new_ metrics:

[[gate.conditions]]
metric = "new_bugs"
op = "gt"
threshold = 0

[[gate.conditions]]
metric = "new_coverage"
op = "lt"
threshold = 80

(The full default adds new vulnerabilities, duplication on new code, and new dependencies with severe known vulnerabilities.)

What "new" means

New code has to be measured against something. On a branch it's the merge base with the target branch: everything changed since the branches diverged. Sonda takes it explicitly (--new-code origin/main) or works it out: in a GitHub Actions pull request, the base branch; on a push, the pushed commits; on a branch that has diverged from the default branch, the merge base; otherwise, uncommitted changes. The report always says which it used, because a gate judging the wrong range is worse than no gate.

Small changes and missing data

Percentages behave badly on small numbers. A two-line change with one uncovered line has 50% coverage. A naive gate fails it; a reasonable one shouldn't. Sonda ignores conditions on new coverage and new duplication when a change has fewer than 20 new lines, and says so.

Missing data needs the same care. No coverage report means coverage is unknown, not zero. Sonda shows such conditions as "no data" and doesn't fail on them. A real run on a branch that adds one function with a loose comparison:

Quality gate: FAILED
  ✗ fails if new_bugs > 0 (actual: 1)
  ✓ fails if new_vulnerabilities > 0 (actual: 0)
  – fails if new_coverage < 80 (actual: no data, ignored: fewer than 20 new lines)
  – fails if new_duplicated_lines_density > 3 (actual: 0, ignored: fewer than 20 new lines)
  – fails if new_severe_vulnerable_dependencies > 0 (actual: no data, ignored: no advisory data)

One real problem fails the gate, and the conditions that couldn't be judged say why.

Exit codes are the interface

A gate is only as useful as its integration, and the simplest integration is the exit code. Sonda uses three:

• 0: the gate passed;
• 1: the gate failed;
• 2: the tool couldn't run (bad configuration, a git error).

Keeping "the code isn't good enough" separate from "the tool broke" matters: the first is the author's to fix, the second is the pipeline's. --no-gate always exits 0, for jobs that should report without blocking.

Don't let a change edit its own gate

If the gate's configuration lives in the repository, a pull request can change it, by deleting a condition or raising a threshold, in the same diff it's judged on. Load the configuration from the target branch instead:

git show origin/main:sonda.toml > /tmp/sonda.toml
sonda scan . --config /tmp/sonda.toml --new-code origin/main

Gate changes still go through review; they just take effect after they merge, not during.

Where the gate lives

Sonda can evaluate the gate in two places:

• In the CLI, from sonda.toml. Nothing else needs to run, so it works on a laptop as well as in CI.
• On the Sonda server, as a server gate admins manage. Uploads then pass or fail on the server's verdict, which also takes triage into account: an issue accepted as a false positive stops failing builds.

Start with the first. Move to the second when several projects should share one gate, or when triage decisions should count.

Quality gates is the reference for Sonda's conditions and metrics, and the GitHub Actions guide wires a gate into a workflow.