Quality gates

A quality gate is a set of conditions a change must meet before it merges: no new bugs, enough coverage on the lines it adds, and so on. Sonda evaluates the gate on every scan, and the CLI's exit code carries the result, so any CI system can stop a build on it.

Why gate new code

Gating the whole codebase makes every change answer for every old issue, so legacy projects either fail forever or get thresholds loose enough to mean nothing. Gating new code asks one thing of each change: don't make it worse. Over time, as code is touched, the old issues get fixed where people are already working. SonarQube calls this "clean as you code", and Sonda reports every issue and metric twice: overall, and on new code.

How Sonda evaluates a gate

A gate is a list of conditions in sonda.toml. Each names a metric, an operator (gt or lt) and a threshold. The gate fails when any condition holds. sonda init writes the default gate:

[[gate.conditions]]
metric = "new_bugs"
op = "gt"
threshold = 0

[[gate.conditions]]
metric = "new_vulnerabilities"
op = "gt"
threshold = 0

[[gate.conditions]]
metric = "new_coverage"
op = "lt"
threshold = 80

[[gate.conditions]]
metric = "new_duplicated_lines_density"
op = "gt"
threshold = 3

# Blocker or critical known vulnerabilities in dependencies the change added
# or upgraded into. No data, so never failing, without advisories.
[[gate.conditions]]
metric = "new_severe_vulnerable_dependencies"
op = "gt"
threshold = 0

What counts as new code

--new-code origin/main compares with the merge base of a branch: everything changed since then, committed or not, plus untracked files. Without the option, Sonda chooses: a GitHub Actions pull request's base branch, a push's pushed commits, the merge base with the default branch on a diverged branch, or otherwise uncommitted changes. The report says which it used. --new-code none treats the whole project as new.

When a condition doesn't count

A condition can't fail without data, and some are set aside on small changes:

• No data, such as coverage without a report, or dependency vulnerabilities without advisory data, is shown and ignored.
• Fewer than 20 new lines: conditions on new_coverage, new_duplicated_lines and new_duplicated_lines_density are ignored, since one uncovered line in a two-line change would be 50% coverage.

An example

A branch adds one function with a loose comparison. Scanned against main with the default gate:

sonda scan . --new-code main

src/discount.js
*    2:12   MAJOR     Bug            js/eqeqeq                        Use '===' or '!==' instead.

Summary (new code = changes since main, marked *)
  Files 2 · Lines of code 11 · Functions 2 · Complexity 4
  Bugs 1 (1 new) · Vulnerabilities 0 (0 new) · Code smells 0 (0 new)
  Duplication 0.0% · Coverage n/a (new code n/a) · Debt 5min

Quality gate: FAILED
  ✗ fails if new_bugs > 0 (actual: 1)
  ✓ fails if new_vulnerabilities > 0 (actual: 0)
  – fails if new_coverage < 80 (actual: no data, ignored: fewer than 20 new lines)
  – fails if new_duplicated_lines_density > 3 (actual: 0, ignored: fewer than 20 new lines)
  – fails if new_severe_vulnerable_dependencies > 0 (actual: no data, ignored: no advisory data)

The scan exits with 1.

Exit codes

--no-gate always exits 0, for a job that should report without blocking.

A custom gate

Any metric works in a condition. This gate allows no new blocker or critical issues, and at most an hour of new technical debt:

[[gate.conditions]]
metric = "new_blocker_issues"
op = "gt"
threshold = 0

[[gate.conditions]]
metric = "new_critical_issues"
op = "gt"
threshold = 0

[[gate.conditions]]
metric = "new_debt_minutes"
op = "gt"
threshold = 60

The metrics a condition can use:

• Size and complexity: files, lines, ncloc, functions, complexity, cognitive_complexity.
• Issues: issues, bugs, vulnerabilities, code_smells, blocker_issues, critical_issues, major_issues, debt_minutes.
• Duplication and coverage: duplicated_lines, duplicated_lines_density, coverage.
• On new code: new_lines, new_issues, new_bugs, new_vulnerabilities, new_code_smells, new_blocker_issues, new_critical_issues, new_major_issues, new_debt_minutes, new_duplicated_lines, new_duplicated_lines_density, new_coverage.
• Dependencies: vulnerable_dependencies and severe_vulnerable_dependencies, and their new_ variants, with data only when checked against advisories.

An unknown metric or operator is an error, not a silently ignored line.

In CI

The exit code is the integration: run sonda scan as a step, and a failing gate fails the job. The GitHub Actions guide has complete workflows.

Load the gate from a trusted place. By default Sonda reads sonda.toml from the repository being scanned, so a pull request could weaken its own gate. Read it from the base branch instead:

git show origin/main:sonda.toml > /tmp/sonda.toml
sonda scan . --config /tmp/sonda.toml --new-code origin/main

On the server

With the Sonda server, a project's gate can also be a server gate, which admins create on the Quality gates page with the same metrics and operators. Once a project uses one, it wins over sonda.toml, and sonda scan --upload passes or fails on the server's verdict. Issues accepted or marked as false positives on the server stop counting, so they no longer fail the build. The Quality gates page shows each gate's projects, its recent evaluations and its most common failure.

Practices that work

Opinions, not rules:

• Start with the default gate, and change it when a real case shows why.
• Gate new code, report everything else. Overall numbers belong on a dashboard, not in a merge check.
• Keep conditions few. A gate that fails for many reasons gets bypassed.
• Treat the gate as code. Review changes to it like any other, and load it from a trusted branch in CI.

Limitations

• Operators are gt and lt only.
• The gate judges a whole project; there's no per-component gate (see component analysis).