Steward
Sign in

Documentation › Contributor guides

Read the verdict on your PR

So you've opened a PR and Steward posted a verdict comment. This guide walks through reading it from the contributor angle — what to do, what to respond to, when to push back.

Background reading: Contributing to a Steward repo covers what the verdict is + what's automatic. This guide assumes you've skimmed it.

The verdict shape

Steward verdict: 78 / 100 — amber

> Solid surgical change to the auth middleware with tests
> alongside, but the diff is on the heavy side for a single
> review pass.

### Correctness — 85
Source files changed alongside test files; ratio above 0.33.

### Security — 65
Touched internal/auth/middleware.go with paired test changes;
amber rather than green because the sensitive surface still
warrants a focused review.

### Complexity delta — 55
432 LoC across 18 files. On the boundary between "medium" and
"large"; consider splitting.

### Docs alignment — 90
docs/auth/middleware.md updated alongside the code.

### Governance alignment — 75
governance.pxf present; touched files map to the `auth` domain.

Five dimensions, each scored 0–100, statused green / amber / red / unknown. The headline number is an unweighted mean of the non-unknown dimensions. Status colours: ≥75 green, 50–74 amber, <50 red.

Read it in this order

  1. The number + colour. Triage signal. Green = your PR looks healthy; the rest of the verdict will probably confirm that. Amber = one dimension to look at. Red = the deterministic scorer flagged something hard.
  2. The headline sentence. One-sentence summary written by the Large Language Model (LLM). If it matches your mental model of the PR, you can skim the dimension paragraphs. If it doesn't match, read every paragraph.
  3. The dimension(s) you care about for THIS PR. A docs-only PR's correctness dimension is unknown and you can ignore it. A security-sensitive change's security dimension is the first one to read.

What to do per dimension

Correctness — red or amber

You changed source files but the test ratio is low. Common fixes:

  • Add tests for what you changed. The threshold is roughly one test per 3 source files; a green here usually means the PR includes a test diff alongside the source diff.
  • If the change genuinely doesn't need tests (a comment fix, a typo, a Markdown rendering tweak that happens to live in a .go file's source tree) — leave a PR comment explaining the exception. Reviewers see the verdict; you can pre-empt the "where are the tests?" question.

Security — red

You touched a file path matching one of Steward's security-sensitive patterns (anything with auth, crypto, jwt, token, password, secret, tls, webhook, etc. in the path) without test changes. The path heuristic is coarse; if your change is a genuine doc/comment fix, leave a PR comment.

If your change is a real security-surface change, you probably want to add tests before merging anyway. Reviewers will ask. The dimension is doing you a favour by flagging it.

Complexity delta — red

Your diff is over 1000 lines of code or touches more than 50 files. Standard advice: split the PR. Some legitimate exceptions:

  • A formatter pass over the whole codebase.
  • A library upgrade with mechanical follow-up across many files.
  • A git mv followed by minor edits.

If you're doing one of those, leave a PR comment explaining it. Reviewers will know to skim rather than read line-by-line.

Docs alignment — amber

You changed multiple source files but no docs. Action: update the relevant docs (typically docs/ or README.md or a per-package doc.go). If the change doesn't have user- facing impact (a refactor, an internal-only API change), a PR comment explaining that is the usual response.

Governance alignment — anything below green

Either the repo has no governance.pxf (you'll see "no governance.pxf on the default branch") or your files don't map to a declared domain. The first case is the maintainer's fix, not yours; the second sometimes is yours (you put a file in the wrong directory) or sometimes the maintainer's (the governance file's path_globs need updating).

If you suspect the latter, leave a comment in the PR mentioning it — it gives the maintainer a heads-up to review the governance config alongside your change.

When you should disagree with a verdict

The verdict is advisory. Three legitimate reasons to merge a red-overall PR anyway:

  1. The PR is a deliberate rule-bender — a 2000-LoC refactor that needed to land atomically. Red on complexity is the correct flag; that's not the right blocker.
  2. The dimensions don't apply. A migration script with no tests is correctly flagged red on correctness, but you and the maintainer might decide that's fine for THIS file.
  3. You disagree with the LLM's prose. The deterministic scores stand on their own; the prose is interpretation. Read the dimension scores, ignore the paragraph, decide.

What you should not do:

  • Don't rename files to game the path heuristic. Renaming auth_check.go to validator.go silences the security dimension but not the maintainer's actual review need.
  • Don't quietly ignore a red correctness without explanation. The dimension is sticky — your future PRs will face it again, and the maintainer's reputation in your performance will track it. Discuss it in a PR comment if you really think the gate is wrong.

When the verdict surprises you

The verdict is much higher than you expected

You wrote a big PR with no tests and it still scored green? Likely cause: the repo has loose governance.pxf thresholds or doesn't have one. Your PR is being scored against a permissive baseline. The verdict is technically green, but a human reviewer might still ask for changes.

The verdict is much lower than you expected

You wrote a careful PR with tests and it scored red? Most common cause: a complexity-delta red — your diff was big. Even high-quality changes get flagged when they're large. Consider splitting; or leave a PR comment explaining why the atomic merge is necessary.

Second-most-common: a security-path heuristic hit. A typo fix in auth/handler.go reads the same as an auth rewrite to a path-only scorer. Leave a PR comment.

Re-running the verdict

If you've pushed changes that address the verdict's points, push them — the verdict updates automatically on pull_request.synchronize. If you want to re-run without pushing (e.g., a transient failure), comment:

/steward evaluate

A fresh verdict lands within seconds.

"Blocked by" vs "Heads-up" gate comments

Alongside the verdict, individual gates (coverage, complexity, and friends) can leave their own comments. The opening words tell you whether action is required:

  • "Hi @you — this PR is blocked by the … gate." The gate is in block mode: fix the listed items and re-push, or the PR won't merge.
  • "Heads-up @you — …" ending with "this repository runs this gate in warn mode, so the merge is not blocked." The maintainers are measuring this rule before enforcing it. Your PR can merge as-is — but the comment shows exactly what would block once they flip the gate on, so treating it as an early fix list is a favour to your future self (and shows up well in your contribution history).

A warn-mode PR can collect several heads-up comments in one push — advisory mode reports every gate that would have fired, not just the first.

What's next