post-pr-review

Post AI review findings on a GitHub PR as inline comments. The producer side of a producer/consumer pair — review-pr is the consumer.

Argument grammar

Flag	Default	Notes
<PR#> positional	autodetect from current branch	optional
--repo OWNER/REPO	autodetect	needed for fork PRs
--agents <csv>	profile-based (see step 5)	dotbabel agents to dispatch (see step 5)
--max-comments N	25	hard cap; >25 requires explicit override
--event	COMMENT	COMMENT / REQUEST_CHANGES / APPROVE
--mode	review	review (atomic batch) or inline (per-comment loop)
--summary-only	off	skip inline comments, post one top-level body
--auto	off	skip confirmation prompt
--dry-run	on in interactive; off only with --auto --confirm-post	prints what would be posted
--confirm-post	off	required alongside --auto to actually POST

Workflow

Before any step, bind args:

NUMBER="$ARGUMENTS"           # may be empty if autodetecting from branch
SCRIPTS="${CLAUDE_PROJECT_DIR:-$(git rev-parse --show-toplevel)}/plugins/dotbabel/scripts"
SCRATCH="${TMPDIR:-/tmp}"     # honors sandboxes that block /tmp; private to orchestrator

1. Resolve PR

"$SCRIPTS/post-pr-review-resolve.sh" "$NUMBER"

Output: JSON {number, headRefOid, state, isDraft, headRepository, baseRepository, isCrossRepository, url}.

Bind from the resolve output and thread --repo through every subsequent script so they don't each pay the gh repo view lookup cost (~100ms per script):

PR=$(... | jq -r .number)
HEAD_SHA=$(... | jq -r .headRefOid)
REPO=$(... | jq -r '"\(.baseRepository.owner.login)/\(.baseRepository.name)"')
GH_REPO_ARGS=(--repo "$REPO")    # pass to every downstream script

If autodetection fails, abort with: No PR found. Pass PR# explicitly: /post-pr-review 123.

2. Preflight gates

• gh auth status — fail at preflight if missing.
• state == OPEN — refuse closed/merged. No --force.
• isDraft — warn + prompt; --auto proceeds.
• isCrossRepository == true — warn the user that posting may 403 if they lack write access to the base repo.

3. Fetch context

"$SCRIPTS/post-pr-review-fetch-context.sh" "$PR" "${GH_REPO_ARGS[@]}" \
  > "$SCRATCH/post-pr-review-${PR}.ctx.json"

Reads PR diff (gh pr diff) into $SCRATCH/post-pr-review-${PR}.diff (where $SCRATCH=${TMPDIR:-/tmp}) and emits JSON with headSHA, diffPath, files, title, body. The exact paths are private to the orchestrator; downstream scripts read them by absolute path passed in as args.

4. Build postable line set

"$SCRIPTS/post-pr-review-build-postable-lines.sh" \
  "$SCRATCH/post-pr-review-${PR}.diff" \
  > "$SCRATCH/post-pr-review-${PR}.lines.json"

Output: {path: [line, line, ...]} — NEW-side line numbers per file that appear in any hunk. This is the gate for step 6 — any agent finding outside this set is rejected pre-POST to prevent 422s from GitHub.

5. Dispatch review agents in parallel

Resolve --agents. An explicit --agents value wins verbatim — the profile below applies only when the caller passed none.

Diff-profile default. Three of the four agents in the full set run on opus, so dispatching all four at every diff size is the single most expensive thing this skill does. Size the fleet to the diff instead. Take the changed files from step 3's context and the changed-line count from gh pr view "$PR" --json additions,deletions (additions + deletions), then apply the first rule that matches:

1. Any changed file matches a protected path in docs/repo-facts.json (when that file exists) → full set. Protected paths are behavior-governing even when they are markdown, so this rule is evaluated before the docs-only rule.
2. Every changed file is documentation (*.md, or under docs/) → documentation-writer + security-auditor.
3. Fewer than 150 changed lines → security-auditor + architect-reviewer.
4. Otherwise → full set.

security-auditor appears in every profile on purpose. Callers may have narrowed their own security step on the assumption that this skill runs one — commands/pre-pr.md step 3 does exactly that under --conductor — so dropping it here would leave a pipeline with no security review at any stage. Markdown is not a safe exemption either: in this repo a SKILL.md body is executable agent instruction, and in a consumer repo CLAUDE.md is the rule floor for every session.

The full set (chosen for high signal-to-noise on dotbabel PRs):

• architect-reviewer — design/coupling/scalability concerns
• security-auditor — secrets, injection vectors, OWASP top 10
• compliance-auditor — gate coverage, declared-vs-enforced invariants
• documentation-writer — comment accuracy, missing docstrings, drift

After selecting the profile (or honoring explicit --agents), write the input {profile, changed_files, changed_lines, candidates} to a private temporary JSON file. profile and candidates are arrays of reviewer IDs. The profile contains the selected reviewers; candidates contain only installed reviewers not already selected. Run:

dotbabel decide --step dispatch < "$DECISION_INPUT" > "$DECISION_OUTPUT"

Union the returned answer with the selected profile; never remove reviewers. Always retain security-auditor. On exit 2 or 64, keep the selected profile and report that the decider is unavailable. The built-in rule adds no reviewer when security is already present. Do not follow shadow: no model step is adopted. Append the decision to the phase's private JSON array and include --decisions "$DECISIONS_FILE" in the phase's pr-stack record call. Outside conductor mode, record post-pr-review once with the observed outcome. A dry run records no attempt and makes no decision call.

Report which profile fired in step 9 — a wrong profile is otherwise invisible, and the failure mode (a logic-heavy diff reviewed by the docs agent alone) is silent. Pass --agents to override whenever the heuristic misjudges a diff.

--agents inline skips agent dispatch and uses heuristics in this skill body only (see Inline-review heuristics below).

In a SINGLE message, invoke Task once per requested agent. Each invocation receives:

• The diff path.
• The list of changed files.
• The head SHA.
• The exact JSON contract from references/agent-contract.md — verbatim, no paraphrasing. Each agent MUST return ONE fenced JSON block per the documented schema.

6. Aggregate, validate, dedup, sort, truncate

1. Parse each agent's JSON. On parse failure, retry once with a stricter prompt; on second failure, log and skip that agent (don't abort the run).
2. Drop findings whose (path, line) is not in the postable line set.
3. Drop findings with confidence < 80.
4. Compute the marker for each finding (see Idempotency below).
5. Fetch existing markers via "$SCRIPTS/post-pr-review-list-markers.sh" "$PR" "${GH_REPO_ARGS[@]}" and drop any finding whose marker is already present.
6. Within this run, dedup by marker (different agents can converge on the same line).
7. Sort: severity DESC (critical → important → suggestion), path ASC, line ASC.
8. Apply --max-comments. Severity-aware: if truncating would drop one or more critical findings, refuse silently. In interactive mode offer [r]aise cap / [s]ummary-only / [a]bort; in --auto exit 1 with: N critical findings exceed --max-comments=M. Re-run with a higher cap or --summary-only.

7. Confirmation (skipped under --auto)

Render a table:

#  severity     agent                  path:line                  title
1  critical     security-auditor       src/api.ts:142             Hard-coded API key in commit
2  important    architect-reviewer     src/types/user.ts:18       Discriminated union would clarify intent
3  suggestion   documentation-writer   src/utils/format.ts:9      Public function lacks docstring
[y]es / [e]dit / [n]o

edit opens $EDITOR on a temp JSON the user can re-curate (the user's editor does the write; the skill reads the result back).

8. Post

• --summary-only: one POST to /reviews with body (markdown checklist of findings) + event. No comments[]. The --max-comments cap applies to body lines; tail collapses to "…and N more (run without --summary-only to see all inline)".
• --mode review (default), count > 1: one atomic POST /reviews with event, optional body (general notes from agents), and comments[]. Counts as one rate-limit event regardless of comment count. Use:

  "$SCRIPTS/post-pr-review-post-batch.sh" "$PR" \
    --comments-json "$SCRATCH/post-pr-review-${PR}.comments.json" \
    --event "$EVENT" \
    --body-file "$SCRATCH/post-pr-review-${PR}.body.md" \
    "${GH_REPO_ARGS[@]}"
  

• --mode inline or count == 1: loop POST /pulls/{n}/comments, sleeping 750ms between calls. Use post-pr-review-post-single.sh with "${GH_REPO_ARGS[@]}".

After each POST, parse rate-limit headers:

REMAINING=$("$SCRIPTS/post-pr-review-ratelimit.sh" \
  "$SCRATCH/post-pr-review-batch-${PR}.headers" | jq -r .remaining)

If REMAINING < 50 → bump sleep to 1s. If REMAINING < 10 → stop, report deferred count, exit 0 with hint at x-ratelimit-reset.

8b. Leave the receipt

Every run ends here, including a run that found nothing and including --summary-only and --mode inline. A run with no findings posts no comment, so without this a pull request whose review ran clean is indistinguishable from one that was never reviewed, and /pr-conductor could not tell the difference on re-entry.

Skip it only under --dry-run, or when step 8 failed and the findings never reached the pull request. Pin it to the head SHA bound in step 1:

"$SCRIPTS/post-pr-review-post-receipt.sh" "$PR" \
  --sha "$HEAD_SHA" --posted "$POSTED" --skipped "$SKIPPED" \
  --agents "$AGENTS_CSV" --profile "$PROFILE" \
  "${GH_REPO_ARGS[@]}"

It posts one body-only review whose commit_id GitHub records. $POSTED is the number of findings step 8 posted and $SKIPPED the number dropped as duplicates. Never write the receipt marker by hand: the guard hook blocks it, and the script is the only sanctioned writer.

9. Final report

post-pr-review summary for PR #123 (owner/repo)
  agents dispatched   <csv>  (profile: docs-only | small-code | full | --agents override)
  receipt             posted | skipped (dry-run)
  posted              N
  skipped (dedup)     M
  skipped (out-diff)  K
  deferred (rate)     R
  errors              E
PR: https://github.com/owner/repo/pull/123

Inline-review heuristics (when --agents inline)

Use these only when no agents are requested:

• Silent failures: scan added lines for catch.*{}, catch.*{\s*//, except.*pass, panic\(\) //ignore. Severity: critical.
• Missing tests: any new file under src/ (or language equivalent) with no matching test file in the diff. Severity: important.
• Comment drift: existing comments above changed code. Severity: suggestion.
• Hard-coded secrets: API_KEY = "...", password.*=.*"...", regex matches against added lines. Severity: critical.

Idempotency

Every comment body has a hidden HTML marker appended:

<!-- post-pr-review:v1:<sha256-hex-truncated-to-16> -->

The hash input (the "finding key") is the literal pipe-joined string:

{path}|{line}|{side}|{agent}|{first_120_chars_of_body_normalized}

normalized = lowercase + collapse whitespace runs to single space + strip trailing punctuation.

On re-run, post-pr-review-list-markers.sh paginates all existing comments, extracts every post-pr-review:v1:[0-9a-f]{16} marker, and drops any new finding whose marker is in the set. The v1 prefix lets us bump the schema later without false-positive dedup against old markers.

Rate-limit guard

GitHub's secondary limit on content-creating endpoints is 80 requests/min, 500/hour. Source: https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api#about-secondary-rate-limits. See references/rate-limits.md.

The atomic POST /reviews endpoint counts as ONE content-creation regardless of how many inline comments it bundles — this is why --mode review is the default. --mode inline should only be used when posting a single comment or when the user wants per-comment posting for some reason.

Concurrency

The dedup story (step 6.5: list existing markers → filter findings → POST) is not safe under concurrent invocations. Between list-markers.sh returning and post-batch.sh posting, another /post-pr-review run (or a human reviewer adding bot-style markers) could insert markers we never saw, producing duplicate posts on the next run.

Run the skill serially per PR. If you need automation, gate it behind a GitHub Actions concurrency group or a repo-level lock — the skill itself intentionally does not implement locking (out of scope for v0.1).

Failure modes

Mode	Detection	Behavior
No PR for branch	gh pr view exits non-zero	Fail with: "No PR found. Pass PR# explicitly: /post-pr-review 123."
PR closed/merged	state != OPEN	Refuse. No --force for merged.
PR is draft	isDraft == true	Warn + prompt; --auto proceeds.
Fork PR	isCrossRepository == true	Warn that posting may 403.
Agent returns no findings	empty findings[]	Note in report, continue.
Agent returns malformed JSON	parse fails	Retry once with stricter prompt; on second fail, skip that agent.
gh auth missing	gh auth status non-zero	Fail at preflight with remediation.
Secondary rate limit hit	403 with secondary rate limit in body	Stop batch, report deferred, hint at x-ratelimit-reset.
Line not in diff	pre-validated against postable line set; if it slips and GitHub returns 422	Skip that comment, log, continue.
Network/transient	non-2xx, non-422, non-403	Retry up to 2x with backoff (1s, 3s).
Receipt POST fails	post-pr-review-post-receipt.sh exits 1	Report it. The findings are already posted; the missing receipt only means a later /pr-conductor run cannot skip the review stage.

See also

• .claude/skills/review-pr/SKILL.md — the CONSUMER. Run after /post-pr-review to fetch and apply fixes from the comments this skill posted.
• references/agent-contract.md — JSON schema each dispatched agent returns.
• references/github-api-gotchas.md — NEW vs OLD line coords, atomic vs loop trade-offs.
• references/rate-limits.md — GitHub's content-creation limits.