test(ci): smoke-run the review workflow in CI - #27
Conversation
There was a problem hiding this comment.
Approach is sound: executing the file is the only check that closes the gap the two outages went through, and the dry run is correctly minimal — skipping the review step alone is enough because all three downstream steps already gate on its outputs or outcome. One non-blocking note inline about the concurrency group on the push-to-main path.
There was a problem hiding this comment.
Reviewed the dry-run gating end to end: on the smoke path inputs.dry_run is true so the claude-code-action step is skipped, and each step after it is gated on steps.review.outputs.execution_file != '', steps.tool-usage.outcome == 'success', or steps.review.outcome being 'failure'/'cancelled' — all false when the step is 'skipped', so no review, comment or artifact is written. On the production pull_request path inputs is empty, so !inputs.dry_run is true and the concurrency group resolves to pr-review-live-<n>, distinct from the smoke group.
One prior nit (no fallback key in the concurrency group on the main-push path) is still open, and one new nit on the duplicated permission list. Neither blocks.
| # | ||
| # Checked as a property of the expression rather than by evaluating it: the PR number must be | ||
| # followed by a `||` fallback, so the group stays unique when there is no pull request. | ||
| group_line=$(grep -n '^ group:' "$WORKFLOW_FILE" | head -1) |
There was a problem hiding this comment.
nit: under set -euo pipefail this assignment makes the guard below unreachable. (not blocking)
If ^ group: is gone from the workflow, grep exits 1, pipefail propagates it through head, and the exit status of an assignment is the status of its command substitution — so set -e terminates the script right here. The FAIL ... this check proves nothing branch on line 113 never runs.
The result is still a red CI, so this isn't a silent pass. But it exits after ok .../*.yml expressions are well formed with no further output, which reads like the harness died rather than like the assertion it is — and this file's own habit is that a check which can no longer prove anything says so. The other extraction of this kind (line 144) is fine because awk exits 0 on no match.
| group_line=$(grep -n '^ group:' "$WORKFLOW_FILE" | head -1) | |
| group_line=$(grep -n '^ group:' "$WORKFLOW_FILE" | head -1 || true) |
There was a problem hiding this comment.
Both prior threads are addressed in code: the concurrency group now falls back to github.run_id and workflow-lint-test.sh asserts the fallback exists, and the permission parity check compares name and value across both files (verified by reading the two permissions: blocks — the awk terminates on steps: and uses: respectively, and both yield the same seven pairs).
One nit left inline on the new group check's error path. Tests / test is green; Tests / Review workflow starts / review had not reported when this review started.
An org ruleset injects this review workflow into every repo resolved from
main, so a PR here is reviewed by main's copy and never by the copy it changes. Twice today a file Actions refuses to parse merged green — an empty expression delimiter in a shell comment, then arun:block over the 21,000-character expression limit — because nothing in CI had executed the version under review.tests.ymlnow calls the review workflow withuses: ./anddry_run: true, which resolves the PR's own copy. If Actions cannot load it, the job cannot start and CI is red. Only theclaude-code-actionstep is skipped; the steps after it are already gated on its outcome or outputs, so no review, comment or artifact is written.