Skip to content

test(ci): smoke-run the review workflow in CI - #27

Merged
zfarrell merged 5 commits into
mainfrom
feat/workflow-smoke-test
Aug 5, 2026
Merged

test(ci): smoke-run the review workflow in CI#27
zfarrell merged 5 commits into
mainfrom
feat/workflow-smoke-test

Conversation

@zfarrell

@zfarrell zfarrell commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

An org ruleset injects this review workflow into every repo resolved from main, so a PR here is reviewed by main's copy and never by the copy it changes. Twice today a file Actions refuses to parse merged green — an empty expression delimiter in a shell comment, then a run: block over the 21,000-character expression limit — because nothing in CI had executed the version under review.

tests.yml now calls the review workflow with uses: ./ and dry_run: true, which resolves the PR's own copy. If Actions cannot load it, the job cannot start and CI is red. Only the claude-code-action step is skipped; the steps after it are already gated on its outcome or outputs, so no review, comment or artifact is written.

@zfarrell
zfarrell requested a review from a team as a code owner August 5, 2026 18:30
@zfarrell
zfarrell requested review from eddietejeda and removed request for a team August 5, 2026 18:30
Comment thread .github/workflows/claude-pr-review.yml Outdated
claude[bot]
claude Bot previously approved these changes Aug 5, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach is sound: executing the file is the only check that closes the gap the two outages went through, and the dry run is correctly minimal — skipping the review step alone is enough because all three downstream steps already gate on its outputs or outcome. One non-blocking note inline about the concurrency group on the push-to-main path.

Comment thread .github/workflows/tests.yml
claude[bot]
claude Bot previously approved these changes Aug 5, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the dry-run gating end to end: on the smoke path inputs.dry_run is true so the claude-code-action step is skipped, and each step after it is gated on steps.review.outputs.execution_file != '', steps.tool-usage.outcome == 'success', or steps.review.outcome being 'failure'/'cancelled' — all false when the step is 'skipped', so no review, comment or artifact is written. On the production pull_request path inputs is empty, so !inputs.dry_run is true and the concurrency group resolves to pr-review-live-<n>, distinct from the smoke group.

One prior nit (no fallback key in the concurrency group on the main-push path) is still open, and one new nit on the duplicated permission list. Neither blocks.

#
# Checked as a property of the expression rather than by evaluating it: the PR number must be
# followed by a `||` fallback, so the group stays unique when there is no pull request.
group_line=$(grep -n '^ group:' "$WORKFLOW_FILE" | head -1)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: under set -euo pipefail this assignment makes the guard below unreachable. (not blocking)

If ^ group: is gone from the workflow, grep exits 1, pipefail propagates it through head, and the exit status of an assignment is the status of its command substitution — so set -e terminates the script right here. The FAIL ... this check proves nothing branch on line 113 never runs.

The result is still a red CI, so this isn't a silent pass. But it exits after ok .../*.yml expressions are well formed with no further output, which reads like the harness died rather than like the assertion it is — and this file's own habit is that a check which can no longer prove anything says so. The other extraction of this kind (line 144) is fine because awk exits 0 on no match.

Suggested change
group_line=$(grep -n '^ group:' "$WORKFLOW_FILE" | head -1)
group_line=$(grep -n '^ group:' "$WORKFLOW_FILE" | head -1 || true)

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both prior threads are addressed in code: the concurrency group now falls back to github.run_id and workflow-lint-test.sh asserts the fallback exists, and the permission parity check compares name and value across both files (verified by reading the two permissions: blocks — the awk terminates on steps: and uses: respectively, and both yield the same seven pairs).

One nit left inline on the new group check's error path. Tests / test is green; Tests / Review workflow starts / review had not reported when this review started.

@zfarrell
zfarrell merged commit a21b338 into main Aug 5, 2026
3 checks passed
@zfarrell
zfarrell deleted the feat/workflow-smoke-test branch August 5, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant