-
Notifications
You must be signed in to change notification settings - Fork 0
94 lines (84 loc) · 4.21 KB
/
Copy pathtests.yml
File metadata and controls
94 lines (84 loc) · 4.21 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
name: Tests
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2
with:
fetch-depth: 1
# actionlint is the check that would have caught the outage, so it has to actually run
# here: it is not on the runner image, and without this step the lint test reports
# "skip actionlint not installed" forever and only its own awk backstop runs.
#
# Downloaded and checksummed rather than pulled in as a third-party action, because
# this repository's workflows are what gate every merge in the org -- an unpinned
# `uses:` here would be a supply-chain hole in the one place it hurts most.
- name: Install actionlint
env:
ACTIONLINT_VERSION: 1.7.12
ACTIONLINT_SHA256: 8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8
run: |
curl -fsSL -o actionlint.tar.gz \
"https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz"
echo "${ACTIONLINT_SHA256} actionlint.tar.gz" | sha256sum --check --strict
tar -xzf actionlint.tar.gz actionlint
sudo install -m 0755 actionlint /usr/local/bin/actionlint
rm -f actionlint actionlint.tar.gz
actionlint --version
# First, because it is the check whose absence took the org down: a workflow that
# Actions cannot parse never starts, so the required check never reports and every pull
# request in the org blocks. Nothing else in this suite can see that -- the file is
# valid YAML and the shell runs it happily.
- name: Workflow validity
run: tests/workflow-lint-test.sh
- name: Review cycle counter
run: tests/review-cycle-test.sh
- name: Tool usage projection
run: tests/tool-usage-test.sh
- name: Frontloaded PR context
run: tests/pr-context-test.sh
- name: Context step end to end
run: tests/context-step-test.sh
# Runs claude-pr-review.yml itself, from this commit, with the review step skipped. The checks
# above all read the workflow as text; this one hands it to Actions and asks whether it starts.
#
# That is the gap the last two outages went through. An org ruleset injects the review workflow
# into every repo resolved from `main`, so the review on a pull request here comes from main's
# copy -- never from the copy the pull request is changing. Both times, a file Actions refuses
# to parse merged with this suite green, and the org lost reviews until someone noticed: an
# empty expression delimiter inside a shell comment, then a `run:` block over the
# 21,000-character expression limit. Neither is visible to yaml.safe_load, to bash, or to
# actionlint, and enumerating the next limit ahead of time is a game with no end. Executing the
# file has no such gap -- if Actions will not load it, this job cannot start, and CI is red.
#
# `uses: ./` resolves from the calling commit rather than from the default branch, which is what
# makes this the pull request's copy and not main's.
smoke:
name: Review workflow starts
# The same set the called job declares. A caller cannot grant a reusable workflow more than
# it holds, and the point of this job is to exercise the real thing: the nine context reads
# each need their permission, and a missing one degrades silently into a "could not read"
# sentence rather than failing. pull-requests: write is unused on this path -- the only two
# steps that write are skipped with the review step -- but it is what production runs with,
# and a smoke test that runs with a different token is testing a different workflow.
permissions:
contents: read
pull-requests: write
id-token: write
actions: read
issues: read
checks: read
statuses: read
uses: ./.github/workflows/claude-pr-review.yml
with:
dry_run: true
# The GitHub App private key, for the token step and the cross-repo prompt checkout. Both run
# in a dry run, so a broken sparse-checkout or an expired key surfaces here rather than in the
# org.
secrets: inherit