Skip to content

Commit 8dac1c5

Browse files
vincentkocvincentkoc
committed
fix(ci): harden release evidence publication
1 parent ef0c28e commit 8dac1c5

7 files changed

Lines changed: 936 additions & 490 deletions

‎.github/workflows/openclaw-release-evidence-from-full-validation.yml‎

Lines changed: 63 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,12 @@ on:
1212
required: true
1313
type: string
1414
release_ref:
15-
description: Optional public OpenClaw release ref, tag, branch, or SHA
16-
required: false
17-
default: ""
15+
description: Exact public OpenClaw release ref, tag, or SHA
16+
required: true
1817
type: string
1918
package_spec:
20-
description: Optional package spec covered by this evidence, for example openclaw@2026.4.27-beta.1
21-
required: false
22-
default: ""
19+
description: Exact package spec covered by this evidence, for example openclaw@2026.4.27-beta.1
20+
required: true
2321
type: string
2422
notes:
2523
description: Optional release-manager notes to append to the evidence markdown
@@ -31,7 +29,7 @@ on:
3129
- openclaw_full_release_validation_completed
3230

3331
concurrency:
34-
group: openclaw-release-evidence-from-full-validation-${{ github.event_name == 'repository_dispatch' && github.event.client_payload.release_id || inputs.release_id }}
32+
group: openclaw-release-evidence-${{ github.event_name == 'repository_dispatch' && github.event.client_payload.release_id || inputs.release_id }}
3533
cancel-in-progress: false
3634

3735
env:
@@ -45,19 +43,22 @@ env:
4543

4644
jobs:
4745
write_evidence:
46+
if: github.ref == 'refs/heads/main'
47+
environment: release-evidence
4848
runs-on: ubuntu-latest
4949
permissions:
5050
actions: read
5151
contents: write
5252
steps:
5353
- name: Checkout release repo
54-
uses: actions/checkout@v6
54+
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
5555
with:
5656
fetch-depth: 0
57-
persist-credentials: false
57+
ref: main
58+
token: ${{ github.token }}
5859

5960
- name: Setup Node.js
60-
uses: actions/setup-node@v6
61+
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
6162
with:
6263
node-version: ${{ env.NODE_VERSION }}
6364

@@ -69,7 +70,7 @@ jobs:
6970
7071
- name: Generate release evidence
7172
env:
72-
GH_TOKEN: ${{ github.token }}
73+
GH_TOKEN: ${{ secrets.OPENCLAW_RELEASES_PRIVATE_PUSH_TOKEN }}
7374
run: |
7475
set -euo pipefail
7576
node scripts/openclaw-release-evidence-from-full-validation.mjs \
@@ -90,31 +91,64 @@ jobs:
9091
} >> "$GITHUB_STEP_SUMMARY"
9192
9293
- name: Commit evidence to main
93-
env:
94-
PUSH_TOKEN: ${{ secrets.OPENCLAW_RELEASES_PRIVATE_PUSH_TOKEN }}
9594
run: |
9695
set -euo pipefail
9796
evidence_path="evidence/${RELEASE_ID}"
98-
if [[ -z "${PUSH_TOKEN// }" ]]; then
99-
echo "::error::OPENCLAW_RELEASES_PRIVATE_PUSH_TOKEN is required to push release evidence directly to main."
100-
exit 1
101-
fi
97+
node scripts/openclaw-release-evidence-contract.mjs verify-evidence \
98+
"${evidence_path}/release-evidence.json" "$RELEASE_ID" "$RELEASE_REF" "$PACKAGE_SPEC" \
99+
--require-full-validation
102100
git config user.name "openclaw-release-bot"
103101
git config user.email "release-bot@openclaw.ai"
104102
git add "${evidence_path}"
103+
generated_tree="$(git write-tree)"
104+
expected_tree="$(git rev-parse "${generated_tree}:${evidence_path}")"
105+
changed=0
105106
if git diff --cached --quiet; then
106107
echo "No evidence changes to commit."
107-
exit 0
108+
else
109+
git commit -m "Evidence: record ${RELEASE_ID}"
110+
changed=1
108111
fi
109-
git commit -m "Evidence: record ${RELEASE_ID}"
110-
git remote set-url origin "https://x-access-token:${PUSH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
111-
for attempt in 1 2 3 4 5; do
112-
git fetch origin main
113-
if git rebase origin/main && git push origin HEAD:main; then
114-
exit 0
112+
113+
if [[ "$changed" == "1" ]]; then
114+
pushed=0
115+
for attempt in 1 2 3 4 5; do
116+
git fetch origin main
117+
if git rebase origin/main; then
118+
if [[ "$(git rev-parse "HEAD:${evidence_path}")" != "$expected_tree" ]]; then
119+
echo "::error::Rebase changed the generated evidence directory."
120+
exit 1
121+
fi
122+
if git push origin HEAD:main; then
123+
pushed=1
124+
break
125+
fi
126+
fi
127+
git rebase --abort || true
128+
sleep $((attempt * 5))
129+
done
130+
if [[ "$pushed" != "1" ]]; then
131+
echo "::error::Could not push evidence to main after retries."
132+
exit 1
115133
fi
116-
git rebase --abort || true
117-
sleep $((attempt * 5))
118-
done
119-
echo "::error::Could not push evidence to main after retries."
120-
exit 1
134+
fi
135+
136+
git fetch origin main
137+
git merge-base --is-ancestor HEAD origin/main || {
138+
echo "::error::Published evidence commit is not reachable from origin/main."
139+
exit 1
140+
}
141+
remote_tree="$(git rev-parse "origin/main:${evidence_path}")"
142+
if [[ "$expected_tree" != "$remote_tree" ]]; then
143+
echo "::error::Remote evidence directory differs from the generated files."
144+
exit 1
145+
fi
146+
remote_json="$RUNNER_TEMP/openclaw-release-evidence/remote-release-evidence.json"
147+
git show "origin/main:${evidence_path}/release-evidence.json" > "$remote_json"
148+
cmp "${evidence_path}/release-evidence.json" "$remote_json" || {
149+
echo "::error::Remote evidence bytes differ from the generated JSON."
150+
exit 1
151+
}
152+
node scripts/openclaw-release-evidence-contract.mjs verify-evidence \
153+
"$remote_json" "$RELEASE_ID" "$RELEASE_REF" "$PACKAGE_SPEC" \
154+
--require-full-validation

‎.github/workflows/openclaw-release-evidence.yml‎

Lines changed: 67 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,12 @@ on:
88
required: true
99
type: string
1010
release_ref:
11-
description: Optional public OpenClaw release ref, tag, branch, or SHA
12-
required: false
13-
default: ""
11+
description: Exact public OpenClaw release ref, tag, or SHA
12+
required: true
1413
type: string
1514
package_spec:
16-
description: Optional package spec covered by this evidence, for example openclaw@2026.4.27-beta.1
17-
required: false
18-
default: ""
15+
description: Exact package spec covered by this evidence, for example openclaw@2026.4.27-beta.1
16+
required: true
1917
type: string
2018
runs:
2119
description: "One per line: <label> <owner/repo> <run-id> <blocking|advisory>"
@@ -34,22 +32,28 @@ concurrency:
3432
env:
3533
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
3634
NODE_VERSION: "24.x"
35+
PACKAGE_SPEC: ${{ inputs.package_spec }}
36+
RELEASE_ID: ${{ inputs.release_id }}
37+
RELEASE_REF: ${{ inputs.release_ref }}
3738

3839
jobs:
3940
write_evidence:
41+
if: github.ref == 'refs/heads/main'
42+
environment: release-evidence
4043
runs-on: ubuntu-latest
4144
permissions:
4245
actions: read
4346
contents: write
4447
steps:
4548
- name: Checkout release repo
46-
uses: actions/checkout@v6
49+
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
4750
with:
4851
fetch-depth: 0
49-
persist-credentials: false
52+
ref: main
53+
token: ${{ github.token }}
5054

5155
- name: Setup Node.js
52-
uses: actions/setup-node@v6
56+
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
5357
with:
5458
node-version: ${{ env.NODE_VERSION }}
5559

@@ -69,48 +73,79 @@ jobs:
6973
run: |
7074
set -euo pipefail
7175
node scripts/openclaw-release-evidence.mjs \
72-
--release-id "${{ inputs.release_id }}" \
73-
--release-ref "${{ inputs.release_ref }}" \
74-
--package-spec "${{ inputs.package_spec }}" \
76+
--release-id "$RELEASE_ID" \
77+
--release-ref "$RELEASE_REF" \
78+
--package-spec "$PACKAGE_SPEC" \
7579
--runs-file "$RUNNER_TEMP/openclaw-release-evidence/runs.txt" \
7680
--notes-file "$RUNNER_TEMP/openclaw-release-evidence/notes.md"
7781
7882
- name: Append run summary
7983
run: |
8084
set -euo pipefail
81-
evidence_md="evidence/${{ inputs.release_id }}/release-evidence.md"
85+
evidence_md="evidence/${RELEASE_ID}/release-evidence.md"
8286
{
8387
cat "$evidence_md"
8488
echo
8589
echo "Evidence path: \`$evidence_md\`"
8690
} >> "$GITHUB_STEP_SUMMARY"
8791
8892
- name: Commit evidence to main
89-
env:
90-
PUSH_TOKEN: ${{ secrets.OPENCLAW_RELEASES_PRIVATE_PUSH_TOKEN }}
9193
run: |
9294
set -euo pipefail
93-
evidence_path="evidence/${{ inputs.release_id }}"
94-
if [[ -z "${PUSH_TOKEN// }" ]]; then
95-
echo "::error::OPENCLAW_RELEASES_PRIVATE_PUSH_TOKEN is required to push release evidence directly to main."
96-
exit 1
97-
fi
95+
evidence_path="evidence/${RELEASE_ID}"
96+
node scripts/openclaw-release-evidence-contract.mjs verify-evidence \
97+
"${evidence_path}/release-evidence.json" "$RELEASE_ID" "$RELEASE_REF" "$PACKAGE_SPEC"
9898
git config user.name "openclaw-release-bot"
9999
git config user.email "release-bot@openclaw.ai"
100100
git add "${evidence_path}"
101+
generated_tree="$(git write-tree)"
102+
expected_tree="$(git rev-parse "${generated_tree}:${evidence_path}")"
103+
changed=0
101104
if git diff --cached --quiet; then
102105
echo "No evidence changes to commit."
103-
exit 0
106+
else
107+
git commit -m "Evidence: record ${RELEASE_ID}"
108+
changed=1
104109
fi
105-
git commit -m "Evidence: record ${{ inputs.release_id }}"
106-
git remote set-url origin "https://x-access-token:${PUSH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
107-
for attempt in 1 2 3 4 5; do
108-
git fetch origin main
109-
if git rebase origin/main && git push origin HEAD:main; then
110-
exit 0
110+
111+
if [[ "$changed" == "1" ]]; then
112+
pushed=0
113+
for attempt in 1 2 3 4 5; do
114+
git fetch origin main
115+
if git rebase origin/main; then
116+
if [[ "$(git rev-parse "HEAD:${evidence_path}")" != "$expected_tree" ]]; then
117+
echo "::error::Rebase changed the generated evidence directory."
118+
exit 1
119+
fi
120+
if git push origin HEAD:main; then
121+
pushed=1
122+
break
123+
fi
124+
fi
125+
git rebase --abort || true
126+
sleep $((attempt * 5))
127+
done
128+
if [[ "$pushed" != "1" ]]; then
129+
echo "::error::Could not push evidence to main after retries."
130+
exit 1
111131
fi
112-
git rebase --abort || true
113-
sleep $((attempt * 5))
114-
done
115-
echo "::error::Could not push evidence to main after retries."
116-
exit 1
132+
fi
133+
134+
git fetch origin main
135+
git merge-base --is-ancestor HEAD origin/main || {
136+
echo "::error::Published evidence commit is not reachable from origin/main."
137+
exit 1
138+
}
139+
remote_tree="$(git rev-parse "origin/main:${evidence_path}")"
140+
if [[ "$expected_tree" != "$remote_tree" ]]; then
141+
echo "::error::Remote evidence directory differs from the generated files."
142+
exit 1
143+
fi
144+
remote_json="$RUNNER_TEMP/openclaw-release-evidence/remote-release-evidence.json"
145+
git show "origin/main:${evidence_path}/release-evidence.json" > "$remote_json"
146+
cmp "${evidence_path}/release-evidence.json" "$remote_json" || {
147+
echo "::error::Remote evidence bytes differ from the generated JSON."
148+
exit 1
149+
}
150+
node scripts/openclaw-release-evidence-contract.mjs verify-evidence \
151+
"$remote_json" "$RELEASE_ID" "$RELEASE_REF" "$PACKAGE_SPEC"

‎README.md‎

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,13 @@ main-branch policy, and disabled administrator bypass aligned with the macOS
4848
release policy. Do not move signing or promotion secrets into repository-level
4949
secrets.
5050

51+
The evidence writer jobs use the `release-evidence` environment. It requires
52+
approval from `openclaw-release-managers`, permits deployments only from
53+
`main`, prevents self-review, and keeps administrator bypass disabled. Keep the
54+
environment free of secrets and variables; its approval gates durable evidence
55+
publication only and must not turn a pending publication into a failed upstream
56+
validation result.
57+
5158
## Release Evidence
5259

5360
The evidence workflows write release summaries under `evidence/<release-id>/`.
@@ -58,9 +65,10 @@ Each evidence directory contains:
5865
- `index.json`
5966
- `runs/<label>.json`
6067

61-
Evidence records include release ref provenance, npm package metadata, run URLs,
62-
workflow names, refs, SHAs, pass/fail state, timing summaries, artifact names,
63-
artifact sizes, and selected release performance summaries.
68+
Evidence records include release ref provenance, current npm and public GitHub
69+
release state, run URLs, workflow names, refs, SHAs, pass/fail state, timing
70+
summaries, artifact names, artifact sizes, and selected release performance
71+
summaries.
6472

6573
Evidence records do not store raw logs, provider payloads, live-channel
6674
transcripts, signing material, credentials, environment dumps, or downloaded
@@ -103,8 +111,16 @@ fail the release by itself.
103111
### Full Validation Ingest
104112

105113
`OpenClaw Release Evidence From Full Validation` takes a completed
106-
`openclaw/openclaw` full-validation run id, reads that parent run's logs,
107-
extracts child run ids, and writes the same evidence directory shape.
114+
`openclaw/openclaw` full-validation run id and reads its attempt-qualified
115+
`full-release-validation-<run-id>-<attempt>` manifest artifact. The ingest
116+
rejects missing, duplicate, expired, malformed, or identity-mismatched
117+
artifacts; child run ids and evidence-reuse provenance come only from the
118+
validated manifest.
119+
120+
Both evidence workflows require an exact package spec and release ref. They
121+
commit with the workflow's same-repository `github.token`, then verify that the
122+
commit is reachable from `origin/main`, the remote evidence directory is
123+
byte-identical, and its JSON still satisfies the release identity contract.
108124

109125
Manual ingest example:
110126

0 commit comments

Comments
 (0)