Skip to content

test: add hermetic CLI end-to-end coverage - #143

Merged
juangaitanv merged 6 commits into
mainfrom
ci/cli-pr-e2e
Aug 4, 2026
Merged

test: add hermetic CLI end-to-end coverage#143
juangaitanv merged 6 commits into
mainfrom
ci/cli-pr-e2e

Conversation

@juangaitanv

@juangaitanv juangaitanv commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Add hermetic real-binary coverage for upload, wait, list, inspect, failure, malicious-verdict, and git-SHA paths.
  • Add a mixed npm/Python monorepo fixture and verify deps scan, deps graph, and CycloneDX deps sbom output with the PR-built CLI.
  • Run the required E2E coverage through the existing Test workflow on pull requests and pushes to main.
  • Remove the private test-cli checkout and TEST_CLI_TOKEN dependency.

Why

Required CLI correctness checks should run against the exact source revision without cross-repository credentials, branch coordination, or private fixtures. The CLI repository already contained stronger fixture coverage for each individual ecosystem; only the nested mixed-ecosystem scenario needed to move.

Validation

  • ./harness ci: 608 tests passed with the 13% coverage gate
  • Pre-commit gate: strict Clippy, formatting, and 608 tests passed
  • cargo test --test cloud_commands_e2e -- --test-threads=1: 12 passed
  • cargo test --test cli_deps cli_mixed_monorepo_commands_cover_npm_and_pypi -- --exact: passed
  • git diff --check

@juangaitanv
juangaitanv requested a review from leenk7991 July 31, 2026 16:11
@juangaitanv
juangaitanv marked this pull request as ready for review July 31, 2026 16:43
Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml Outdated
@corgea-security corgea-security added the dennis-reviewed Dennis completed an automated review label Jul 31, 2026

@corgea-security corgea-security left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review risk: 5/5.

The workflow currently fails without TEST_CLI_TOKEN; configuring it would expose the credential and private fixtures to same-repository pull-request code.

Critical or high-priority changes must be addressed.

Automatic approval was not submitted: checks failed: rust-tests.

@corgea-security corgea-security left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review risk: 5/5.

The workflow exposes a private-repository credential to same-repository PR-controlled execution and currently fails trusted builds when that credential is unavailable.

Critical or high-priority changes must be addressed.

Automatic approval was not submitted: automated review found critical or high-priority findings.

@corgea-security corgea-security left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review risk: 5/5.

The workflow exposes a private-repository credential and fixtures to pull-request-controlled code, and currently fails trusted builds because the required secret is unavailable.

Critical or high-priority changes must be addressed.

Automatic approval was not submitted: automated review found critical or high-priority findings.

@juangaitanv juangaitanv changed the title test: run CLI fixtures on trusted pull requests test: add hermetic CLI end-to-end coverage Aug 3, 2026
@juangaitanv
juangaitanv merged commit 460716d into main Aug 4, 2026
17 checks passed
@juangaitanv
juangaitanv deleted the ci/cli-pr-e2e branch August 4, 2026 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dennis-reviewed Dennis completed an automated review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants