Skip to content

Add plan-native scan execution - #9240

Draft
joseph-isaacs wants to merge 1 commit into
vortex-plan-zonedfrom
vortex-plan-scan-v2
Draft

Add plan-native scan execution#9240
joseph-isaacs wants to merge 1 commit into
vortex-plan-zonedfrom
vortex-plan-scan-v2

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a separate vortex-scan-v2 crate copied from the existing scan orchestration and backed only by physical plans
  • bind raw projection and filter expressions once at the scan-builder boundary, then optimize and execute only BoundExpression plan nodes
  • extend ExpressionRowIdxRule from Add plan optimizer rules and push expressions #9196's zero-partition reduction to row-index-only, child-values-only, and mixed row-index/data expressions
  • add plan execution for flat, chunked, dictionary, struct, list, row-index, expression, and zoned nodes; expression execution uses apply_bound
  • preserve the row-count endpoint when split discovery reaches a childless Pack, so an empty projection still schedules the correct scan range
  • add plan-native split discovery plus debug/trace logging for source layouts, optimized plans, split ranges, and task results
  • add synthetic end-to-end tests and a TPC-H example that compares every result value with the existing LayoutReader scan

Boundary of the COUNT(*) fix

The optimizer-side zero-reference reduction is in #9196. This execution PR owns the childless-Pack split behavior and a direct childless_pack_preserves_its_row_count regression. The later pruning PR #9244 contains the end-to-end assertion that all rows are returned without any struct-field segment reads.

Current scope

  • zoned plans execute their data child but do not perform zone-map pruning yet
  • the existing scanner's dynamic conjunct ordering, reader caching, and pruning behavior are not copied into this correctness-first path

Validation

  • cargo test -p vortex-layout -p vortex-scan-v2 — 233 layout tests and 4 scan-v2 tests passed
  • cargo +nightly fmt --all -- --check
  • cargo clippy --all-targets --all-features -- -D warnings
  • git diff --check

Stacked on #9187.

@codspeed-hq

codspeed-hq Bot commented Aug 6, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 1959 untouched benchmarks
⏩ 89 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation decompress[u64, (10000, 4)] 310.2 µs 403.7 µs -23.16%
Simulation in_place_non_nullable[65536] 65.9 µs 49.1 µs +34.19%
Simulation cold_misaligned[(64, 256)] 5.2 ms 4.4 ms +17.79%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing vortex-plan-scan-v2 (b157ff9) with vortex-plan-zoned (8191486)2

Open in CodSpeed

Footnotes

  1. 89 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on vortex-plan-zoned (8798f72) during the generation of this report, so 92585db was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@joseph-isaacs joseph-isaacs added the changelog/feature A new feature label Aug 7, 2026
@joseph-isaacs
joseph-isaacs force-pushed the vortex-plan-scan-v2 branch 2 times, most recently from fba0962 to 648d126 Compare August 7, 2026 15:33
@joseph-isaacs
joseph-isaacs force-pushed the vortex-plan-scan-v2 branch 3 times, most recently from 9260967 to c7abf62 Compare August 10, 2026 10:16
@joseph-isaacs
joseph-isaacs force-pushed the vortex-plan-scan-v2 branch 2 times, most recently from 9024c43 to ecff2cc Compare August 10, 2026 13:47
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant