HIVE-25124: PTF: Vectorize cume_dist function

[ramitg254]

Change-Id: I4b03ac73b116d9e61f482745a2ff1f8f6ad1ab77

What changes were proposed in this pull request?

added support for vectorised cumulative distribution ptf

Why are the changes needed?

currently cume_dist() evaluates only in non vectorized manner so added evaluator for vectorized as well

Does this PR introduce any user-facing change?

No

How was this patch tested?

added q test as well as ci results

[Copilot]

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

This PR adds vectorized PTF support for cume_dist() in Hive, including a new vector evaluator and updated LLAP golden outputs plus new query-based coverage.

Changes:

• Add VectorPTFEvaluatorCumeDist and wire it into VectorPTFDesc/Vectorizer supported-function handling.
• Extend the vectorized PTF execution path to handle peer-group aggregated streaming (needed by cume_dist) and partition-size dependency.
• Add a new positive QTest and update multiple .q.out baselines to reflect newly vectorized plans.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
ql/src/java/org/apache/hadoop/hive/ql/plan/VectorPTFDesc.java	Registers cume_dist as a supported vectorized PTF function and adds helper to detect “purely streaming” evaluator sets.
ql/src/java/org/apache/hadoop/hive/ql/plan/PTFDesc.java	Reuses new “purely streaming” detection when reporting streaming capability.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java	Allows cume_dist through validation path similar to rank/dense_rank.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFOperator.java	Adjusts operator control flow to use “purely streaming” vs buffered partition finishing.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFGroupBatches.java	Adds peer-group row-count aggregation + partition-size propagation to support cume_dist precomputation.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFEvaluatorBase.java	Adds partition-size plumbing and APIs for peer-group aggregated streaming evaluators.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFEvaluatorCumeDist.java	Implements vectorized cume_dist using a two-pass approach (precompute per peer group, then stream results).
ql/src/test/queries/clientpositive/vector_ptf_cume_dist.q	New QTest covering vectorized and non-vectorized cume_dist behavior + vectorization explain.
ql/src/test/results/clientpositive/llap/vector_ptf_cume_dist.q.out	Golden output for the new vector_ptf_cume_dist.q test.
ql/src/test/results/clientpositive/llap/vectorized_ptf.q.out	Updates baseline explain output due to new vectorization behavior/reasons.
ql/src/test/results/clientpositive/llap/vector_windowing.q.out	Updates baseline to reflect cume_dist now being supported in vectorized PTF.
ql/src/test/results/clientpositive/llap/vector_windowing_rank.q.out	Updates baseline to reflect cume_dist now vectorizing in the tested plan.
ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out	Updates baseline supported-function list due to cume_dist addition.
ql/src/test/queries/clientpositive/sketches_rewrite_cume_dist.q	Temporarily disables vectorized PTF for a query with complex-type passthrough columns.
ql/src/test/queries/clientpositive/sketches_rewrite_cume_dist_partition_by.q	Same as above for the partition-by variant.
ql/src/test/results/clientpositive/llap/sketches_materialized_view_cume_dist.q.out	Baseline changes due to altered vectorization/execution mode after the feature change.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.0% Duplication on New Code

See analysis details on SonarQube Cloud