IN LIST: index 32- and 64-bit integers by byte

[geoffreyclaude]

Which issue does this PR close?

• Part of Further improve performance of IN list evaluation #19241.
• Stacked on #24088 and #24102, which follow #23014.

Note

This description covers only the final commit, 689cae2a42, relative to the combined lower-stack head b0443b6bb0. The earlier commits visible in this stacked PR belong to the PRs above.

Rationale for this change

DataFusion builds a reusable static filter when every expression in an IN list is constant. The lower PRs in this stack already use a branchless comparison chain for short primitive lists. Once that existing path's limit is exceeded—32 non-null list entries for Int32/UInt32, or 16 for Int64/UInt64—the integer filters previously used a general-purpose hash-table lookup for each ordinary primitive-array input value.

A hash table works for every value distribution, but an integer constant set often contains a cheaper index: one of the integer's bytes may divide the set into very small groups. If every possible value of that byte identifies at most four list members, membership can be evaluated as:

input integer
    -> extract one selected byte
    -> directly select 1 of 256 buckets
    -> compare with the bucket's 4 full-width integers

This replaces hashing and hash-table probing with one table lookup and a fixed amount of comparison work. The index is constructed once with the static filter and reused across input batches.

What changes are included in this PR?

Where this fits in strategy selection

This commit does not change the existing small-list thresholds. For constant IN lists, normal routing remains:

Type	Existing short-list strategy	Larger-list strategy after this commit
Int32, UInt32	Branchless comparisons for up to 32 non-null entries	New IntegerSet
Int64, UInt64	Branchless comparisons for up to 16 non-null entries	New IntegerSet

The branchless cutoff counts non-null list entries before deduplication. IntegerSet then deduplicates those entries, so its admission rules below use the number of distinct non-null values.

Building the byte index

IntegerSet builds the indexed representation as follows:

1. Insert all non-null IN-list values into DataFusion's existing HashSet. This removes duplicates without changing membership semantics and also prepares the fallback representation.
2. Consider indexing only when there are 1–1,024 distinct values. The upper bound follows directly from 256 byte values × 4 slots per bucket.
3. Starting at the least-significant byte, examine all four byte positions of a 32-bit value or all eight positions of a 64-bit value.
4. For each position, count how many distinct values have each of its 256 possible byte values.
5. Select the first position for which every bucket contains at most four values.
6. Allocate 256 buckets of four full-width integers and place every set member in the bucket selected by that byte.

For example, 0, 256, 512, 768, and 1024 all have the same lowest byte, so the lowest-byte bucket would need five slots and cannot be used. Their next byte differs, so that byte provides a valid index.

The byte is extracted numerically with shifts and a mask rather than read from memory, so selection is independent of machine endianness. Signed values are partitioned by their bit representation, but the final membership checks still compare the original full-width signed or unsigned values.

Padding buckets safely

Lookup always compares four candidates, even when a bucket contains fewer than four real members. To avoid storing a length for every bucket or branching on it, construction initializes every slot with an arbitrary real member of the set, then overwrites the populated slots.

This padding cannot create a false match:

• A query equal to the padding value necessarily has the same selected byte, so it routes to that value's real bucket, where the value genuinely is a member.
• A query routed to any other bucket differs from the padding value in the selected byte and therefore cannot equal it.

The selected byte only narrows the candidates. All four final checks use complete 32- or 64-bit equality, so sharing a bucket is not itself a match.

Probing the index

For each input value, the indexed path:

1. extracts the selected byte;
2. directly loads that byte's four-entry bucket;
3. compares the input with all four full-width candidates; and
4. combines the comparisons with bitwise |.

In simplified form:

[a, b, c, d] = buckets[selected_byte(input)]
found = (input == a) | (input == b) | (input == c) | (input == d)

Using | rather than short-circuiting || makes the work independent of which slot matches and exposes a regular, branch-free four-comparison expression that is amenable to compiler auto-vectorization. The implementation requires no target-specific SIMD intrinsics; whether the compiler emits SIMD remains target- and compiler-dependent. Even without vectorization, the probe is bounded to one direct lookup and four comparisons.

Hash fallback

IntegerSet retains the already-built HashSet when:

• the distinct set is empty;
• it contains more than 1,024 values; or
• every byte position has at least one bucket containing more than four values.

The last case matters for correlated or adversarial distributions: having at most 1,024 values is necessary for the fixed table to hold the set, but it is not sufficient for any single byte to partition it well. Because construction creates the HashSet first, fallback reuses it rather than rebuilding the set.

Construction and memory cost

The indexed representation contains exactly 1,024 full-width slots: a 4 KiB bucket payload for Int32/UInt32, or 8 KiB for Int64/UInt64, plus small enum and allocation metadata. This is a fixed cost even when many slots are padding.

Construction still pays the previous one-time hash/deduplication cost, then scans the distinct values for at most four or eight candidate byte positions. While constructing an accepted index, peak memory briefly includes both the hash set and the fixed table; the hash set is dropped when the indexed representation is returned. The tradeoff is therefore a small amount of one-time work and bounded memory for cheaper repeated probes.

Preserving IN / NOT IN semantics

The index changes only how raw membership is computed. Its membership bitmap is passed to the shared build_result_from_contains helper, which applies SQL three-valued logic:

• a null input value produces NULL;
• a matching non-null value produces TRUE for IN and FALSE for NOT IN, even when the list also contains NULL;
• a non-matching value produces NULL when the list contains NULL; and
• otherwise, a non-match produces FALSE for IN and TRUE for NOT IN.

Duplicate non-null values are removed before indexing, while list nulls are tracked separately, so neither changes the result. Existing handling for sliced arrays and dictionary-encoded inputs is retained; dictionary values are evaluated and then remapped through their keys.

Related primitive-filter cleanup

To plug IntegerSet into the existing primitive filters without duplicating result logic, this commit also:

• generalizes the primitive-filter macro so each type can provide its set representation, constructor, and batched membership function;
• routes precomputed membership bitmaps through the common result builder; and
• replaces the custom Float32/Float64 hash-key wrappers with [u8; 4] and [u8; 8] keys.

Floats remain hash-backed; they do not use the integer byte index. Their fixed byte-array keys preserve the previous exact bit-pattern equality, including the distinction between 0.0 and -0.0 and between different NaN payloads.

Scope

• The new byte index applies only to constant/static filters for Int32, UInt32, Int64, and UInt64.
• The existing branchless strategy continues to handle their short lists.
• Narrow integers keep their existing bitmap strategies.
• Floats and temporal types with 32- or 64-bit physical representations are not routed through IntegerSet by this change.
• Dynamic, non-constant IN lists and all other data types keep their existing strategies.
• No public API or SQL-visible behavior changes.

Are these changes tested?

Yes. The tests added by the final commit:

• compare IntegerSet membership with std::collections::HashSet over empty, small, and larger generated inputs;
• cover minimum and maximum values, negative values, misses adjacent to bounds, and duplicates for all four supported integer types;
• verify that an index can select a higher byte when the lowest byte is overfull;
• force a distribution-based hash fallback with 625 distinct values for which every byte position has an overfull bucket;
• exercise a sliced, nullable UInt32 list with dictionary-encoded needles for both IN and NOT IN; and
• verify that the refactored Float32 keys preserve exact bit equality for 0.0/-0.0 and distinct NaN payloads.

Are there any user-facing changes?

No. This is an internal representation and evaluation optimization for constant integer IN lists. Query results and public APIs are unchanged.

Local benchmark snapshot

This final commit does not modify the benchmark source. Both revisions use the same existing datafusion/physical-expr/benches/in_list_strategy.rs inherited from the lower stack, so the comparison isolates the integer-filter implementation. The benchmark was built and run in separate target directories after compilation completed:

cargo bench --target-dir <target-dir> \
  -p datafusion-physical-expr \
  --bench in_list_strategy -- '<filter>' --noplot

Criterion defaults and central point estimates were used. Filter construction is outside the timed loop, so these measurements cover repeated evaluation rather than index-construction cost. Each timed iteration evaluates 8,192 input rows. Lower is better.

Before is the exact combined head of #24088 and #24102 (b0443b6bb0); After is this PR (689cae2a42).

The nine reported cases improve 1.85x–5.31x, with a 3.32x geometric mean speedup (69.9% less time).

Benchmark	Before	After	Change
i32 / 64 / miss	17.13 us	6.53 us	-61.9% (2.62x)
i32 / 64 / 50% hit	34.77 us	6.55 us	-81.2% (5.31x)
i32 / 256 / miss	15.28 us	6.56 us	-57.1% (2.33x)
i32 / 256 / 50% hit	32.20 us	6.54 us	-79.7% (4.92x)
i64 / 32 / miss	13.92 us	7.53 us	-45.9% (1.85x)
i64 / 32 / 50% hit	30.59 us	7.53 us	-75.4% (4.06x)
i64 / 128 / miss	16.23 us	7.76 us	-52.2% (2.09x)
i64 / 128 / 50% hit	31.91 us	7.66 us	-76.0% (4.16x)
nullable i32 / 64 / 50% hit / 20% null	32.01 us	6.79 us	-78.8% (4.72x)

The baseline and this PR were measured without competing Cargo or rustc processes.

These cases cover random full-width signed-integer distributions. The reported set does not measure filter construction, deliberately forced hash fallback, unsigned integers, or a NULL inside the IN list; the nullable row has nulls in the input array.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 86.28049% with 90 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.07%. Comparing base (426b351) to head (689cae2).
⚠️ Report is 33 commits behind head on main.

Files with missing lines	Patch %	Lines
...rc/expressions/in_list/fixed_size_binary_filter.rs	84.72%	15 Missing and 27 partials ⚠️
...l-expr/src/expressions/in_list/byte_view_filter.rs	85.31%	9 Missing and 17 partials ⚠️
...atafusion/physical-expr/src/expressions/in_list.rs	76.31%	2 Missing and 7 partials ⚠️
...l-expr/src/expressions/in_list/primitive_filter.rs	79.54%	4 Missing and 5 partials ⚠️
...ysical-expr/src/expressions/in_list/integer_set.rs	97.59%	0 Missing and 2 partials ⚠️
.../physical-expr/src/expressions/in_list/strategy.rs	92.85%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24181      +/-   ##
==========================================
+ Coverage   80.91%   81.07%   +0.16%     
==========================================
  Files        1102     1109       +7     
  Lines      377102   382850    +5748     
  Branches   377102   382850    +5748     
==========================================
+ Hits       305143   310410    +5267     
- Misses      53769    54069     +300     
- Partials    18190    18371     +181     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.