Retire seed_times; visit only the keys with work due

[frankmcsherry]

Removes seed_times from ProxyReduceBackend by presenting the novel input run apart from the accumulated one, and narrows the set of keys a retire visits to those with work actually due in the interval.

The second of those is worth 2-3x on SCC over the corgi backend, growing with n.

Why seed_times existed

next_window handed back one input bridge holding the accumulated history and the novel delta consolidated together, which loses which records were novel. A novel record that nets to zero against a history record compaction has advanced onto its time disappears, and its interesting time with it. The seeds therefore had to come from a second, separate read of the delta.

The conventional reduce has never had this problem: it keeps three cursors, and the batch history is loaded unadvanced and is the only thing that makes a time interesting. The window now carries the same three runs, so the seeds are the novel run's own times, read where they already are. The two input runs combine only in the per-moment accumulation, never as presentations.

Protocol

next_window now takes from: &mut Option<u64> — a key hash rather than an index into a list the harness built — on the same contract as ProxyJoinBackend::advance: strictly increase it or report the key space exhausted, and report a key entirely within the window that first mentions it. Both are debug_asserted. The window is caller-owned and refilled, so its three runs keep their allocations.

Neither side now scans the whole key space. The harness contributes the keys it knows must be revisited; the backend adds the ones its novel batches touch, which it discovers in the scan the presentation needs anyway.

Where the time went

Not the seeding pass. changed was every pending key and is now only the keys with an interesting time due in [lower, upper). A key whose times all lie at or beyond upper was being gathered into the presentation — history rows, output rows — and walked through determination on every retire, only to produce no moments.

The conventional tactic never did this: cursors::CursorTactic filters to !upper.less_equal(time) and takes the no-work branch when nothing is left, and the model-derived reference::ReferenceTactic does the same. This change brings the proxy tactic in line with both.

Carrying the times of unvisited keys is now explicit, since a key that is not changed is never presented and would otherwise be forgotten. The split against upper happens up front: due times drive determination, carried times seed the next round's pending set directly, and a key with both unions them. The old no-work early return returned an empty frontier, which was safe only while changed held every pending key; the restored form additionally requires the novel batches to be empty and returns the frontier over the untouched pending set.

Measurements

ddir on the corgi backend, single worker, both binaries built in the same target directory. Binary layout between build trees moves these programs by more than the effect, so vec is carried as a control — it uses the cursor tactic and is untouched by this PR.

program	vec	corgi before	corgi after	ratio before	ratio after
scc n=1000 e=2000 batch=100 rounds=100	3.91s	12.38s	4.86s	3.17x	1.25x
scc n=2000 e=4000 batch=100 rounds=60	4.55s	18.51s	5.62s	4.07x	1.24x
reach n=20000 e=40000 batch=200 rounds=100	290ms	297ms	292ms	1.02x	1.01x
adt n=2000 e=4000 batch=100 rounds=100	59ms	93ms	94ms	1.58x	1.60x
unnest n=1000 e=2000 batch=50 rounds=100	1.08s	1.14s	1.10s	1.06x	1.02x
tour n=200 e=400 batch=20 rounds=50	181ms	194ms	192ms	1.07x	1.06x

The corgi/vec spread across these programs was [0.97x, 3.17x] and is now [0.98x, 1.62x]. SCC was the outlier and it collapsed; nothing else moved, which is what to expect if the redundancy only bit where a large pending set is carried across retires. adt is now the widest gap and did not move, so whatever it is, it is not this.

This bears on int-proxy-columnar-findings.md, which attributed the 2.24-2.59x SCC corgi/vec gap to PointStamp allocation and corgi's structural compare and concluded "framework is a tie". On this shape most of it was the tactic re-presenting and re-walking every pending key.

Testing

DD has no int_proxy test, so the gate is cargo test -p interactive --test corgi_backend — 12 programs asserting the corgi and vec backends agree — run in debug, where debug_assert_sorted_bridge and the reduce_with_tactic contract checks are live. Green, along with the DD suite.

Given the size of the speedup I also compared program output directly rather than relying on the gate: byte-identical to master-next across scc, reach, tour, adt, binders and unnest, on both backends, at several sizes (5,624 lines on tour, 2,332 on unnest).

Also measured and rejected

Advancing the accumulated input and output presentations by instance.lower in the corgi backend, as its join backend already does in stage_runs. The reasoning holds — every moment is at or above lower, so advancing preserves each record's downset while letting the consolidation net more — but the driver already sets logical compaction to upper on both traces each activation, so the times largely arrive advanced and the pass is work for nothing. On scc it was a stable 5.09-5.10s against 4.85-4.89s, about 4% worse, and flat elsewhere. ReduceInstance::lower is consequently still unread by the only backend.

Open

The multi-window path is unexercised. corgi covers the key space in a single call, so the harness code that runs on a second window — the cursor walking changed across windows, the merge that forms each window's key list, and both new debug_asserts — has never executed. A ProxyReduceBackend over a conventional OrdValBatch trace would exercise it and would also let the tactic be benchmarked against the cursor tactic on identical storage, which nothing currently does.

Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com