ptx: add benchmark - #13480
Conversation
|
Can we use file for benchmark and use |
Done. Removed the extra dependencies as well. I did the redirection earlier because running the benchmark locally was flooding the terminal. But I suppose that's a non-issue. |
|
GNU testsuite comparison: |
|
@sylvestre Hi, when you have a chance, could you please take a look at this PR? Thank you! Happy to make any changes |
Merging this PR will degrade performance by 8.1%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | df_with_path |
573.7 µs | 704.6 µs | -18.58% |
| ⚡ | Simulation | complex_relative_date |
330.2 µs | 318.3 µs | +3.74% |
| 🆕 | Memory | ptx_input_references_short_lines[100000] |
N/A | 222.2 MB | N/A |
| 🆕 | Memory | ptx_long_lines[100] |
N/A | 29.5 MB | N/A |
| 🆕 | Memory | ptx_short_lines[100000] |
N/A | 240.1 MB | N/A |
| 🆕 | Memory | ptx_input_references_long_lines[1000] |
N/A | 29.1 MB | N/A |
| 🆕 | Memory | ptx_tex[10000] |
N/A | 20.6 MB | N/A |
| 🆕 | Simulation | ptx_input_references_short_lines[100000] |
N/A | 16 s | N/A |
| 🆕 | Simulation | ptx_long_lines[100] |
N/A | 1.7 s | N/A |
| 🆕 | Simulation | ptx_short_lines[100000] |
N/A | 14 s | N/A |
| 🆕 | Simulation | ptx_input_references_long_lines[1000] |
N/A | 2 s | N/A |
| 🆕 | Simulation | ptx_tex[10000] |
N/A | 2.9 s | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing Aisha630:ptx-add-benchmark (2ba6147) with main (822aa83)
Footnotes
-
46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
ptx: add benchmark
Summary
I'm adding Divan benchmarks for
ptx.This is a benchmark-only change. No utility code is touched.
Why
ptxhad no benchmark. Its cost is dominated by converting keyword byte offsets to character offsets while building the output, which grows with the line length. So the benchmarks hold the total input size roughly constant and vary the line length, which is what surfaces that behavior. It also gives us a baseline to measure the offset-conversion optimization I'm proposing separately.Benchmarks
ptx_short_lines100_000linesptx_long_lines100linesptx_input_references_short_lines100_000lines-r, short linesptx_input_references_long_lines100lines-ron fewer, longer linesI followed the guidelines in the performance.md file and drew inspiration from the
tr,nl,catbenchmarks that were already set up.