Skip to content

fix(review): measure the prompt budget in escaped bytes - #32

Merged
zfarrell merged 1 commit into
fix/bound-review-prompt-sizefrom
fix/escaped-prompt-budget
Aug 7, 2026
Merged

fix(review): measure the prompt budget in escaped bytes#32
zfarrell merged 1 commit into
fix/bound-review-prompt-sizefrom
fix/escaped-prompt-budget

Conversation

@zfarrell

@zfarrell zfarrell commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

This adds two changes to #31. The base branch is fix/bound-review-prompt-size.

Why the budget must count escaped bytes

#31 counts the raw bytes of the prompt. But the prompt does not go to the reviewer only as raw bytes.

The action.yml file of claude-code-action sets ALL_INPUTS: toJson(inputs) on the step that it runs. Thus the step carries the prompt two times:

variable contents
PROMPT the prompt as written
ALL_INPUTS the same prompt, JSON-escaped, in a JSON object

The escaped copy is always larger. Therefore the escaped copy always reaches the limit first. If you bound the raw string, the string that fails stays unbounded.

Pull request hotdata-dev/monopoly#1670 shows this. Its prompt was 123,401 raw bytes, which is inside the limit. The same prompt was 135,366 bytes after escaping. The review failed two times.

I ran the script of #31 against that same pull request:

PROMPT env var (raw):   124,499   limit 131,072 -> fits
ALL_INPUTS (escaped):   137,231   limit 131,072 -> OVER

The cap of #31 does not operate at all here. The context is near 115 KB, which is less than the budget of 121 KB. The prompt goes through with no change, and exec fails again.

You cannot correct for this with a constant factor. Prose costs approximately 1.02x after escaping. A quote-dense JSON diff costs approximately 1.20x. The same 3,000 lines can be on one side of the limit or the other. Only the content decides.

What this changes

  • All budgets count escaped bytes. A new function escaped_bytes measures them with jq -Rs ., which does the same operation as toJson. Against the run that failed, jq was within 0.5%, and it was high. High is the safe direction.
  • A new constant ALL_INPUTS_OTHER_BYTES holds the 38 other inputs that toJson(inputs) writes beside the prompt. They are 1,356 bytes in the same variable. fix(review): bound the assembled prompt below MAX_ARG_STRLEN #31 does not count them.
  • A new function cap_file_escaped trims a file until the escaped size fits. It measures, scales, and measures again, because the ratio is a property of the content. It also removes the incomplete last line, so the context does not stop in the middle of a patch line.
  • The two diff blocks share one line budget of 3,000 lines. fix(review): bound the assembled prompt below MAX_ARG_STRLEN #31 leaves this out of its scope.

Why the two diff blocks need one budget

The since-last-review diff is a subset of the full diff. On a pull request with one file, it is almost all of it. With independent caps the two blocks can reach 5,000 lines of nearly the same patch. On PR #1670 this was 42 KB of "since your last review" on top of 66 KB of "full diff".

The budget of #31 keeps that inside the limit. But it does not stop the cost. Every line the copy spends is a line that the rest of the context does not get.

The since-diff keeps its share, because on cycle 2 and later the new changes are the subject of the review. The full diff gives up its lines. Its notice tells the reviewer to run gh pr diff for the remainder.

Result

The same pull request, with this branch:

ALL_INPUTS: 129,191   limit 131,072 -> fits

Tests

The tests of #31 keep their structure. I changed the totals to count escaped bytes and to include the other inputs. Two new cases:

  • Both diff blocks oversized. The pair must stay inside the shared budget.
  • A quote-dense diff, which is the shape that failed in production. A raw total lets it pass. An escaped total does not.

All five suites pass. The workflow YAML does not change.

One point for your decision

The margin is 1,881 bytes, which is 1.4% of the limit. This is a result of the 1 KB of slack in #31, which your comment explains.

The risk: claude-code-action can add a new input. Then ALL_INPUTS_OTHER_BYTES grows past its 2,000-byte allowance and the failure returns.

I did not change your slack, because the small number is your decision and your reason for it is written down. Tell me if you want it larger.

@zfarrell
zfarrell requested a review from a team as a code owner August 6, 2026 17:12
@zfarrell
zfarrell requested review from eddietejeda and removed request for a team August 6, 2026 17:12
@zfarrell
zfarrell merged commit 69d2b2b into fix/bound-review-prompt-size Aug 7, 2026
3 of 6 checks passed
@zfarrell
zfarrell deleted the fix/escaped-prompt-budget branch August 7, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant