fix(tracing): mark the agent span when a non-streaming run fails

[hsusul]

Summary

Runner.run() and Runner.run_sync() left the active agent span unmarked when a run failed, while Runner.run_streamed() attached an Error in agent run SpanError. Exported traces from non-streaming runs therefore under-reported failures, and failures raised outside a generation or function span produced a trace with no error marker anywhere.

Affected component: src/agents/run.py (non-streaming failure handler), with the shared exclusion predicate moved into src/agents/run_internal/error_handlers.py and reused by src/agents/run_internal/run_loop.py.

Root cause

start_streaming() in run_loop.py wraps its turn loop and outer body in except Exception as e: handlers that call:

if current_span and _should_attach_generic_agent_error(e):
    _error_tracing.attach_error_to_span(
        current_span,
        SpanError(message="Error in agent run", data={"error": _error_tracing.get_trace_error(...)}),
    )

The corresponding block in run.py is except BaseException as exc:, which assigns run_exception, attaches RunErrorDetails to AgentsExceptions, and re-raises. It never touches current_span; the finally block then calls current_span.finish(reset_current=True) with no error recorded.

Parity gap between execution paths

The gap was already visible in the committed snapshots — tests/test_tracing_errors.py::test_single_turn_model_error had an agent span with no error key, while tests/test_tracing_errors_streamed.py::test_single_turn_model_error had "error": {"message": "Error in agent run", "data": {"error": "test error"}}.

I mapped the behavior of the streamed path per exception type and used it as the reference for the fix:

Exception raised by the model	Streamed agent-span error	Non-streamed before	Non-streamed after
ValueError	Error in agent run	(none)	Error in agent run
ModelRefusalError	Error in agent run	(none)	Error in agent run
UserError	Error in agent run	(none)	Error in agent run
AgentsException	Error in agent run	(none)	Error in agent run
ModelBehaviorError	(none — generation span reports it)	(none)	(none)
Input/output guardrail tripwire	Guardrail tripwire triggered	Guardrail tripwire triggered	unchanged
MaxTurnsExceeded	Max turns exceeded	Max turns exceeded	unchanged

Runner.run_sync() delegates to the same AgentRunner.run() code path, so it is fixed by the same change and is covered by its own test.

Behavioral change

A non-streaming run that fails now marks the active agent span with the same SpanError message and data shape the streamed path uses, honouring RunConfig.trace_include_sensitive_data through the existing _error_tracing.get_trace_error() helper. No new error schema is introduced.

The attach is guarded by three conditions:

if (
    current_span is not None
    and current_span.error is None
    and isinstance(exc, Exception)
    and should_attach_generic_agent_error(exc)
):

• current_span.error is None — a more specific error already on the span (Max turns exceeded, Guardrail tripwire triggered) is never overwritten, and the generic error can never be applied twice. SpanImpl.set_error() overwrites unconditionally, so this guard is what makes both properties hold.
• isinstance(exc, Exception) — the handler catches BaseException, but asyncio.CancelledError is not an agent failure and the streamed path (which catches only Exception) never marks it.
• should_attach_generic_agent_error(exc) — the existing exclusion list, so ModelBehaviorError and guardrail tripwires keep their more specific reporting.

Everything else is untouched: the exception still propagates unchanged, run_exception assignment and RunErrorDetails attachment are unchanged, and successful runs never enter this branch.

_should_attach_generic_agent_error moved from run_loop.py into error_handlers.py as should_attach_generic_agent_error so both paths share one definition of the exclusion list rather than duplicating it. run_loop.py now imports it; its two call sites are otherwise unchanged.

Tests added

In tests/test_tracing_errors.py, using the repository's existing FakeModel and SPAN_PROCESSOR_TESTING / fetch_span_errors utilities:

• test_run_marks_agent_span_with_generic_error — Runner.run() marks the agent span, and the original ValueError still propagates (pytest.raises(ValueError, match="test error")).
• test_run_sync_marks_agent_span_with_generic_error — same through Runner.run_sync().
• test_run_agent_span_error_matches_streamed_path — runs the identical failure through Runner.run() and Runner.run_streamed() and asserts both produce the same agent-span error.
• test_run_agent_span_error_redacts_sensitive_data — mirrors the streamed test_streamed_agent_error_redacts_sensitive_data; with trace_include_sensitive_data=False the detail is "Error details are redacted.".
• test_run_does_not_mark_agent_span_for_model_behavior_error — the excluded case stays clean.
• test_run_marks_agent_span_for_other_agents_exceptions[model-refusal-error|user-error] — non-excluded AgentsException subclasses match the streamed path.
• test_run_keeps_specific_max_turns_agent_span_error — a pre-existing, more specific Max turns exceeded error is preserved rather than overwritten.
• test_run_attaches_agent_span_error_exactly_once — wraps SpanImpl.set_error via monkeypatch and asserts exactly one error is recorded on the agent span.
• test_successful_run_leaves_agent_span_without_error — successful runs are unchanged.

All tests are deterministic: no sleeps, no concurrency, no network, and no API key.

Snapshot changes. Two committed inline snapshots change, both by exactly one added line on the agent span:

"error": {"message": "Error in agent run", "data": {"error": "test error"}},

in test_single_turn_model_error and test_multi_turn_no_handoffs. Nothing else in those snapshots moves — the generation span error, agent data, tools and children are byte-identical, and the added line makes each snapshot match its streamed counterpart. I inspected both diffs and hand-applied the line in the repo's existing formatting rather than accepting the tool's rewrite, because --inline-snapshot=fix also reflowed several unrelated lines to a narrower width.

Pre-fix proof. With only the three src/agents/ files reverted to upstream main and the tests in place:

FAILED tests/test_tracing_errors.py::test_single_turn_model_error
FAILED tests/test_tracing_errors.py::test_multi_turn_no_handoffs
FAILED tests/test_tracing_errors.py::test_run_marks_agent_span_with_generic_error
FAILED tests/test_tracing_errors.py::test_run_sync_marks_agent_span_with_generic_error
FAILED tests/test_tracing_errors.py::test_run_agent_span_error_matches_streamed_path
FAILED tests/test_tracing_errors.py::test_run_agent_span_error_redacts_sensitive_data
FAILED tests/test_tracing_errors.py::test_run_marks_agent_span_for_other_agents_exceptions[model-refusal-error]
FAILED tests/test_tracing_errors.py::test_run_marks_agent_span_for_other_agents_exceptions[user-error]
FAILED tests/test_tracing_errors.py::test_run_attaches_agent_span_error_exactly_once
9 failed, 9 passed, 2 errors

The three no-regression guards (..._model_behavior_error, ..._max_turns_agent_span_error, ..._successful_run_...) pass both before and after. All 18 pass with the fix.

Compatibility

No public API, exception type, or serialized-state change. Exported traces for failing non-streaming runs gain an agent-span error they previously lacked; consumers that assert the absence of that field would see the new value, which is the documented parity intent in .agents/references/runner-lifecycle.md ("Streaming and non-streaming paths must produce equivalent … guardrail results, session history, and interruption state for the same model behavior"). Successful runs are byte-identical.

Non-goals. The streamed path's own double-attach across its nested handlers, guardrail-span error wording, and any change to run_exception or RunErrorDetails semantics.

Test plan

Run from the repository root on fix/4070-nonstreaming-agent-span-error (Python 3.12.13, macOS 15.7.3):

Command	Result
make format	842 files left unchanged; ruff check --fix → All checks passed!
make lint	All checks passed!
make typecheck	mypy Success: no issues found in 833 source files; pyright 0 errors, 0 warnings, 0 informations
make tests	5968 passed, 3 skipped, 2 warnings (parallel) and 45 passed, 4 skipped, 5971 deselected (serial)
make tests-asyncio-stability	5/5 runs passed
git diff --check	clean
uv run pytest tests/test_tracing_errors.py tests/test_tracing_errors_streamed.py -q	29 passed
uv run pytest tests/test_tracing_errors.py tests/test_tracing_errors_streamed.py tests/test_agent_tracing.py tests/test_trace_processor.py tests/test_agent_runner.py tests/test_agent_runner_sync.py -q	274 passed (run 5× consecutively, no flakes)
UV_PROJECT_ENVIRONMENT=.venv_310 uv run --python 3.10 -m pytest tests/test_tracing_errors.py tests/test_tracing_errors_streamed.py tests/test_agent_tracing.py -q	55 passed

No OpenAI API key, network access, or paid model call was used; the tests rely on tests/fake_model.py::FakeModel and the in-repo SpanProcessorForTests.

Not run: make integration-tests* (requires live provider credentials and external services) and make build-docs (no documentation files changed). No lockfile or dependency changes.

Issue number

Fixes #4070

Checks

• I've added new tests, if relevant
• I've run .agents/skills/code-change-verification/scripts/run.sh
• I've confirmed all verification steps pass
• If using Codex, I've run /review before submitting this PR

Note on the unchecked boxes: I ran the documented verification stack directly (make format, make lint, make typecheck, make tests, plus make tests-asyncio-stability and a Python 3.10 run) rather than through the skill script wrapper, and I did not use Codex, so /review does not apply.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77b93795b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[seratch]

Before merge, please move the complete generic agent-error attachment policy into a run_internal helper used by both streaming and non-streaming paths. That helper should own eligibility, preservation of an existing span error, redaction, message construction, and attachment, leaving run.py with a single orchestration-level call.

Please also ensure that redacted tracing never evaluates str(exc) and that any exception-formatting failure cannot replace the original run exception. Add a regression test using an exception whose __str__ raises and assert that the original exception still propagates. Keep the existing ModelBehaviorError and guardrail exclusion policy for this PR; broader coverage of currently unmarked ModelBehaviorError paths should be handled separately.

[hsusul]

Thanks, all four points are addressed in 43f8c95.

Policy moved into a shared run_internal helper. attach_generic_agent_error(span, exc, *, trace_include_sensitive_data) in run_internal/error_handlers.py now owns the complete policy: eligibility (including the isinstance(exc, Exception) cancellation exclusion that previously lived in run.py), preservation of an existing span error, redaction, message construction, and the attach. run.py's failure handler is a single call, and both streaming handlers in run_loop.py call the same helper, so there is one implementation of the exclusion list, message, and redaction instead of parallel blocks. The previously exported predicate is now private to the helper.

One behavior note: the streamed path now also preserves a more specific span error, which it did not before. Its snapshots are unchanged, because its double-attach across the nested handlers was writing the identical value twice.

Redacted tracing never evaluates str(exc). The exception is stringified only inside the trace_include_sensitive_data=True branch; the redacted branch returns the constant. Formatting goes through _format_agent_error_detail, which catches Exception (not BaseException), logs a warning naming only the exception class, and returns "Error details are unavailable." — so an exception-formatting failure can never replace the exception the run is propagating.

Regression tests. UnformattableError.str raises and counts its calls; RaisingHooks raises it from on_agent_start, i.e. user code inside the agent span (FakeModel stringifies its own exception output before raising, so it can't carry this case). test_run_propagates_exception_whose_str_raises and test_streamed_run_propagates_exception_whose_str_raises assert exc_info.value is error on both paths and that the span records the placeholder. test_redacted_tracing_never_stringifies_the_exception calls the helper directly with redaction on and asserts str_calls == 0 — a unit test rather than end-to-end because the pre-existing sandbox-memory finally block calls terminal_metadata_for_exception(run_exception), which stringifies independently (inside its own try/except, so propagation is unaffected).

Exclusion policy unchanged. ModelBehaviorError and the guardrail tripwires stay excluded; the broader unmarked-ModelBehaviorError coverage is left for a separate PR.

Re-ran on the updated branch: make format, make lint, make typecheck (mypy 833 files, pyright 0 errors), make tests (5971 passed, 3 skipped + 45 passed, 4 skipped serial), make tests-asyncio-stability 5/5, and the tracing tests on Python 3.10.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 43f8c959eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Catch BaseException from trace-only formatting

When a custom exception's __str__ raises KeyboardInterrupt, SystemExit, or asyncio.CancelledError, this except Exception does not catch it, so the trace-only formatter replaces the original run exception despite the helper's stated guarantee. The fresh centralized helper now guards ordinary formatter failures, but it should catch BaseException specifically around str(exc) so every secondary formatting failure is reduced to the placeholder without changing run semantics.

AGENTS.md reference: AGENTS.md:L124-L124

Useful? React with 👍 / 👎.