[MLI-8206] Set an explicit, configurable timeout on the sync forwarder

[lorenzo-norcini-scale]

Problem

The sync http-forwarder posts to the local inference server without an explicit aiohttp timeout, so aiohttp's client default of total=300s applies. Non-streaming generations that take longer than 5 minutes are cut off with a 500 (TimeoutError in forwarding.py:forward) while the inference server keeps computing the response. Observed on the qwen3-5-9b model-zoo endpoint (MLI-8206).

Change

• Add a timeout_seconds field to Forwarder and LoadForwarder, passed as aiohttp.ClientTimeout(total=...) on the forward request.
• Default is 3600s, overridable per deployment via the existing forwarder config path (forwarder.sync.timeout_seconds).
• The sync __call__ path (requests-based) and the streaming forwarder are unchanged.

Testing

• New parametrized unit test covers the default and an override propagating through LoadForwarder.load.
• tests/unit/inference: 53 passed, 2 skipped locally.
• black / isort / ruff / mypy clean on changed files.

🤖 Generated with Claude Code

Greptile Summary

The PR adds a configurable one-hour default timeout to asynchronous non-streaming forward requests and validates overrides during forwarder loading.

• Passes the configured timeout to aiohttp.ClientTimeout.
• Rejects nonnumeric, Boolean, non-finite, zero, and negative timeout values at startup.
• Tests default and overridden propagation plus invalid configurations.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
model-engine/model_engine_server/inference/forwarding/forwarding.py	Adds and validates the configurable sync-forwarding timeout before applying it to asynchronous HTTP requests.
model-engine/tests/unit/inference/test_forwarding.py	Covers default and overridden timeout propagation and rejects representative invalid values.

_{Reviews (2): Last reviewed commit: "Validate timeout_seconds at forwarder lo..." | Re-trigger Greptile}

[lorenzo-norcini-scale]

Superseded by the consolidated incident-hardening PR: see the PR referencing this one.

[lorenzo-norcini-scale]

Superseded by #868.