Python: send a Pydantic response_format to Gemini as response_schema - #7879
Conversation
microsoft#5893 taught the Gemini client to forward mapping-shaped response_format values as a Gemini response_schema, and scoped itself to those shapes. A Pydantic model class - the first shape the option's own documentation offers - still falls through _extract_response_schema, so the request carries response_mime_type="application/json" with no schema attached. Gemini is then asked for JSON but is not constrained by the model, and the free-form JSON it returns is handed to that same model for parsing on the way back. That is the failure mode microsoft#5888 described, on the shape microsoft#5893 did not cover. Convert the model class with model_json_schema(), matching what the Anthropic, Bedrock and Mistral clients already do for this option.
GitHub Actions
Active
GitHub Actions
Active
GitHub Actions
Active
agent-framework-automation
Bot
added
the
python
GitHub Actions
Active
There was a problem hiding this comment.
Pull request overview
Adds missing Gemini schema forwarding for Pydantic response models.
Changes:
- Converts Pydantic model classes using
model_json_schema(). - Tests schema forwarding and explicit-schema precedence.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
_chat_client.py |
Extracts Gemini schemas from Pydantic models. |
test_gemini_client.py |
Adds structured-output regression tests. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
| Handles Pydantic model classes and the mapping shapes (raw JSON schema, | ||
| ``json_schema`` envelopes, ``format`` envelopes). |
Luis Rodriguez (luisangelrod)
left a comment
Luis Rodriguez (luisangelrod)
left a comment
There was a problem hiding this comment.
Reviewed the full change at 320e4791. The Pydantic model guard is correctly ordered before the mapping handling, and the existing explicit response_schema precedence remains intact. I also checked the downstream structured-response parsing and the existing raw-schema and envelope paths.
On Windows with Python 3.12, I ran the complete packages/gemini/tests/test_gemini_client.py file: 148 passed and 8 integration tests skipped. The focused response_format group passed 21 tests, Ruff reported no findings for the Gemini package, and Pyright completed with 0 errors and 0 warnings.
The existing inline note about documenting the direct {"schema": ...} envelope is valid, but I found no additional functional or test issue.
What
response_formataccepts "a Pydantic model type or JSON schema mapping" per its own documentation, but only the mapping half reaches Gemini as aresponse_schema. A model class falls through_extract_response_schema, so the request goes out withresponse_mime_type="application/json"and no schema attached.This converts the model class with
model_json_schema().Why it slipped through
#5893 fixed exactly this failure for mapping-shaped values, and said so explicitly — it extracts schemas from "supported mapping-shaped
response_formatvalues"._extract_response_schemaopens withif not isinstance(response_format, Mapping): return None, which is correct for that scope. The Pydantic class shape simply was not in it.The existing tests sit either side of the gap without covering it:
test_response_format_sets_json_mime_typepasses a Pydantic model and assertsresponse_mime_type, but notresponse_schema.test_response_format_populates_value_on_chat_responseasserts the inbound parse into that model, which works regardless.So the outbound half is never checked for this shape, and the request has been going out unconstrained.
Effect
Gemini is asked for JSON without being told what shape, and
_process_generate_responsethen parses whatever comes back into the model the caller asked for. Depending on what the model returns, that is either aValidationErrorthe caller did not expect or — worse — JSON that happens to validate while ignoring the intent of the schema. It is #5888's failure mode on a different shape.Consistency with the other clients
_prepare_response_formatin Mistral,_prepare_output_configin Bedrock, and the Anthropic equivalent all branch onisinstance(x, type) and issubclass(x, BaseModel)and callmodel_json_schema(). Gemini was the one client in that group without the branch. Ollama had the same gap and it was closed in #6782.Tests
Two added:
test_response_format_pydantic_model_sets_response_schema— the regression. Verified red-before-green: with the source change reverted it fails withassert None == {'properties': {'text': ...}, 'title': 'Reply', 'type': 'object'},NonebeingGenerateContentConfig(response_mime_type='application/json').response_schema.test_response_schema_option_wins_over_pydantic_response_format— a guard that an explicitresponse_schemastill takes precedence, as it already does for mapping shapes.ruff checkandruff format --checkare clean on the package.On the suite run:
packages/gemini/tests/hangs for me on the same test before and after this change — 146 tests pass then it stalls in an asyncio loop onmain, 148 with these two added, no failures either way. It looks environmental on my machine rather than related to this diff, but flagging it rather than claiming a clean full-suite run I did not get.One question
The docstring already promised Pydantic support, so I treated this as a defect rather than a feature and kept the change to the one branch. If you would rather
_extract_response_schemastay mapping-only and have the conversion happen upstream where the option is validated, say so and I will move it — I did not want to widen the surface without asking.