Skip to content

[AI] support on-device structured output - #8395

Merged
VinayGuthal merged 32 commits into
mainfrom
mila-support-structured-ouput-locally
Aug 12, 2026
Merged

[AI] support on-device structured output#8395
VinayGuthal merged 32 commits into
mainfrom
mila-support-structured-ouput-locally

Conversation

@milaGGL

@milaGGL milaGGL commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

This PR introduces Structured Output support on device, allowing developers to generate strongly-typed objects using ML Kit's backend. It defines the public API (GenerateObjectResponse and generateObject()), handles the necessary Kotlin Symbol Processing (KSP) logic for schema translation.

How to use On-Device Structured Output

  1. Define a data class annotated with @generable (and @Guide). Note: You must provide an empty companion object so the KSP processor can generate the necessary schema extensions.
  2. Call generateObject
import com.google.firebase.ai.annotations.Generable
@Generable
data class Recipe(
    val recipeName: String,
    @Guide(description = "A list of ingredients", minItems = 2)
    val ingredients: List<String>,
    @Guide(description = "Preparation time in minutes", minimum = 1.0)
    val prepTimeMinutes: Int
) {
    companion object
}

val schema = Recipe.firebaseAISchema()
val prompt = "Give me a recipe for chocolate chip cookies"

val response = model.generateObject(schema, prompt)
val recipe: Recipe? = response.getObject()
recipe?.let {
    println("Recipe: ${it.recipeName}")
    println("Ingredients: ${it.ingredients.joinToString()}")
    println("Prep time: ${it.prepTimeMinutes} mins")
}

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

@google-oss-bot

Copy link
Copy Markdown
Collaborator
1 Warning
⚠️ Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

@gemini-code-assist

Copy link
Copy Markdown
Contributor
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@google-oss-bot

Copy link
Copy Markdown
Collaborator

The public api surface has changed for the subproject ai-logic_firebase-ai-ondevice-interop:
error: Added class com.google.firebase.ai.ondevice.interop.GenerateObjectResponse [AddedClass]
error: Added method com.google.firebase.ai.ondevice.interop.GenerativeModel.generateObject(com.google.firebase.ai.ondevice.interop.GenerateContentRequest,kotlin.reflect.KClass,kotlin.coroutines.Continuation<? super com.google.firebase.ai.ondevice.interop.GenerateObjectResponse>) [AddedAbstractMethod]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@milaGGL
milaGGL marked this pull request as ready for review July 28, 2026 14:25
@milaGGL milaGGL changed the title initial implementation [AI] support on-device structured output Jul 28, 2026
@milaGGL

milaGGL commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for structured object generation (generateObject) in the on-device model interop and implementation layers, including updates to the KSP processor to generate ML Kit companion classes and support enum values in the @Guide annotation. Feedback on these changes highlights a type-safety issue in GenerateObjectResponse.kt regarding MutableList invariance, as well as an opportunity to replace unsafe non-null assertions (!!) with idiomatic safe calls in SchemaSymbolProcessorVisitor.kt.

@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@firebase firebase deleted a comment from google-oss-bot Jul 28, 2026
@milaGGL
milaGGL requested a review from VinayGuthal August 4, 2026 20:17

@emilypgoogle emilypgoogle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave a quick pass and left some comments

Comment thread ai-logic/firebase-ai-ksp-processor/test-app/test-app.gradle.kts
Comment thread ai-logic/firebase-ai-ksp-processor/test-app/test-app.gradle.kts Outdated
@VinayGuthal
VinayGuthal force-pushed the mila-support-structured-ouput-locally branch from 6c82cfd to fee271c Compare August 11, 2026 18:30
Comment thread ai-logic/firebase-ai/CHANGELOG.md
@VinayGuthal
VinayGuthal merged commit c62af76 into main Aug 12, 2026
326 of 334 checks passed
@VinayGuthal
VinayGuthal deleted the mila-support-structured-ouput-locally branch August 12, 2026 16:06
@github-actions github-actions Bot mentioned this pull request Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants