SkillToolset ignores tool_filter when injecting system instruction (mentions run_skill_script / load_skill_resource even when filtered out)

[nicobytes]

Summary

SkillToolset(tool_filter=[...]) correctly filters which tools are exposed to the model via get_tools(), but process_llm_request always injects a system instruction built by _build_skill_system_instruction() that unconditionally advertises run_skill_script and load_skill_resource.

When those tools are filtered out (e.g. skills that only have SKILL.md and no scripts/ / references/), the model still tries to call them. In ADK 2.5 this becomes ValueError: Tool 'run_skill_script' not found. and can crash the entire workflow invocation unless on_tool_error_callback recovers it.

Skills are marked experimental in the docs (adk.dev/skills), but this looks like a clear inconsistency between tool_filter and the injected instruction.

Environment

• google-adk: 2.5.0
• Python: 3.12
• Runtime: Vertex AI Agent Runtime / Reasoning Engine
• Model: gemini-flash-latest

Minimal reproduction

from google.adk import Agent
from google.adk.skills import models
from google.adk.tools.skill_toolset import SkillToolset

skill = models.Skill(
    frontmatter=models.Frontmatter(
        name="whatsapp-formatting",
        description="Format messages for WhatsApp native rendering.",
    ),
    instructions="Use *bold* and _italic_ WhatsApp syntax. No scripts.",
)

toolset = SkillToolset(
    skills=[skill],
    tool_filter=["list_skills", "load_skill"],  # intentionally hide script/resource tools
)

agent = Agent(
    model="gemini-flash-latest",
    name="demo",
    instruction="Help the user. Load the formatting skill when needed, then reply as text.",
    tools=[toolset],
)

Ask the agent to produce a formatted WhatsApp reply. Observed model behavior:

1. Calls load_skill(skill_name="whatsapp-formatting") (correct).
2. Then often wraps the already-written user reply in:
   run_skill_script(skill_name="whatsapp-formatting", file_path="scripts/format_text.py", args={...})
3. Vertex marks finish_reason: UNEXPECTED_TOOL_CALL (tool not declared).
4. ADK raises:
   ValueError: Tool 'run_skill_script' not found.

Root cause (in installed 2.5.0 sources)

In google/adk/tools/skill_toolset.py:

• get_tools() respects tool_filter via _is_tool_selected.
• process_llm_request() always does:

instructions = [
    _build_skill_system_instruction(prefix=self.tool_name_prefix)
]
llm_request.append_instructions(instructions)

And _build_skill_system_instruction() hardcodes steps that tell the model to use `{p}run_skill_script` and `{p}load_skill_resource`, with no awareness of tool_filter.

So the model is instructed to call tools that are not present in tools_dict.

Expected behavior

When tool_filter excludes run_skill_script / load_skill_resource, the injected system instruction should:

1. Only mention tools that remain available after filtering, and/or
2. Explicitly tell the model those tools are unavailable and to emit the final reply as normal model text (never wrap the user-facing answer in a tool call).

Ideally tool_filter and the skill system instruction stay consistent by construction.

Actual behavior

• Tools are filtered out of declarations.
• Instruction still advertises filtered tools.
• Model hallucinates the call → UNEXPECTED_TOOL_CALL / ValueError: Tool '...' not found.
• Without on_tool_error_callback, the workflow invocation dies and upstream clients see an empty text response.

Workaround

Subclass SkillToolset and override process_llm_request to inject an instruction built only from the allowed tool names; plus register on_tool_error_callback so unknown-tool errors become recoverable tool responses instead of crashing the run.

Happy to contribute a PR if maintainers agree on the desired API (e.g. make _build_skill_system_instruction take the active tool names / filter).

Related evidence

Telemetry / stream snippet after applying on_tool_error_callback (defense in depth; root cause still present):

{
  "role": "user",
  "parts": [{
    "functionResponse": {
      "name": "run_skill_script",
      "response": {
        "error": "La herramienta 'run_skill_script' no existe. No la reintentes; escribe tu respuesta final directamente como texto.",
        "error_code": "TOOL_NOT_FOUND"
      }
    }
  }]
}

[nicobytes]

Related issues

This looks closely related to (and partly overlaps with):

• SkillToolset exposes run_skill_script tool causing hallucination loop when skill has no scripts/ directory #6281 — run_skill_script is exposed / hallucinated when a skill has no scripts/ directory (infinite retry loop). Same user-visible symptom: the model invents run_skill_script(...) for instruction-only skills.
• SkillToolset causes infinite list_skills/load_skill loop when skill instructions reference a tool not present in the available tool list #6280 — skill system instruction drives loops when referenced tools are unavailable.
• [Bug] RunSkillScriptTool retries SCRIPT_NOT_FOUND indefinitely; default max_llm_calls=500 is the only backstop #5684 — historical SCRIPT_NOT_FOUND retry storm (closed; mitigations landed in later releases).

How this issue differs

#6281 focuses on when run_skill_script should be registered at all (ideally only if scripts/ exists).

This issue (#6448) focuses on a second inconsistency: even after using the documented escape hatch tool_filter=["list_skills", "load_skill"] to remove run_skill_script from get_tools(), process_llm_request still injects _build_skill_system_instruction() text that tells the model to call run_skill_script / load_skill_resource. So filtering the tools is not enough — the instruction and the tool list diverge, and ADK 2.5 then raises ValueError: Tool 'run_skill_script' not found (or UNEXPECTED_TOOL_CALL from the model provider).

A complete fix likely needs both:

1. Don't advertise / register run_skill_script when no scripts exist (SkillToolset exposes run_skill_script tool causing hallucination loop when skill has no scripts/ directory #6281), and
2. Make the injected skill system instruction respect tool_filter / the actually selected tools (SkillToolset ignores tool_filter when injecting system instruction (mentions run_skill_script / load_skill_resource even when filtered out) #6448).

Happy to send a PR for (2) if maintainers agree on the approach.

[llalitkumarrr]

Hello @nicobytes ,

Thank you for reporting this issue. We were able to reproduced it on our end.

Currently, using the tool filter argument causes the script to fail. Any tools (whether one, multiple, or all) in the filter leads to unexpected behavior. Additionally, we tested the changes suggested in #6449 , but they do not seem to resolve the issue.

Hello @sasha-gitg ,

Could you please take a look at this?