Fix: Structured Output + Thinking + Tool Use Bugs

Written by Michael Lip · Solo founder of Zovo · $400K+ on Upwork · 100% JSS Join 50+ builders · More at zovo.one

The Error

When using structured outputs (output_config with json_schema) combined with thinking and tool use, you hit one or both of these bugs:

Bug 1: Model returns only thinking + empty text, no tool_use blocks

{
  "content": [
    {"type": "thinking", "thinking": "I need to call tool_1 and tool_2 in parallel..."},
    {"type": "text", "text": ""}
  ],
  "stop_reason": "end_turn"
}

The model’s thinking correctly identifies the tools to call, but no tool_use blocks are emitted. stop_reason is "end_turn" instead of "tool_use".

Bug 2: Final structured output contains invalid JSON

{
  "content": [
    {
      "type": "text",
      "text": "{\"my_json\": \"is_broken \n\n\n\n\n\n\n {\"my_json\": \"is_not_broken\"}"
    }
  ]
}

Partial/malformed JSON followed by blank lines, then valid JSON — all in a single text block.

Quick Fix

Disable structured outputs when using thinking + tool use. Validate output manually:

from pydantic import BaseModel

class Output(BaseModel):
    param1: str
    param2: str

# Remove output_config, keep thinking and tools
response = await client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=10000,
    messages=[{"role": "user", "content": query}],
    tools=tools,
    thinking={"type": "adaptive"},
    # Do NOT use output_config with json_schema here
)

# Parse and validate manually on the final turn
if response.stop_reason == "end_turn":
    text_block = next(b for b in response.content if b.type == "text")
    result = Output.model_validate_json(text_block.text)

What’s Happening

The combination of three features — structured outputs, extended thinking, and tool use — creates an unstable interaction in the API’s response generation:

Bug 1: Missing tool_use Blocks

When the model has structured output format active and needs to call a tool, there is a conflict in the output constraints:

  1. The structured output grammar constrains the text block to be valid JSON matching your schema
  2. Tool use requires generating tool_use blocks (which are not JSON text blocks)
  3. Extended thinking adds a preliminary reasoning step

The model’s thinking block correctly determines that tools should be called. But when it transitions to generating content, the structured output grammar captures the output stream and constrains it to produce JSON. The tool_use block generation never fires, and the model ends the turn with an empty text block.

Bug 2: Invalid JSON in Structured Output

In the final turn (after tool results are provided), the model sometimes generates:

This appears to be a token generation issue where the model starts producing JSON, hits an internal constraint, resets, and generates the correct JSON appended to the failed attempt.

Feature Interaction Matrix

Thinking Tools Structured Output Status
Off Off On Works
Off On Off Works
On Off On Works (usually)
On On Off Works
On On On Broken

Step-by-Step Solution

Use thinking + tools in the first phase, then structured output in the final extraction:

from anthropic import AsyncAnthropic
from pydantic import BaseModel

client = AsyncAnthropic()

class FinalOutput(BaseModel):
    summary: str
    confidence: float
    sources: list[str]

# Phase 1: Think and use tools (no structured output)
messages = [{"role": "user", "content": query}]

for i in range(10):  # Bounded tool loop
    response = await client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=10000,
        system="Use the provided tools to gather information. When done, provide your final answer.",
        messages=messages,
        tools=tools,
        thinking={"type": "enabled", "budget_tokens": 5000},
        # NO output_config here
    )

    if response.stop_reason != "tool_use":
        break

    # Append assistant message and tool results
    messages.append({"role": "assistant", "content": response.content})
    tool_results = execute_tools(response)
    messages.append({"role": "user", "content": tool_results})

# Phase 2: Extract structured output (no tools, no thinking)
extraction_messages = messages + [{
    "role": "user",
    "content": "Based on the above conversation, provide your final structured answer."
}]

structured_response = await client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=5000,
    messages=extraction_messages,
    output_config={
        "format": {
            "type": "json_schema",
            "schema": FinalOutput.model_json_schema()
        }
    },
    # NO tools, NO thinking
)

result = FinalOutput.model_validate_json(structured_response.content[0].text)

Option 2: Structured Output Without Thinking

If you need tools + structured output but can sacrifice thinking:

response = await client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=10000,
    messages=messages,
    tools=tools,
    output_config={
        "format": {
            "type": "json_schema",
            "schema": schema
        }
    },
    # NO thinking parameter
)

Note: this can sometimes produce empty content. Add retry logic:

for attempt in range(3):
    response = await client.messages.create(...)
    if response.content and any(
        b.type == "text" and b.text.strip() for b in response.content
    ):
        break
    await asyncio.sleep(1)

Option 3: Thinking + Tools, Manual JSON Parsing

Use thinking and tools freely, then extract JSON from the final response with robust parsing:

import json
import re

def extract_json(text: str) -> dict:
    """Extract the last valid JSON object from potentially garbled text."""
    # Try the full text first
    try:
        return json.loads(text.strip())
    except json.JSONDecodeError:
        pass

    # Find all JSON-like objects in the text
    candidates = []
    brace_depth = 0
    start = None

    for i, char in enumerate(text):
        if char == '{':
            if brace_depth == 0:
                start = i
            brace_depth += 1
        elif char == '}':
            brace_depth -= 1
            if brace_depth == 0 and start is not None:
                candidate = text[start:i + 1]
                try:
                    parsed = json.loads(candidate)
                    candidates.append(parsed)
                except json.JSONDecodeError:
                    pass
                start = None

    if not candidates:
        raise ValueError(f"No valid JSON found in response: {text[:200]}...")

    # Return the last valid JSON object (most likely the complete one)
    return candidates[-1]

Option 4: Use messages.parse with Retry

from anthropic import AsyncAnthropic

client = AsyncAnthropic()

async def parse_with_retry(params, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = await client.messages.parse(**params)
            if response.parsed_output is not None:
                return response.parsed_output

            # Empty parsed output — retry
            if attempt < max_retries - 1:
                continue
        except Exception as e:
            if attempt < max_retries - 1:
                continue
            raise

    raise RuntimeError("Failed to get valid structured output after retries")

Prevention

Tools That Help

For developers building multi-step AI agent pipelines, a dev tool extension can help debug the intermediate tool calls and responses, making it easier to identify where structured output generation breaks down.