Reasoning / Thinking Parameters¶

Modern LLMs can perform explicit chain-of-thought reasoning before generating their final answer. Each provider exposes this capability through different parameter names, structures, and semantics. LLM-Rosetta's ReasoningConfig provides a unified IR layer that maps to all supported providers.

Provider Comparison¶

Mode Control¶

How each provider controls whether reasoning is on, off, or automatic:

Effort Level¶

How hard the model should think. Not all providers support the same granularity:

Budget Tokens¶

Maximum tokens the model can spend on reasoning:

Summary Matrix¶

IR ReasoningConfig¶

LLM-Rosetta defines a unified ReasoningConfig TypedDict with three fields:

class ReasoningConfig(TypedDict, total=False):
    mode: Literal["auto", "enabled", "disabled"]
    effort: Literal["minimal", "low", "medium", "high", "max"]
    budget_tokens: int   # Max tokens for reasoning

All three fields are optional. You can set any combination depending on what you want to control.

Field Semantics¶

• mode -- Controls reasoning behavior: "enabled" (always on), "disabled" (off), or "auto" (let the model decide). Omit to let the provider use its default behavior.
• effort -- How much effort the model should put into reasoning. This is a cross-cutting concern independent of the mode.
• budget_tokens -- Hard cap on reasoning token count. Only meaningful for providers that support it (Anthropic, Google).

IR-to-Provider Mapping¶

mode: "enabled" (no effort, no budget)¶

The simplest way to turn on reasoning:

ir_request: IRRequest = {
    "model": "claude-sonnet-4-20250514",
    "messages": [...],
    "reasoning": {"mode": "enabled"},
}

{
  "thinking": {"type": "adaptive"}
}

{
  "reasoning": {"type": "enabled"}
}

mode: "enabled" + budget_tokens¶

Explicit control over reasoning budget:

ir_request: IRRequest = {
    "model": "claude-sonnet-4-20250514",
    "messages": [...],
    "reasoning": {"mode": "enabled", "budget_tokens": 10000},
}

{
  "thinking": {
    "type": "enabled",
    "budget_tokens": 10000
  }
}

{}

{
  "reasoning": {"type": "enabled"}
}

{
  "thinking_config": {
    "thinking_budget": 10000
  }
}

mode: "disabled"¶

Explicitly disable reasoning:

ir_request: IRRequest = {
    "model": "claude-sonnet-4-20250514",
    "messages": [...],
    "reasoning": {"mode": "disabled"},
}

{
  "thinking": {"type": "disabled"}
}

{
  "reasoning": {"type": "disabled"}
}

effort (with or without mode)¶

Set reasoning effort level:

ir_request: IRRequest = {
    "model": "claude-sonnet-4-20250514",
    "messages": [...],
    "reasoning": {"effort": "high"},
}

{
  "thinking": {
    "type": "adaptive",
    "effort": "high"
  }
}

{
  "reasoning_effort": "high"
}

{
  "reasoning": {
    "effort": "high"
  }
}

{
  "thinking_config": {
    "thinking_level": "high"
  }
}

Effort Level Mapping¶

The IR supports five effort levels. Not all providers accept every value:

:material-alert: = downgraded with a warning

Provider-to-IR Mapping (Reverse)¶

When converting provider-native requests to IR:

Design Decisions¶

Why mode is a three-way field¶

The IR uses an explicit mode: "auto" | "enabled" | "disabled" instead of a boolean:

1. Direct provider alignment. Anthropic's thinking.type has three values ("adaptive", "enabled", "disabled"), as does OpenAI Responses' reasoning.type. A three-way mode maps 1:1, enabling lossless round-trips.
2. Omission is still valid. When mode is not set, the provider uses its default behavior -- which is automatic reasoning for thinking-capable models. This is distinct from mode: "auto", which explicitly requests adaptive behavior.
3. Effort as a cross-cutting concern. Setting effort alone (without mode) lets the model decide whether to think while controlling how much effort to use when it does.

Why effort has 5 levels¶

The IR supports minimal, low, medium, high, and max to be a superset of all provider levels:

• Google supports minimal but others do not (downgrade to low)
• Anthropic supports max but others do not (downgrade to high)
• The three middle levels (low, medium, high) are universally supported

This ensures lossless round-trips within a single provider while providing best-effort mapping across providers.

Budget tokens: Anthropic and Google only¶

OpenAI (both Chat and Responses) does not support explicit reasoning budget control. When budget_tokens is set in the IR and the target is OpenAI, a warning is emitted and the field is silently dropped. This is by design -- the IR is a superset, and lossy conversions are explicitly flagged.

Full Example: Cross-Provider Reasoning¶

from llm_rosetta import convert

# Anthropic request with adaptive thinking
anthropic_request = {
    "model": "claude-sonnet-4-20250514",
    "max_tokens": 8096,
    "thinking": {
        "type": "adaptive",
        "effort": "high",
    },
    "messages": [
        {"role": "user", "content": "Explain quantum entanglement step by step."}
    ],
}

# Convert to OpenAI Chat format
openai_request = convert(anthropic_request, target="openai_chat")
# Result includes: {"reasoning_effort": "high", ...}

# Convert to Google GenAI format
google_request = convert(anthropic_request, target="google_genai")
# Result includes: {"thinking_config": {"thinking_level": "high"}, ...}