Skip to main content
TalkOps Framework›Configuration & Models

Model Providers & Smart Mode Routing

TalkOps supports 20+ LLM providers through LangChain's unified init_chat_model interface and pairs them with the Dynamic Model Pool (DynamicModelPoolManager) so Jev Smart Mode can route every prompt across 3 cost/intelligence tiers (fast, standard, powerful) in <70ms.

Supported model providers​

Set the corresponding API key via talkops auth set <provider> <key> or environment variable to activate any provider:

ProviderPrefixDefault / Recommended ModelsCredential Env Var / Auth Key
Anthropicanthropic:claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5ANTHROPIC_API_KEY (talkops auth set anthropic)
OpenAIopenai:gpt-5.4, o3-pro, o3, gpt-5.4-miniOPENAI_API_KEY (talkops auth set openai)
Google Geminigoogle_genai:gemini-3.1-pro, gemini-3-flash, gemini-3.1-flash-liteGOOGLE_API_KEY (talkops auth set google)
AWS Bedrockbedrock:us.anthropic.claude-sonnet-4-6-v1:0, us.anthropic.claude-haiku-4-5-v1:0Native AWS chain (AWS_PROFILE, AWS_REGION, IAM Role)
Azure OpenAIazure_openai:gpt-5.4, o3AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT
Google Vertex AIvertexai:gemini-3.1-pro, claude-sonnet-4-6GOOGLE_APPLICATION_CREDENTIALS, GCP_PROJECT_ID
OpenRouteropenrouter:Any hosted model (anthropic/claude-sonnet-4-6, etc.)OPENROUTER_API_KEY (talkops auth set openrouter)
DeepSeekdeepseek:deepseek-reasoner, deepseek-chatDEEPSEEK_API_KEY (talkops auth set deepseek)
Groqgroq:llama-3.3-70b-versatileGROQ_API_KEY (talkops auth set groq)
Fireworks AIfireworks:accounts/fireworks/models/deepseek-v3FIREWORKS_API_KEY (talkops auth set fireworks)
Together AItogether:meta-llama/Llama-3.3-70B-Instruct-TurboTOGETHER_API_KEY (talkops auth set together)
Mistral AImistralai:mistral-large-latest, codestral-latestMISTRAL_API_KEY (talkops auth set mistral)
xAI (Grok)xai:grok-3, grok-3-miniXAI_API_KEY (talkops auth set xai)
Cerebras / SambaNova / Cohere / Perplexitycerebras:, sambanova:, cohere:, perplexity:Provider-specific high-throughput modelsCEREBRAS_API_KEY, SAMBANOVA_API_KEY, COHERE_API_KEY, PERPLEXITY_API_KEY
Ollama (Local)ollama:qwen2.5-coder:32b, llama3.3:70bOLLAMA_HOST (default http://localhost:11434)

How Smart Mode 3-tier model & reasoning routing works​

When you run TalkOps in Smart Mode (talkops --smart or /smart), you don't pay frontier model prices or wait for chain-of-thought thinking tokens on simple prompts.

Implemented in src/talkops/model/pool.py (DynamicModelPoolManager) and src/talkops/middleware/jev_model_router.py (JevDynamicModelRouterMiddleware), TalkOps maintains a 3-slot model pool (fast, standard, powerful) and routes each turn in <70ms:

1. Tier assignment & automatic reasoning effort adjustment​

Tier SlotDefault Auto-Populated ModelsReasoning Effort (_resolve_dynamic_effort)When Jev Routes Here
Tier 0: fastclaude-haiku-4-5, gpt-5.4-mini, gemini-3.1-flash-liteoff (reasoning disabled)Greetings, conversational follow-ups, single-file reads, status checks (kubectl get pods, git status). Reduces token cost by 80–90% and eliminates thinking latency.
Tier 1: standardclaude-sonnet-4-6, gpt-5.4, gemini-3-flashmedium (or "low" for lightweight models)Focused single-domain DevOps tasks: writing a Dockerfile, editing a Terraform module, or inspecting service logs.
Tier 2: powerfulclaude-opus-4-6, o3-pro, gemini-3.1-promax (Anthropic) / xhigh (OpenAI) / highMulti-agent delegation, SRE incident root-cause correlation (CrashLoopBackOff, OOMKilled), FinOps cost optimization, security/IAM audits, and cross-service refactoring.

2. Orchestrator vs. subagent routing criteria​

JevDynamicModelRouterMiddleware passes routing_state["agent_role"] (orchestrator vs. subagent) into Jev:

  • Orchestrator (ORCHESTRATOR_ROUTE_CRITERIA): Evaluates whether the overall user request requires multi-agent synthesis (powerful), single-domain execution (standard), or reflex lookup (fast).
  • Subagents (SUBAGENT_ROUTE_CRITERIA): Evaluates the narrow delegated sub-task independently. For example, while the orchestrator uses claude-opus-4-6 (effort="max") to correlate an outage, a subagent dispatched solely to list recent Git commits or fetch a pod log can run on fast or standard, multiplying cost savings across parallel subagent fan-outs.

3. Confidence safeguards & automatic baseline restoration​

  • Confidence Safeguards (_apply_confidence_safeguards): If router confidence is < 0.55 on prompts containing high-complexity operational markers (optimi, cost, finop, crashloop, oomkill, incident, outage, architect, refactor, secur), TalkOps automatically escalates the turn to powerful (standard for subagents). If a fast classification has confidence < 0.55, it promotes to standard.
  • Baseline Restoration on /manual or /auto: When you switch from /smart back to /manual or /auto, TalkOps automatically restores your baseline model (/model) and clears any sticky checkpoint route.

Configuring your model and Smart Mode pool​

Method 1: CLI flags (-M / --model)​

Pass a single model or a comma-separated 3-tier pool (fast,standard,powerful) at startup:

# Pin a single baseline model
talkops -M anthropic:claude-sonnet-4-6

# Define a custom 3-tier pool and launch in Smart Mode
talkops --smart -M "anthropic:claude-haiku-4-5,anthropic:claude-sonnet-4-6,anthropic:claude-opus-4-6"

Method 2: Interactive slash commands (/model, /pool, /effort, /fast)​

Inside the TUI:

/model anthropic:claude-sonnet-4-6
/pool anthropic:claude-haiku-4-5,anthropic:claude-sonnet-4-6,anthropic:claude-opus-4-6
/effort high
/fast

Method 3: config.toml​

Persist your default model, 3-tier Smart Mode pool, and custom OpenAI-compatible endpoints in ~/.talkops/config.toml or .talkops/config.toml:

[models]
default = "anthropic:claude-sonnet-4-6"
reasoning_effort = "medium"
pool = [
  "anthropic:claude-haiku-4-5",
  "anthropic:claude-sonnet-4-6",
  "anthropic:claude-opus-4-6"
]

# Optional: Mix providers across tiers (e.g., Gemini Flash-Lite for Tier 0, Claude Opus for Tier 2)
# pool = [
#   "google_genai:gemini-3.1-flash-lite",
#   "anthropic:claude-sonnet-4-6",
#   "anthropic:claude-opus-4-6"
# ]

Next steps​

On this page