Model providers
TalkOps Kubernetes Agent (k8s-autopilot) supports more than 20 LLM providers with streaming output, structured tool calling, and native reasoning controls. You can switch models mid-conversation from the chat header without restarting the stack.
Specify a model
Reference models using provider:model or set MODEL and let k8s-autopilot infer the provider automatically:
google_genai:gemini-3.7-flash
anthropic:claude-opus-4-7
openai:gpt-5.4
deepseek:deepseek-reasoner
ollama:llama3.3
You can change the active model in three ways:
- Chat model picker: Select any configured provider and model directly in the web UI header.
- Settings UI: Add API keys under Settings → Auth & Keys to unlock models immediately.
- Environment variables: Set
MODEL(and optionallyMODEL_PROVIDER) in~/.k8s-autopilot/.env.
Automatic provider detection
When MODEL_PROVIDER is omitted, k8s-autopilot infers the provider from the model prefix:
| Model prefix | Resolved provider |
|---|---|
gemini | google_genai |
claude-, sonnet, opus, haiku | anthropic |
gpt-, o1, o3, o4, chatgpt | openai |
deepseek | deepseek |
llama | ollama |
Supported providers
| Provider | Provider ID | Auth environment variable | Custom base URL variable |
|---|---|---|---|
| Google GenAI | google_genai | GOOGLE_API_KEY | GOOGLE_GEMINI_BASE_URL |
| Anthropic | anthropic | ANTHROPIC_API_KEY | ANTHROPIC_BASE_URL |
| OpenAI | openai | OPENAI_API_KEY | OPENAI_BASE_URL |
| Google Vertex AI | google_vertexai | GOOGLE_CLOUD_PROJECT (ADC) | — |
| Azure OpenAI | azure_openai | AZURE_OPENAI_API_KEY | AZURE_OPENAI_ENDPOINT |
| AWS Bedrock | bedrock | AWS IAM / ADC | — |
| Groq | groq | GROQ_API_KEY | GROQ_BASE_URL |
| DeepSeek | deepseek | DEEPSEEK_API_KEY | DEEPSEEK_API_BASE |
| Together AI | together | TOGETHER_API_KEY | TOGETHER_API_BASE |
| Fireworks AI | fireworks | FIREWORKS_API_KEY | FIREWORKS_BASE_URL |
| OpenRouter | openrouter | OPENROUTER_API_KEY | OPENROUTER_API_BASE |
| Mistral AI | mistralai | MISTRAL_API_KEY | MISTRAL_BASE_URL |
| NVIDIA NIM | nvidia | NVIDIA_API_KEY | NVIDIA_BASE_URL |
| Perplexity | perplexity | PPLX_API_KEY | PERPLEXITY_BASE_URL |
| Cohere | cohere | COHERE_API_KEY | CO_API_URL |
| IBM watsonx | ibm | WATSONX_APIKEY | WATSONX_URL |
| HuggingFace | huggingface | HUGGINGFACEHUB_API_TOKEN | HF_INFERENCE_ENDPOINT |
| LiteLLM | litellm | LITELLM_API_KEY | — |
| xAI | xai | XAI_API_KEY | XAI_API_BASE |
| Baseten | baseten | BASETEN_API_KEY | BASETEN_BASE_URL |
| Ollama | ollama | (none required) | OLLAMA_HOST |
Extended thinking & reasoning effort
For reasoning-capable models, REASONING_EFFORT controls the thinking token budget allocated before the model emits tool calls or final answers:
REASONING_EFFORT=high # Options: low, medium, high, max
k8s-autopilot maps this setting to each provider's native reasoning parameters:
| Provider | Native parameter mapping (REASONING_EFFORT=high) |
|---|---|
| Google GenAI | thinking_level="HIGH", thinking_budget=8192 |
| Anthropic | thinking={"type": "enabled", "budget_tokens": 8192} |
| OpenAI | reasoning={"effort": "high"} |
Enterprise & local provider setup
Google Vertex AI
To authenticate via Google Cloud Application Default Credentials (ADC) or GKE Workload Identity instead of an API key:
GOOGLE_GENAI_USE_VERTEXAI=true
GOOGLE_CLOUD_PROJECT=your-gcp-project-id
GOOGLE_CLOUD_LOCATION=us-central1
Air-gapped local inference with Ollama
To run k8s-autopilot completely offline using local open-weights models:
ollama pull qwen2.5-coder:32b
# In ~/.k8s-autopilot/.env:
MODEL=qwen2.5-coder:32b
MODEL_PROVIDER=ollama
OLLAMA_HOST=http://host.docker.internal:11434
Multi-step Kubernetes diagnostics and MCP tool schemas work best with models of 14B parameters or larger (such as qwen2.5-coder:32b or llama3.3:70b).