Skip to main content

HITL governance & approval modes

TalkOps Kubernetes Agent (k8s-autopilot) enforces Human-in-the-Loop (HITL) governance before executing any operation that modifies cluster state, application routing, or observability configurations. Read-only inspection tools run automatically, while state-changing mutations pause and present an interactive A2UI approval card with parameter diffs and blast-radius classification.

Read-only vs. gated actions​

CategoryBehaviorExamples
Read-only inspectionExecutes automatically in all modesListing pods, deployments, services, or namespaces; viewing Helm release values; running PromQL, LogQL, or TraceQL queries
Gated mutationsPauses for human review in Manual mode (and when risky in Auto mode)Installing, upgrading, or rolling back Helm charts; scaling workloads; syncing ArgoCD apps; updating Traefik traffic splits; creating Alertmanager silences

Three approval modes​

Switch approval modes at any time from Settings → Runtime Config or configure the default via APPROVAL_MODE:

ModeBehaviorRecommended environment
Manual (default)Pauses and renders an interactive Approval Card before every mutating tool callProduction clusters and shared staging environments
AutoAuto-approves read-only tools and low-risk operations; prompts before high-impact or destructive changesDaily development and staging clusters
YOLOExecutes all tool calls without interactive prompts after an explicit initial risk acknowledgementDisposable local clusters (kind, minikube) and ephemeral CI sandboxes

Manual mode (default)​

In Manual mode, every mutating operation pauses the LangGraph workflow and renders an interactive Approval Card in the web console showing:

  • The proposed operation and target MCP server tool
  • Target cluster context, namespace, and resource name
  • Risk badge (Low, Medium, High, or Critical)
  • Diagnostic rationale and parameter diff

Click Approve to resume execution or Reject to cancel and provide alternative instructions.

Graceful rejection handling

When you reject a proposed plan, the operator does not blindly retry the same tool call. It asks how you want to adjust the parameters—and if you reject twice in succession, it pauses to let you reframe your objective.

Auto mode​

Auto mode evaluates incoming tool calls through a three-stage decision pipeline:

  1. Deterministic pattern check: Tier 1 read-only MCP tools and safe inspection commands (kubectl get, kubectl describe, helm list, helm status) execute immediately. Known destructive verbs (delete, uninstall, drain, purge) immediately route to human approval.
  2. AI safety classifier: Intermediate mutations (such as updating a ConfigMap or scaling a non-production deployment) are evaluated against the target namespace, resource type, and blast radius.
  3. Fail-safe fallback: If the classifier is uncertain or unavailable, it fails safe and prompts you with an Approval Card.
RBAC boundary

Approval modes operate at the agent orchestration layer and do not replace Kubernetes Role-Based Access Control (RBAC). Always bind k8s-autopilot to least-privilege Kubernetes ServiceAccounts.

YOLO mode​

YOLO mode executes all tool calls without pausing for confirmation. Enabling YOLO mode requires acknowledging an explicit risk confirmation modal in the UI.

Interactive clarifications (ask_user)​

Separate from mutation approvals, operators invoke the ask_user tool whenever a request is ambiguous:

  • Selecting between multiple target namespaces (staging vs. production)
  • Choosing a specific Helm chart version or rollback revision
  • Disambiguating between similarly named services

Clarifications render as interactive option cards in the chat stream so you can select validated values with a single click.

Shell and MCP security tiers​

AST shell validation​

Whenever an operator runs a shell command, k8s-autopilot validates the command across four layers:

  1. Prefix and verb matching against known read-only and destructive patterns.
  2. Full AST parsing across pipes (|), logical chains (&&, ||), and subshells so destructive commands hidden inside pipelines (for example, echo ok && kubectl delete ns prod) are always caught.
  3. Injection detection for command substitution, backticks, and unexpected redirections.
  4. Allowlist validation against configured command prefixes.

Four-tier MCP tool profiling​

Every tool exposed by a connected MCP server is classified at registration time using MCP tool annotations (readOnlyHint, destructiveHint) and semantic verb profiling:

Security tierClassificationRuntime behaviorExample tools
Tier 1 — Read-onlyIdempotent state inspectionRuns automatically in all modesget_pods, query_metrics, list_releases
Tier 2 — Low-impactDry-runs or transient metadata updatesAuto mode evaluates; Manual mode asksdry_run_install, add_label, annotate
Tier 3 — MutatingModifies live cluster or GitOps statePrompts for approval in Manual and Auto modesinstall_chart, scale_deployment, sync_app
Tier 4 — DestructiveIrreversible deletion or node evictionAlways prompts with elevated Critical warningdelete_namespace, uninstall_release, drain_node

When running headless without a web UI, HeadlessMCPGuardMiddleware blocks all Tier 3 and Tier 4 tool calls unless explicitly pre-authorized.

Domain-specific operator guardrails​

Each specialist operator enforces domain-level safety invariants on top of the global approval gate:

  • Helm Operator: Runs a dry-run validation before every chart installation, defaults to --reuse-values on upgrades, and verifies exact revision numbers from release history before rolling back.
  • App Operator: Caps autonomous canary promotions at 50% traffic weight, requires human approval before creating TCP routes, caps traffic mirroring at 20%, and guarantees at least one backend service retains non-zero traffic weight.
  • K8s Operator: Elevates risk classification for prod, production, and kube-system namespaces, redacts Kubernetes Secret values in chat responses, and requires explicit confirmation before kubectl exec.
  • Observability Operator: Previews the matching alert blast radius before creating Alertmanager silences (capped at 24 hours), runs OpenTelemetry Collector and Tempo CRD mutations in dry_run mode first, and keeps Loki strictly read-only.

Double-gate verification​

Every state-changing operation passes through two independent checks:

  1. Gate 1 (Operator plan confirmation): The operator presents its structured execution plan and waits for your approval.
  2. Gate 2 (Middleware tool interception): When the operator invokes the underlying MCP tool, the framework middleware independently verifies the tool's security tier and approval state before forwarding the call to the MCP server.

Next steps​