Skip to main content

Goals and rubrics

When you assign TalkOps Kubernetes Agent (k8s-autopilot) a multi-step infrastructure task—such as deploying a stateful workload with Prometheus monitoring and network isolation—the agent proposes a Goal with concrete acceptance criteria, tracks execution against a live checklist, and verifies completion through an independent Rubric Grader before marking the task done.

End-to-end workflow​

1
Describe a multi-step outcome

Give the agent a complex infrastructure objective in natural language:

Deploy a 3-replica Redis cluster with persistent storage, create a Prometheus ServiceMonitor for the exporter, and enforce a NetworkPolicy restricting ingress to the backend namespace.

2
Review the proposed goal and acceptance criteria

The agent inspects your cluster state and proposes verifiable acceptance criteria in an interactive Goal Card:

  • Redis StatefulSet deployed with 3/3 ready replicas and bound PersistentVolumeClaims
  • ServiceMonitor resource created and matching the Redis exporter labels
  • NetworkPolicy applied in the target namespace restricting ingress on port 6379

You can Confirm the criteria as-is, Edit individual items, or Reject with feedback.

3
Track tactical progress

Once confirmed, the acceptance criteria lock in as the active rubric, and the agent maintains a real-time tactical checklist (pending, in_progress, completed) in the UI as it delegates work across operators.

4
Independent rubric grading

When the worker agent stages a completion request, an independent Rubric Grader inspects the live cluster state against every criterion:

  • All criteria pass: The goal transitions to Complete.
  • Deficiency detected: The grader returns a structured deficiency report citing the exact failed criterion, and the worker agent remediates the gap automatically.

Goal lifecycle states​

StatusMeaningControlled by
ActiveAccepted criteria are locked in and driving operator executionUser confirmation
BlockedThe agent encountered an external blocker and requires user inputWorker agent
PausedGoal is preserved in thread state but temporarily suspendedUser action
CompleteEvery acceptance criterion has passed independent rubric verificationRubric Grader

Rubric grader architecture​

To prevent an agent from prematurely declaring success, the rubric grader enforces three invariants:

  1. Read-only verification: The grader can only inspect cluster resources, command outputs, and files referenced in the worker's evidence—it cannot mutate state.
  2. Cross-model grading: You can configure a separate model for grading (for example, grading a Gemini worker with Claude or GPT-4o) to eliminate self-evaluation bias.
  3. Mid-task amendments: You can amend an active goal at any point during a conversation (for example, "Also require TLS encryption on Redis client connections"), and the agent updates the rubric without losing completed steps.