Course Content
Eval-Driven Agent Architecture
Designing an agent so it can be measured — traces, spans, structured events, replayable tool layers, and the seams where evaluators attach
You Can’t Evaluate What You Can’t See
Most agent code is written to produce an answer. Evaluation needs more than the answer: it needs every decision, every tool call and its arguments, every observation, the state of the world before and after, the token counts, and the timings.
If that information is buried in print statements or lost inside a framework’s internals, your evaluation ceiling is the final string. This lesson is about raising that ceiling architecturally, so that evaluation is a property of how the agent is built, not an afterthought bolted on.
The principle is simple: design the agent so that a complete, structured record of every run falls out for free.
The Four Seams
There are four places an evaluator needs to attach. Design each as an explicit boundary in the code.
┌──────────────────────────────────────────┐
input ──▶ │ Agent loop │ ──▶ response
│ ┌────────────┐ ┌───────────────┐ │
│ │ Model call │ ───▶ │ Tool executor │ │
│ └────────────┘ └───────┬───────┘ │
│ ▲ ▼ │
│ └─────── observation ─┘ │
└───────────────┬──────────────────────────┘
▼
Environment / state- Model seam. Every LLM call goes through one function. You can record its inputs and outputs, swap the model, or replay a recorded response.
- Tool seam. Every tool call goes through one executor. You can record it, mock it, inject failures, or inject adversarial content.
- State seam. The world the agent changes (refunds, tickets, files) lives behind an interface you can snapshot and assert against.
- Trace seam. Every step emits a structured event into a trace that evaluators read after the run.
Get these four right and every lesson after this one gets easier. Get them wrong and you’ll spend your time scraping logs.
A Minimal Trace Model
You don’t need a vendor SDK to start. A trace is a tree of spans. Each span has a type, a start and end time, inputs, outputs, and attributes.
from __future__ import annotations
import time, uuid
from dataclasses import dataclass, field
from typing import Any, Literal
SpanKind = Literal["agent", "llm", "tool", "retrieval", "guardrail"]
@dataclass
class Span:
name: str
kind: SpanKind
trace_id: str
span_id: str = field(default_factory=lambda: uuid.uuid4().hex[:16])
parent_id: str | None = None
start: float = field(default_factory=time.time)
end: float | None = None
input: Any = None
output: Any = None
attributes: dict[str, Any] = field(default_factory=dict)
error: str | None = None
@property
def duration_ms(self) -> float:
return ((self.end or time.time()) - self.start) * 1000
@dataclass
class Trace:
trace_id: str = field(default_factory=lambda: uuid.uuid4().hex)
spans: list[Span] = field(default_factory=list)
def of_kind(self, kind: SpanKind) -> list[Span]:
return [s for s in self.spans if s.kind == kind]
def tool_calls(self) -> list[tuple[str, dict]]:
return [(s.name, s.input) for s in self.of_kind("tool")]
def total_tokens(self) -> int:
return sum(s.attributes.get("tokens.total", 0) for s in self.of_kind("llm"))The attribute names here mirror the OpenTelemetry GenAI semantic conventions (gen_ai.request.model, gen_ai.usage.input_tokens, and so on) closely enough that exporting to Langfuse, Arize Phoenix, LangSmith, or any OTel backend later is a mapping exercise, not a rewrite.
A context manager makes instrumentation nearly invisible:
from contextlib import contextmanager
class Tracer:
def __init__(self):
self.trace = Trace()
self._stack: list[Span] = []
@contextmanager
def span(self, name: str, kind: SpanKind, input: Any = None, **attrs):
s = Span(
name=name, kind=kind, trace_id=self.trace.trace_id,
parent_id=self._stack[-1].span_id if self._stack else None,
input=input, attributes=dict(attrs),
)
self.trace.spans.append(s)
self._stack.append(s)
try:
yield s
except Exception as e:
s.error = f"{type(e).__name__}: {e}"
raise
finally:
s.end = time.time()
self._stack.pop()The Model Seam
All model calls go through one client wrapper. It records the request and response as an llm span and can be swapped for a replay client in tests.
class ModelClient:
def __init__(self, tracer: Tracer, model: str, llm_fn):
self.tracer, self.model, self.llm_fn = tracer, model, llm_fn
def complete(self, messages: list[dict], tools: list[dict]) -> dict:
with self.tracer.span("chat", "llm", input=messages,
**{"gen_ai.request.model": self.model}) as s:
resp = self.llm_fn(model=self.model, messages=messages, tools=tools)
s.output = resp["message"]
s.attributes["tokens.input"] = resp["usage"]["input_tokens"]
s.attributes["tokens.output"] = resp["usage"]["output_tokens"]
s.attributes["tokens.total"] = sum(resp["usage"].values())
return resp["message"]llm_fn is whatever calls your provider. Keeping it injected means the same agent runs against Claude, GPT, or a local model, and an eval can compare them with no code changes.
The Tool Seam: Live, Mock, Replay
This is the most valuable seam. Evaluation needs to run tools in three modes:
- Live: real APIs, for end-to-end smoke tests in staging
- Mock: deterministic fakes backed by fixture data, for the bulk of offline evals
- Replay: return exactly what a recorded production trace returned, to reproduce a failure
from typing import Callable, Protocol
class ToolBackend(Protocol):
def call(self, name: str, args: dict) -> Any: ...
class ToolExecutor:
def __init__(self, tracer: Tracer, backend: ToolBackend, hooks=None):
self.tracer, self.backend = tracer, backend
self.hooks = hooks or [] # fault and adversarial injection, Lesson 9
def run(self, name: str, args: dict) -> Any:
with self.tracer.span(name, "tool", input=args) as s:
result = self.backend.call(name, args)
for hook in self.hooks:
result = hook(name, args, result)
s.output = result
return result
class ReplayBackend:
"""Returns tool outputs recorded in a previous trace, in order."""
def __init__(self, recorded: Trace):
self._queue = [(s.name, s.input, s.output) for s in recorded.of_kind("tool")]
def call(self, name, args):
if not self._queue:
raise RuntimeError(f"replay exhausted at {name}")
exp_name, exp_args, output = self._queue.pop(0)
if exp_name != name:
raise RuntimeError(f"replay diverged: expected {exp_name}, got {name}")
return outputReplay mode is how you turn a production bug report into a reproducible test case: take the trace, replay the tool outputs, and check whether a new prompt fixes the decision that went wrong.
The State Seam
For Quill, the “world” is customers, invoices, and refunds. Put it behind an interface the mock backend reads and writes, and that evaluators can inspect.
@dataclass
class BillingState:
customers: dict[str, dict]
invoices: dict[str, dict]
refunds: list[dict] = field(default_factory=list)
escalations: list[dict] = field(default_factory=list)
def snapshot(self) -> dict:
import copy
return copy.deepcopy(self.__dict__)
class MockBillingBackend:
REFUND_LIMIT = 200.0
def __init__(self, state: BillingState):
self.state = state
def call(self, name, args):
s = self.state
if name == "lookup_customer":
return next((c for c in s.customers.values() if c["email"] == args["email"]),
{"error": "not_found"})
if name == "get_invoices":
return [i for i in s.invoices.values() if i["customer_id"] == args["customer_id"]]
if name == "issue_refund":
inv = s.invoices.get(args["invoice_id"])
if inv is None:
return {"error": "invoice_not_found"}
if args["amount"] > self.REFUND_LIMIT:
return {"error": "requires_approval"}
s.refunds.append(dict(args))
return {"status": "refunded", "refund_id": f"R{len(s.refunds)}"}
if name == "escalate_to_human":
s.escalations.append(dict(args))
return {"status": "escalated"}
if name == "search_kb":
return [{"title": "Duplicate charges", "body": "Refund the duplicate within 30 days."}]
return {"error": f"unknown_tool:{name}"}Now an outcome grader can ask a precise question: does state.refunds contain exactly one refund, for invoice INV-2291, for $49? No parsing of prose required.
Putting the Loop Together
import json
@dataclass
class RunResult:
response: str
trace: Trace
final_state: dict
stopped_reason: str # "final_answer" | "max_steps" | "error"
class QuillAgent:
def __init__(self, model: ModelClient, tools: ToolExecutor,
tracer: Tracer, system_prompt: str, max_steps: int = 10):
self.model, self.tools, self.tracer = model, tools, tracer
self.system_prompt, self.max_steps = system_prompt, max_steps
def run(self, user_message: str, state: BillingState) -> RunResult:
messages = [{"role": "system", "content": self.system_prompt},
{"role": "user", "content": user_message}]
with self.tracer.span("quill", "agent", input=user_message) as root:
for step in range(self.max_steps):
msg = self.model.complete(messages, TOOL_SCHEMAS)
messages.append(msg)
if not msg.get("tool_calls"):
root.output = msg["content"]
return RunResult(msg["content"], self.tracer.trace,
state.snapshot(), "final_answer")
for call in msg["tool_calls"]:
result = self.tools.run(call["name"], call["arguments"])
messages.append({"role": "tool", "tool_call_id": call["id"],
"content": json.dumps(result)})
root.error = "max_steps"
return RunResult("", self.tracer.trace, state.snapshot(), "max_steps")Notice what an eval gets from a single run():
response: for response-quality graderstrace: every LLM and tool span, for trajectory, cost, and latency gradersfinal_state: for outcome assertionsstopped_reason: for completion-rate metrics
Architecture Decisions That Help (or Hurt) Evaluation
Structured outputs over free text. If the agent must end with a JSON object ({"resolution": "refunded", "invoice_id": ...}) alongside its message, you can grade the decision deterministically and the prose separately.
Explicit stop reasons. “Final answer,” “max steps,” “tool error,” and “guardrail blocked” are different failures. Collapse them into “it returned something” and your completion metric becomes meaningless.
Config as data. System prompt, model name, temperature, tool list, and max steps should be one versioned config object. Every eval result is then tagged with exactly what produced it, which is what makes comparing two versions possible (Lesson 12).
@dataclass(frozen=True)
class AgentConfig:
model: str
system_prompt_version: str
temperature: float = 0.0
max_steps: int = 10
tools: tuple[str, ...] = ("lookup_customer", "get_invoices", "search_kb",
"issue_refund", "escalate_to_human")
def fingerprint(self) -> str:
import hashlib, json
return hashlib.sha256(json.dumps(self.__dict__, sort_keys=True).encode()).hexdigest()[:12]Avoid hidden framework state. Agent frameworks are fine, but make sure you can get the full message history and every tool call out of them. If a framework hides retries or silently truncates context, your traces will lie to you.
Deterministic IDs and clocks in eval mode. Inject the clock and ID generator. Otherwise “refund created at 14:03:22” makes snapshot comparisons flaky.
Where Existing Tools Fit
| Seam | What to use |
|---|---|
| Trace | OpenTelemetry with GenAI conventions; Langfuse, Arize Phoenix, LangSmith, Braintrust, Weights & Biases Weave as backends |
| Model | Your own thin wrapper; LiteLLM if you need many providers |
| Tool | Your own executor; MCP servers can sit behind it as a backend |
| State | Your domain code, a test database, or containerized sandboxes for code and computer-use agents |
The pattern is the same regardless: own the seams in your code, and let tools consume what flows through them.
Key Takeaways
- Evaluation quality is capped by what the architecture exposes. Design for observability from day one.
- Build four explicit seams: model, tool, state, and trace
- A run should return the response, the full trace, the final state, and a stop reason
- Tool backends should support live, mock, and replay modes
- Version the agent’s configuration as data, so every eval result knows exactly what it measured
Exercise
Refactor an agent you’ve built (or implement Quill from the snippets above) so that:
- Every LLM and tool call produces a span
- Tools run against a mock backend backed by fixture state
run()returns aRunResultwith trace, final state, and stop reason- You can take one recorded trace and replay it through
ReplayBackend
Print a trace as an indented tree. If you can read the agent’s entire decision process from that tree, you’re ready for Lesson 3.
