Multi-Turn, Memory, and Multi-Agent Evaluation

Simulated users, conversation-level success, memory recall tests, and attributing failures to the right agent in supervisor/worker systems.

🚀 advanced
⏱️ 120 minutes
👤 Bhanu Pratap Singh

· Agentic AI · 7 min read

📋 Prerequisites

  • Trajectory and Tool-Use Evaluation (previous lesson)

🎯 What You'll Learn

  • Build an LLM-driven user simulator with goals, personas, and hidden information
  • Define conversation-level success and turn-level metrics for multi-turn agents
  • Design memory evaluations for recall, update, and forgetting
  • Attribute failures to specific agents and handoffs in a multi-agent system

Single-Turn Evals Hide Conversational Failures

Everything so far assumed one user message per case. Real support conversations aren’t like that. The customer leaves out their email, the agent asks, the customer gives the wrong one, then corrects it. The customer changes their mind halfway through. The customer gets frustrated after the second clarifying question.

Failures that only show up across turns:

  • Asking for information already given, two turns earlier
  • Forgetting a constraint (“don’t escalate me, I’ve been waiting three days”)
  • Over-asking: four clarifying questions when one would do
  • Under-asking: guessing the account when the email was ambiguous
  • State drift: the agent’s understanding of the problem diverges from the user’s

To evaluate these, the test needs a counterpart that talks back.

The User Simulator

A user simulator is an LLM playing the customer. It gets a goal, a persona, and hidden information it reveals only when asked. That last part is critical. If the simulator states everything upfront, you’re back to a single-turn eval.

from dataclasses import dataclass, field

@dataclass
class UserScenario:
    goal: str                      # what the user wants, in their terms
    persona: str                   # tone, patience, expertise
    opening: str                   # first message
    hidden_facts: dict[str, str]   # revealed only if the agent asks
    constraints: list[str] = field(default_factory=list)
    max_turns: int = 8

SIMULATOR_PROMPT = """You are role-playing a customer contacting billing support.

Goal: {goal}
Persona: {persona}
Facts you know (reveal a fact ONLY if the agent asks for it or it is clearly relevant):
{facts}
Your constraints: {constraints}

Rules:
- Stay in character. Write short, realistic messages.
- Never volunteer facts the agent has not asked about.
- If your goal is fully achieved, reply with exactly: [DONE]
- If the agent has clearly failed or is going in circles, reply with exactly: [GIVE_UP]
"""

class UserSimulator:
    def __init__(self, scenario: UserScenario, llm):
        self.s, self.llm = scenario, llm
        facts = "\n".join(f"- {k}: {v}" for k, v in scenario.hidden_facts.items())
        self.system = SIMULATOR_PROMPT.format(goal=scenario.goal, persona=scenario.persona,
                                              facts=facts,
                                              constraints="; ".join(scenario.constraints) or "none")

    def reply(self, transcript: list[dict]) -> str:
        # Flip roles: the agent's messages are the 'user' from the simulator's view
        flipped = [{"role": "user" if m["role"] == "assistant" else "assistant",
                    "content": m["content"]} for m in transcript]
        return self.llm(system=self.system, messages=flipped)

Running a conversation:

def run_conversation(agent, sim: UserSimulator, state) -> dict:
    transcript = [{"role": "user", "content": sim.s.opening}]
    for turn in range(sim.s.max_turns):
        agent_msg = agent.respond(transcript, state)     # may call tools internally
        transcript.append({"role": "assistant", "content": agent_msg})
        user_msg = sim.reply(transcript)
        if user_msg.strip() in ("[DONE]", "[GIVE_UP]"):
            return {"transcript": transcript, "end": user_msg.strip(),
                    "turns": turn + 1, "final_state": state.snapshot()}
        transcript.append({"role": "user", "content": user_msg})
    return {"transcript": transcript, "end": "max_turns",
            "turns": sim.s.max_turns, "final_state": state.snapshot()}

An example scenario:

UserScenario(
    goal="Get refunded for a duplicate $49 March charge",
    persona="Frustrated, terse, has already contacted support once",
    opening="charged twice again. fix it",
    hidden_facts={
        "email": "dana@example.com",
        "month": "March",
        "previous_ticket": "Contacted support 3 days ago, no answer",
    },
    constraints=["Gets annoyed if asked more than two questions",
                 "Does not want to be transferred to a human"],
)

Simulator Pitfalls

The simulator is part of the test. If it’s too cooperative, the agent looks better than it is. If it’s inconsistent (forgets its own facts, changes its goal), you get noisy, unfair failures. Validate it:

  • Read 20 simulated conversations by hand before trusting the setup
  • Check that the simulator never reveals hidden facts unprompted (a simple grader can check this)
  • Run each scenario several times. Simulator variance stacks on top of agent variance, so you need more trials than for single-turn cases (Lesson 10).

Don’t let the simulator decide success. [DONE] is a useful stopping signal, but the simulator can be fooled by a confident agent just like a real user. Grade success from state and a separate judge, not from the simulator’s own verdict.

Conversation-Level Metrics

MetricHow to measure
Goal successState assertion (refund exists) plus judge on the final resolution
Turns to resolutionCount turns until success; compare to a reference
Clarification efficiencyNumber of agent questions vs number of hidden facts actually needed
Redundant asksAgent asked for a fact already provided earlier in the conversation
Constraint adherenceJudge checks each scenario constraint against the transcript
Abandonment rateFraction ending in [GIVE_UP] or max turns

Redundant asks are checkable without a judge if you tag facts:

def redundant_asks(transcript: list[dict], fact_patterns: dict[str, str]) -> list[str]:
    """fact_patterns: fact name -> regex that detects the agent asking for it."""
    import re
    provided, redundant = set(), []
    for m in transcript:
        if m["role"] == "user":
            for fact, _ in fact_patterns.items():
                if fact_in_message(fact, m["content"]):
                    provided.add(fact)
        else:
            for fact, pattern in fact_patterns.items():
                if fact in provided and re.search(pattern, m["content"], re.I):
                    redundant.append(fact)
    return redundant

Evaluating Memory

Agents with memory across sessions (user preferences, prior tickets, account notes) need their own tests. Three properties matter:

Recall. Information stored in session 1 is used correctly in session 2.

Update. When information changes (“I moved to the annual plan”), the agent uses the new value, not the old one.

Forgetting and isolation. Information that should be deleted is gone; one user’s memory never appears in another user’s session.

Memory tests are multi-session scenarios:

memory_case = {
    "sessions": [
        {"user": "Please always email receipts to billing@acme.example, not my personal address."},
        {"user": "Can you resend my April receipt?",
         "expect_tool_args": {"send_receipt": {"to": "billing@acme.example"}}},
        {"user": "Actually, send receipts to finance@acme.example from now on."},
        {"user": "Resend May's receipt please.",
         "expect_tool_args": {"send_receipt": {"to": "finance@acme.example"}}},
    ]
}

Isolation tests are safety tests. Seed memory for user A, run a session as user B, and assert that none of A’s data appears in B’s tool calls or responses. This belongs in the adversarial suite (Lesson 9) as a gate.

For long single conversations, also test context-window pressure: pad the conversation with many turns, then check whether an early constraint is still honored. Agents that summarize or truncate history often lose exactly the constraint that mattered.

Evaluating Multi-Agent Systems

In a supervisor/worker design, a request flows through several agents. Imagine Quill v2:

             ┌─────────────┐
 user ─────▶ │ Supervisor  │ ─────▶ response
             └──┬───────┬──┘
        route   │       │   route
          ┌─────▼──┐ ┌──▼──────────┐
          │Billing │ │ Policy/KB   │
          │ worker │ │   worker    │
          └────────┘ └─────────────┘

A wrong final answer could come from a bad routing decision, a worker error, a lossy handoff, or the supervisor misreading a worker’s result. End-to-end evaluation tells you that it failed. You need attribution to know where.

Evaluate Each Role, Then the System

  1. Routing accuracy. Step cases for the supervisor: given this request, which worker should handle it?
  2. Worker quality. Each worker evaluated in isolation with its own dataset, as a single agent.
  3. Handoff fidelity. Does the message the supervisor sends a worker contain everything the worker needs? A judge comparing the original request to the handoff message works well.
  4. Synthesis. Given correct worker outputs, does the supervisor compose a correct final response?
  5. End to end. The full system on the golden dataset.

Attribution With Traces

Because each agent’s activity is a subtree of spans (Lesson 2), you can attribute failures automatically: walk the trace and run the per-role graders on each subtree.

def attribute_failure(trace, case) -> str:
    route = first_span(trace, kind="agent", name="supervisor.route")
    if route.output["worker"] not in case.expected["acceptable_workers"]:
        return "routing"
    worker_root = first_span(trace, kind="agent", name=route.output["worker"])
    if not worker_graders_pass(worker_root, case):
        return f"worker:{route.output['worker']}"
    if not handoff_complete(route.input, worker_root.input):
        return "handoff"
    return "synthesis"

Report failures by attribution bucket:

end-to-end pass rate   81%
failures by stage:
  routing              31%
  worker:billing       22%
  handoff              38%
  synthesis             9%

A breakdown like that changes what you work on next. Here, improving handoff messages is worth more than improving the billing worker.

Multi-Agent Specific Failure Modes

  • Ping-pong: agents hand a task back and forth without progress. Grade with a max-handoffs budget.
  • Duplicated work: two workers both perform the same action (two refunds!). The no-collateral-changes and cardinality gates from earlier lessons catch this.
  • Authority confusion: a worker takes an action only the supervisor should approve. Check permissions per agent in the trace.

Key Takeaways

  • Many agent failures only appear across turns; single-turn evals can’t see them
  • A user simulator needs a goal, persona, and hidden facts revealed only when asked
  • Validate the simulator, and never let it be the judge of success; grade from state and a separate judge
  • Track turns to resolution, clarification efficiency, redundant asks, constraint adherence, and abandonment
  • Memory evals test recall, update, and isolation; isolation is a safety gate
  • In multi-agent systems, evaluate each role, then attribute failures using trace subtrees

Exercise

  1. Build a user simulator and write 10 scenarios with hidden facts and constraints for your agent
  2. Run each scenario 5 times; report goal success, mean turns, redundant asks, and abandonment
  3. Read five transcripts and note any simulator misbehavior; fix the simulator prompt
  4. If you have a multi-agent system, implement attribute_failure and report failures by stage. If not, write a two-session memory test with an update step.

Part of a structured course

Agent Evaluation & Architecture

Design and build a complete Agent Evaluation Framework — traces, golden datasets, deterministic graders, calibrated LLM judges, trajectory and safety evals, reliability metrics, production monitoring, and CI regression gates. Advanced course for AI engineers and architects shipping agents to production.

Lesson 8 of 13 ⏱ 8 weeks advanced Free

Related Tutorials

🚀advanced ⏱️ 300 minutes

Capstone: Build an Agent Evaluation Framework

Assemble every piece into one framework for a real agent — dataset, graders, judge, reliability report, CI gate, and production feedback loop.

Agentic AI6 min read
agent evaluationcapstoneproject +2
🚀advanced ⏱️ 120 minutes

Evals in CI/CD and Regression Gates

Running evals on every prompt, model, and tool change — tiered suites, baselines, statistically sound gates, and a working GitHub Actions workflow.

Agentic AI9 min read
agent evaluationci/cdgithub actions +3
🚀advanced ⏱️ 90 minutes

Deterministic Graders and Assertions

Exact match, schema validation, state checks, and environment assertions — the cheap, reliable graders that should carry most of your eval weight.

Agentic AI8 min read
agent evaluationgradersassertions +2
🚀advanced ⏱️ 120 minutes

Eval-Driven Agent Architecture

Designing an agent so it can be measured — traces, spans, structured events, replayable tool layers, and the seams where evaluators attach.

Agentic AI8 min read
agent evaluationagent architectureobservability +2