Context Management & Reliability (15%)
What the exam tests: whether you keep critical facts intact over long sessions, escalate for the right reasons, propagate errors usefully across agents, explore large codebases without exhausting context, calibrate human review, and preserve source attribution through synthesis. About 9 of 60 items, but this domain also shapes answers in four scenarios.
6.1 Keeping critical facts through long conversations (Task 5.1) Three forces erode context:
Progressive summarisation turns "refund of US$84.50 on order 7731, promised by 14 March" into "customer wants a refund".
Lost in the middle: models handle the start and end of long inputs reliably and are more likely to miss findings in the middle.
Tool-result bloat: an order lookup that returns 40+ fields when 5 matter fills the window with noise.
The fixes:
Keep a case facts block (amounts, dates, order numbers, statuses, what the customer was promised) outside the summarised history and include it in every prompt. For multi-issue sessions, keep one structured entry per issue.
Trim tool output to the relevant fields before it enters the conversation; a PostToolUse hook is a natural place.
Order aggregated input deliberately: put a summary of key findings first and use clear section headers for the details. Ask upstream agents for structured output (key facts, citations, relevance scores, dates, source locations) rather than long prose and reasoning, especially when the downstream agent has a small context budget.
Remember that the API is stateless: each request must carry the conversation history the model needs.
6.2 Escalation and ambiguity (Task 5.2)
- Escalate Do not escalate (just because)
- The customer asks for a human: do it immediately,
- without investigating first
The customer is frustrated but the fix is within your ability: acknowledge and offer to resolve; escalate if they ask again
- Policy is silent or ambiguous, such as a competitor
- price match when policy covers only your own site
- The case is long or complex but policy covers it
- The agent cannot make meaningful progress Sentiment is negative
A policy exception is needed The model's self-reported confidence is low The guide calls sentiment and self-reported confidence unreliable proxies for case complexity. The proportionate fix for poor escalation calibration is explicit escalation criteria in the system prompt, with few-shot examples of escalate versus resolve. When get_customer returns several matches, ask for another identifier (email, order number, postcode). Never pick one by heuristic.
6.3 Error propagation across agents (Task 5.3)
An error report a coordinator can act on includes the failure type, what was attempted, any partial results and possible alternatives. Two anti-patterns sit at opposite extremes: Silent suppression: catching a timeout and returning an empty result marked as success. The coordinator cannot recover from a failure it never sees.
Total failure: letting one subagent's exception terminate the whole workflow when the rest could continue.
The middle path: subagents retry transient failures themselves, pass up what they cannot fix with context, and the final synthesis carries coverage annotations saying which findings are well supported and which areas have gaps because sources were unavailable.
6.4 Large codebase exploration (Task 5.4)
Signs of context degradation in a long session: inconsistent answers, and references to "typical patterns" instead of the specific classes found earlier. Counter-measures:
Delegate narrow questions to subagents ("find all test files", "trace the refund flow's dependencies") so their verbose output stays in their context while the main agent keeps the big picture. Keep a scratchpad file of key findings and consult it for later questions.
Summarise each phase before starting the next and seed new subagents with that summary. Use /compact when the window fills with discovery output.
Design for crash recovery: each agent exports its state to a known location, and the coordinator loads a manifest on resume and injects it into the agents' prompts.
6.5 Human review and confidence calibration (Task 5.5)
A 97% overall accuracy figure can hide a document type or field that is wrong half the time. Before reducing human review:
Figure 7. Route by calibrated confidence, and keep sampling what you automate
Measure accuracy by document type and by field, not just in aggregate.
Have the model output field-level confidence, then set review thresholds against a labelled validation set, because raw model confidence is not calibrated.
Route low-confidence fields and documents with ambiguous or contradictory sources to humans first, since reviewer time is limited. Keep a stratified random sample of high-confidence extractions under review to track error rates and catch new error patterns.
6.6 Provenance and uncertainty in synthesis (Task 5.6)
Attribution is easily lost when findings are compressed. Protect it structurally: Subagents output claim-source mappings: claim, supporting excerpt, source URL or document name, and publication or collection date. Synthesis must keep and merge them. When credible sources disagree, report both values with their sources and let the coordinator decide how to reconcile; do not silently pick one. Dates prevent false contradictions: a 2023 figure and a 2025 figure may both be right. Structure reports into well-established versus contested findings, and render content to fit: financial data as tables, news as prose, technical findings as structured lists.
CONTEXT MANAGEMENT & RELIABILITY (15%)
Worked examples
Worked example 6.A: The case facts block
Situation. A billing dispute runs 45 turns. Early on, the customer says they were charged US$129.00 twice on 3 October for order ORD-55120 and were promised a callback by Friday. By turn 40, the agent's summary says "customer has a billing concern" and it offers a US$12.90 credit.
Diagnosis. Progressive summarisation compressed exact values into vague prose, and the important facts now sit in the middle of a long context. Fix. Maintain a structured block, updated by code or by a dedicated extraction step, and place it at the top of every request, outside the summarised history:
<case_facts> customer_id: C-88213 (verified via get_customer)
issue_1: duplicate charge | order ORD-55120 | US$129.00 x2 | 2026-10-03 | status: refund pending approval
issue_2: callback promised by 2026-10-09 (Friday) | status: open customer_expectation: full refund of one charge, not a credit
</case_facts>
The history can be summarised freely because the facts that matter are never summarised.
Worked example 6.B: Trimming tool results
Situation. lookup_order returns 43 fields, including warehouse codes, internal SKU mappings and audit timestamps. After six lookups, a third of the context is order metadata the agent never uses. Fix. Trim before the result enters the conversation, either in the MCP server or in a PostToolUse hook. For a returns agent, keep order ID, status, items (name, quantity, price), delivery date and return-window end date. Six fields instead of 43 keeps context for reasoning and reduces lost-in-the-middle effects.
Worked example 6.C: Three escalation calls
Customer says Right action Reasoning
"I want to speak to a person." Escalate now, with a hand-off summary Explicit request; do not investigate first
Hand-off summary format. Humans picking up the case do not see the transcript: {"customer_id": "C-88213", "issues": ["duplicate charge ORD-55120 US$129.00"], "root_cause": "payment retried after gateway timeout; both captures settled", "actions_taken": ["verified identity", "confirmed two captures in billing"], "recommended_action": "refund one capture of US$129.00",
"reason_for_escalation": "refund above agent limit requires approval", "customer_sentiment_note": "frustrated; promised callback by Friday"} Worked example 6.D: Error propagation in the research system
Situation. The search subagent hits a rate limit on one provider and a timeout on another. Today it returns {"results": []}, and the report confidently says "no recent studies exist on this topic".
Diagnosis. Silent suppression turned an access failure into a false negative. Fix. The subagent retries the transient failures itself (for example once, with a pause). If they still fail, it returns:
{"status": "partial",
"completed_queries": ["AI hiring regulation EU 2025"],
"failed_queries": [{"query": "AI hiring regulation US state laws 2025", "failure": "timeout", "attempts": 2,
"alternatives": ["try the legal-database tool", "narrow to the 5 largest states"]}], "results": [ ... ]}
The coordinator can now retry with a different tool, narrow the query or proceed and annotate the report: "US state coverage is incomplete because sources were unavailable." Neither silence nor a crash.
Worked example 6.E: Two hours into a legacy codebase
Situation. An engineer has used one Claude Code session for two hours to understand a payments monolith. Answers are getting vague ("payment processors typically...") and contradict earlier findings about LedgerService.
Fix, in order of effort.
Ask Claude to write notes/payments-findings.md: key classes, flows, invariants and open questions found so far. Refer to it for later questions.
Run /compact to shrink the verbose discovery output.
CONTEXT MANAGEMENT & RELIABILITY (15%)
For the next phase ("trace every write to the ledger"), delegate to a subagent or the Explore subagent, seeded with the findings file, so its grep-and-read output stays out of the main session. If the session is badly degraded, start a new one with the findings file as its starting context. For long automated multi-agent jobs, apply the same idea as crash recovery: each agent writes its state to a known path, and the coordinator keeps a manifest of what is done, so a restart can load the manifest and continue instead of starting over.
Worked example 6.F: Is 97% good enough to automate?
Situation. An insurance-claims extractor scores 97% field accuracy on a 2,000-document validation set. The business wants to drop human review for extractions with confidence above 0.9. Analysis, broken down.
- Document type Share Accuracy
- Standard claim form 70% 99.4%
- Repair invoice 20% 96.8%
- Handwritten supplement 10% 81.0%
By field, incident_date is 99% but policy_number on handwritten supplements is 64%. The aggregate hides a weak segment. Plan.
Calibrate: on the labelled set, check what accuracy a stated confidence of 0.9 actually corresponds to, per document type. Set thresholds per type, not one global number. Automate standard claim forms above the calibrated threshold first.
Keep handwritten supplements in human review; prioritise them, along with any document where sources conflict.
Keep reviewing a stratified random sample of automated high-confidence extractions from every type, to keep measuring error rates and spot new failure patterns.
Worked example 6.G: Synthesis that keeps its sources
Situation. Two analysis subagents report the share of companies using AI in recruiting: 31% (survey of 1,200 firms, fieldwork March 2024) and 38% (survey of 600 firms, fieldwork January 2025). The synthesis agent writes "about a third of companies (35%)".
What went wrong. It averaged two figures with different dates and methods and dropped both
sources.
Fix. Require claim-source mappings with dates and methodology from every analysis subagent, and instruct synthesis to preserve them:
### Adoption of AI in recruiting (contested)
- 31% of firms (Survey A, n=1,200, March 2024)
- 38% of firms (Survey B, n=600, January 2025)
The difference may reflect growth over 10 months as well as different samples. Put the report's well-established findings in one section and contested ones in another, and use tables for numeric comparisons rather than forcing everything into prose.
Deep dive: a context budget for a support agent
A practical way to apply this domain is to budget the context window explicitly. The numbers below are illustrative, for a single support conversation:
Summary of older turns 8% Summarised; never holds the only copy of a fact Headroom for the response 10% Reserved
When a conversation approaches its budget, summarise the oldest turns, never the case facts, and never the policy block.
Deep dive: position effects in aggregated input
When a coordinator hands a synthesis agent results from ten subagents, order and structure matter:
Start with a short key findings list across all results.
Follow with each subagent's results under a clear header (## Source group 3: regulatory filings).
Put the task instructions and output format at the end, close to where the model writes. This mitigates the lost-in-the-middle effect, because the most important information appears at the start and the instructions at the end, while headers make the middle navigable.
Common traps
Escalating on sentiment or on low self-reported confidence.
Investigating first when the customer has explicitly asked for a human.
CONTEXT MANAGEMENT & RELIABILITY (15%)
Picking the most likely customer from several matches.
Returning empty results as success, or killing the workflow on one failure.
Trusting aggregate accuracy before automating.
Letting synthesis drop sources or average conflicting figures.
Check yourself
A customer says "I want to talk to a person" in their first message. What does the agent do?
After 40 turns, the agent quotes the wrong refund amount. What structure prevents this?
Two reputable sources give 31% and 38% for the same statistic. What should the report do?
Overall extraction accuracy is 97%. What do you check before cutting human review?