Practice exam 2 (full length) · CCAR-P

This exam matches the real one in length and mix: 63 items, 120 minutes, domains in proportion to the official weights (Integration 12, Solution Design 11, Evaluation 10, Governance 9, Stakeholder and Lifecycle 9, Models and Context 8, Developer Productivity 4). Items are mixed across domains, as on exam day. Sit it in one session, then use the answer key to score by domain.

Questions 1 to 21

An HR team wants an assistant that answers questions from a 60-page employee handbook that changes twice a year. Which architecture is most appropriate?

A. A multi-agent system with one agent per handbook chapter
B. An augmented LLM: the handbook in a cached prompt, or simple retrieval if it grows
C. An autonomous agent with web search
D. A fine-tuned model retrained after each handbook change

A support agent was given shell access to a production server so it can read application logs. What is the best change?

A. Replace shell access with a read-only log query tool scoped to the application's logs
B. Keep shell access but log every command
C. Add an instruction not to run destructive commands
D. Require the agent to explain each command before running it

A summarizer must output nine fields for a downstream database, but about 3% of outputs have missing or renamed fields. Which change is most effective?

A. Add "follow the format exactly" to the end of the prompt
B. Upgrade to the most capable model
C. Add two more examples of the format
D. Define the fields as a JSON schema through structured output or a tool, and validate before writing

A new prompt raises the average rubric score from 4.1 to 4.3 but increases critical errors from 0.4% to 1.2%. The threshold for critical errors is 0.5%. What should the architect do?

A. Ship it, because the average improved
B. Ship it to half of users
C. Do not ship; investigate the critical errors and iterate until both thresholds are met
D. Lower the critical-error threshold to 1.2%

A recruiting agent reads uploaded resumes and can email candidates. A resume contains hidden text instructing the agent to send all other candidates' resumes to an outside address. Which control best prevents harm?

A. A system prompt rule to ignore instructions in resumes
B. Restrict emails to the candidate on the current application and require recruiter approval before sending
C. Log all emails for weekly review
D. Use the most capable model tier

A business sponsor says the only requirement is that the assistant be "accurate." What should the architect do next?

A. Work with the sponsor to define measurable criteria with examples of acceptable and unacceptable answers
B. Start building and measure accuracy later
C. Choose the most capable model to maximize accuracy
D. Promise 99% accuracy to set a high bar

To improve answer quality, a team raised retrieval from 5 to 40 chunks. Quality barely changed, latency rose 70%, and cost doubled. What is the better approach?

A. Raise to 80 chunks
B. Remove retrieval and rely on the model's knowledge
C. Retrieve a larger candidate set, rerank, and pass only the best few chunks
D. Switch to a larger context window model

A workflow must respond within a p95 of 2 seconds. The design chains four model calls that each take about 1.5 seconds. What should the architect do?

A. Accept the SLA miss
B. Add a fifth call to verify the answer
C. Move all four calls to the most capable tier
D. Redesign: run independent steps in parallel, merge steps, use a faster tier where evaluations allow, and stream output

Engineers report that Claude Code behaves differently across the team's repositories: some run tests, some do not, some use the wrong build command. What is the best fix?

A. Ask each engineer to write their own instructions
B. Commit a concise CLAUDE.md and shared project settings to each repository from a team template
C. Disable Claude Code in repositories without tests
D. Use a larger model

A team plans to use an LLM judge to grade 10,000 responses a week. What must be done before its scores are trusted?

A. Measure its agreement with expert human labels on a representative sample and refine the rubric
B. Use the largest model as the judge
C. Ask the judge to be strict
D. Nothing; LLM judges are reliable by default

A customer submits a GDPR erasure request. Their conversations with the assistant are stored in application logs and an analytics warehouse. What does the architecture need?

A. Nothing, because logs are exempt
B. A system prompt that forgets the customer
C. The ability to find and delete or anonymize that customer's data in every store, following the retention policy
D. Encryption of the logs, which satisfies erasure

A multi-turn assistant has a long system prompt and a conversation that grows each turn. Where should cache breakpoints go to reuse the most?

A. Before the system prompt
B. Only after the latest user message
C. Nowhere; multi-turn chats cannot use caching
D. After the system prompt and after the stable earlier turns of the conversation Which TWO practices most improve the security of an enterprise MCP server?
A. Accept only tokens issued for this server (validate the audience)
B. Forward the client's token to downstream APIs
C. Use short-lived, scoped credentials for downstream systems
D. Disable logging to protect privacy

A single prompt that writes a full market-entry report keeps skipping sections and mixing facts across countries. What is the best redesign?

A. Make the prompt longer and more detailed
B. Decompose into a chain: research per country, outline, write each section, then a review step against a checklist
C. Use a multi-agent debate between three agents
D. Increase max output tokens

A steering committee asks how a pilot is going. Which artifact best answers them?

A. A list of prompts used
B. Raw traces from production
C. A scorecard of agreed success metrics against targets and baseline, with risks and decisions needed
D. The architecture diagram

Time to first token is high on every request. Each request begins with the same 12,000-token instruction block. What is the best fix?

A. Enable prompt caching with the instruction block as a stable prefix
B. Shorten the instruction block by half
C. Switch to the most capable tier
D. Disable streaming

A legal assistant drafted a brief that cited two cases that do not exist. Which control is most effective?

A. Ask the model to be careful with citations
B. Use a higher temperature
C. Ban the assistant from legal work
D. Verify every citation against the legal research database in code and require attorney review before filing

Users ask an assistant for "industrial motors over 220 volts with IP65 rating under $2,000." Product specs are structured. Which retrieval approach fits?

A. Embedding search over product descriptions only
B. Structured queries or metadata filters on the spec fields, with semantic search for freetext needs
C. Load the full catalog into every prompt
D. A multi-agent system per product category

When is an evaluator-optimizer pattern most appropriate?

A. When there are no clear quality criteria
B. When latency must be minimal
C. When clear criteria exist and iterative critique measurably improves output
D. When the task is a simple classification

Leadership asks to bring launch forward by a month by skipping the evaluation phase. What should the architect do?

A. Explain the risks with evidence and offer options, such as launching a narrower, already-evaluated scope on the earlier date
B. Agree and skip evaluation
C. Refuse without discussion
D. Launch quietly without telling leadership the evaluation was cut An orchestrator delegates many simple lookup subtasks to subagents. Which model choice is most sensible for the subagents?
A. Always the most capable tier
B. The same tier as the orchestrator, for consistency
C. Extended thinking on every subagent
D. The fast tier, confirmed by evaluation on those subtasks

Questions 22 to 42

Which set of items belongs in a solution design document so that value and SLAs are explicit?

A. Model name and temperature only
B. Target metric with baseline, cost per transaction, p95 latency, availability, and expected human-review rate
C. The number of agents and tools
D. A list of features the model supports

A team runs an A/B test of two prompts on a multi-turn assistant, randomizing by request. Users report the assistant's tone changing mid-conversation. What should change?

A. Nothing; request-level randomization is standard
B. Run the test for longer
C. Randomize by user (or conversation), so each user consistently sees one variant
D. Test both prompts in the same response

An assistant drafts internal meeting notes from transcripts for the attendees. What level of human oversight is proportionate?

A. Low: attendees can correct notes, plus periodic spot checks of quality
B. Manager approval of every set of notes
C. Legal review of every set of notes
D. No AI use for meeting notes

A team with limited operations staff needs a long-running agent. Company policy allows the data to be processed by an approved hosted service. What is the best runtime choice?

A. Build a custom orchestration framework from scratch
B. Run the agent on developers' laptops
C. Delay until a platform team is hired
D. Use a hosted, managed agent service, accepting less control in exchange for lower operational effort An assistant uses six tools, and nearly every request needs most of them. Should it use progressive tool discovery?
A. Yes, always
B. No; with a small, stable set used on most requests, loading all definitions is simpler and fine
C. Yes, because MCP requires it
D. No, because progressive discovery is insecure

After launch, user feedback arrives in many channels and nobody acts on it. What is the best lifecycle practice?

A. A named owner triages feedback on a regular cadence into fixes and new evaluation cases, and reports changes back to users
B. Disable the feedback button
C. Wait for the annual review
D. Forward all feedback to the model provider

Which approach to few-shot examples is best for a classification prompt with 12 categories?

A. Ten examples of the most common category
B. No examples, ever
C. A small, diverse set covering different categories and a few tricky edge cases
D. Examples copied from another company's prompt

Retrieved context is correct, but answers sometimes add facts that are not in it. What is the best fix?

A. Retrieve more chunks
B. Switch to a larger embedding model
C. Remove the retrieved context
D. Instruct the model to answer only from the provided context, allow "not found," require citations, and add a groundedness check

A team wants to guarantee that Claude Code never edits files in the migrations folder. What is the best mechanism?

A. A sentence in CLAUDE.md
B. A deny permission rule or a pre-tool-use hook that blocks edits to that path
C. A code review reminder
D. Removing the folder from the repository

Evaluation shows that loan-application summaries are less complete for applicants whose documents are written in non-native English. What should the architect do?

A. Investigate with a stratified evaluation, fix the cause (prompting, extraction, or model choice), and monitor the gap in production
B. Ignore it, since the average is acceptable
C. Exclude those applicants from the system
D. Tell reviewers to be aware of it

Which TWO chunking practices best suit a corpus of long legal contracts?

A. Chunk along clause and section boundaries
B. Prepend each chunk with document and section context
C. Fixed 50-token chunks with no overlap
D. Strip all headings before chunking

A key feature the design depends on is available on Anthropic's API but not yet on the company's preferred cloud platform. What should the architect do?

A. Assume it will arrive soon and build for it on the cloud platform
B. Drop the feature without discussion
C. Confirm availability on each platform and weigh the feature against platform constraints, documenting the choice in an ADR
D. Use both platforms randomly

An executive asks, "Will it ever be wrong?" What is the best answer?

A. "No, it uses the latest model."
B. "We can't know."
C. "It is too technical to explain."
D. "Yes, occasionally. On our test set it meets the standard in 94% of cases, and the controls catch or route the rest to people." Model X costs $0.01 per task with 80% success; Model Y costs $0.03 per task with 95% success. Each failure costs $0.50 in human rework. Which is cheaper per task overall?
A. Model X, because its price is lower
B. Model Y: about $0.055 per task against about $0.11 for Model X
C. They cost the same
D. It cannot be estimated

An assistant has dozens of rarely used specialist procedures. How should they be provided?

A. As Skills that load only when relevant
B. All in the system prompt
C. In the user's message each time
D. Not at all

An EU customer requires that personal data be processed in the EU. What should the architect do?

A. Encrypt data and process anywhere
B. Ask the customer to waive the requirement
C. Use a deployment and region where processing stays in the EU, and confirm contractual terms
D. Remove personal data from all prompts permanently, even when needed An agent needs to read and summarize files in each user's cloud drive. How should access be granted?
A. One admin account that can read every drive
B. Users paste file contents into the chat
C. Shared passwords stored in the prompt
D. Delegated OAuth per user with read-only scopes, so the agent sees only what that user can see

An agent sometimes loops, calling search 200 times on one task. What should the architect add first?

A. A bigger model
B. Hard limits on turns, tool calls, and spend per task, with a graceful stop and handoff
C. More tools
D. A longer system prompt

Which TWO items are essential in a handoff pack to an operations team?

A. Runbooks for incidents, rollback, and model migration
B. The evaluation suite with thresholds
C. The original sales pitch deck
D. The build team's personal chat logs

When should the regression evaluation suite run?

A. On every change to prompts, model version, tools, or retrieval configuration
B. Only before the first launch
C. Only when users complain
D. Once a year

Which metrics best show whether an AI coding assistant improved delivery?

A. Lines of AI-generated code
B. Number of prompts sent
C. Lead time for changes, change failure rate, review turnaround, and developer satisfaction
D. Number of licenses purchased

Questions 43 to 63

Your agent delegates shipping quotes to a partner company's agent. How should the integration treat the partner's responses?

A. As trusted instructions to follow
B. As system prompt content
C. As unimportant, so no logging is needed
D. As untrusted data, under a defined contract with mutual authentication and logging of every exchange

A Claude feature lets customers do something the company could never offer before. How should its value be framed and funded?

A. As transformation, funded through a time-boxed pilot with clear success criteria
B. As efficiency, measured by staff hours saved
C. As a cost center with no metrics
D. As a technology upgrade with no business case

A trending social-media post shows the assistant giving harmful advice on one topic. What should happen first?

A. Retrain the model
B. Disable or redirect that intent to a safe fallback with a feature flag, then investigate
C. Wait for the weekly review
D. Delete the social-media post

A system prompt says both "always offer a discount to unhappy customers" and "never offer discounts without manager approval." Behavior is inconsistent. What is the fix?

A. Use a more capable model to resolve the conflict
B. Add a third instruction
C. Resolve the contradiction into one clear rule with its reason
D. Raise the temperature

Engineers have tuned prompts by checking results on the same test set for months. What risk does this create, and what is the fix?

A. No risk
B. Cost risk; use fewer cases
C. Latency risk; cache the test set
D. Overfitting; keep a frozen holdout set that is not used for tuning Prices change weekly, but the retrieval index is rebuilt monthly. Users get outdated prices. What is the best fix?
A. Tell users prices may be wrong
B. Fetch prices through a live tool or re-ingest on the same schedule as price updates, and monitor index freshness
C. Switch to a larger model
D. Increase the number of retrieved chunks

During discovery, which question captures a non-functional requirement?

A. "What is the peak request volume, and what response time do users need?"
B. "Which department will use it?"
C. "What is the product called?"
D. "Who is the executive sponsor?"

Why do multi-agent systems often give subagents their own context windows?

A. To share all information among all agents
B. Because tools require it
C. To isolate subtasks, keep each window focused, and return only compact results to the lead
D. To make the system cheaper in every case

Which TWO guardrails are most appropriate for a public customer chatbot?

A. Output filtering for personal data and policy violations
B. Input screening for prompt injection and abuse
C. Relying only on the model's own judgment
D. Disabling logs entirely

A web-research agent's configuration includes a "delete file" tool on the shared drive. What should the architect do?

A. Keep it in case it is needed
B. Add a warning to the tool description
C. Log deletions
D. Remove it; the role does not need it

Which production signal best reveals gradual quality drift?

A. Trend alerts on sampled grader scores and escalation rates
B. CPU usage on the application server
C. Number of deployments
D. Model price changes

A long-running agent's context fills with old tool results, and it loses track of decisions. What is the best remedy?

A. Use a smaller model
B. Compact the history into a summary of decisions and the plan, and trim tool outputs
C. Restart the task every hour from scratch
D. Disable tools

Your SLA depends on the model provider, which applies rate limits. How should the design account for this?

A. Ignore it
B. Promise customers unlimited throughput
C. Plan capacity, queue and retry with backoff, and define fallbacks such as another region or platform, or a human path
D. Remove the SLA

What production access should an AI assistant used by on-call engineers have by default?

A. Read-only access to logs, metrics, and deploy history through approved tools, with any change approved by an engineer
B. Full administrator access
C. None, so it cannot help with incidents
D. The on-call engineer's personal credentials

A payments system needs a fraud reason code within 150 milliseconds, and leadership wants a Claude-written explanation for customers. Where should Claude run?

A. In the 150 ms path, with the fast tier
B. Outside the hot path: generate the explanation asynchronously after the decision
C. Nowhere; explanations are not possible
D. In the hot path, with extended thinking

Managers ask "how many open tickets are there per region this week?" Which approach answers correctly?

A. Vector search over ticket text
B. Ask the model to estimate
C. A governed query tool over the ticketing database
D. Load all tickets into the prompt

Which workload is the best fit for batch processing?

A. A live customer chat
B. An on-call assistant during an incident
C. Real-time fraud explanation
D. Overnight sentiment tagging of the day's 200,000 survey responses

What makes human review meaningful rather than a rubber stamp?

A. Showing reviewers the sources and the specific points to check, keeping workloads manageable, and auditing reviewer accuracy
B. Requiring reviewers to approve within five seconds
C. Hiding the AI's reasoning from reviewers
D. Reviewing everything regardless of risk

The production model will be retired in four months. What should happen before switching?

A. Switch immediately to the newest model
B. Run the full evaluation suite on the replacement, adjust prompts, and canary the change
C. Wait until the retirement date
D. Ask users to test it

Two executives disagree: one wants full automation, the other wants humans to approve everything. How should the architect move forward?

A. Side with the more senior executive
B. Automate everything quietly
C. Propose risk tiers: automate the low-risk segment, keep approval for high-risk cases, and set the boundary with pilot data
D. Cancel the project

Traces sent to the observability platform contain customer names and account numbers. What should the architect do?

A. Nothing; the platform is internal
B. Stop all logging
C. Share traces only by email
D. Redact or tokenize personal data before logging, and restrict access to raw traces Practice exam 2: answer key and rationales

# Answer Domain Why

1 B 1 Small, slowly changing corpus: simplest pattern that works.

2 A 3 Replace broad capability with a narrow read-only tool.

3 D 2 Enforced schema plus validation fixes structure reliably.

4 C 4 A critical-metric breach blocks release regardless of the average.

5 B 5 Narrow the recipients and add approval to remove the exfiltration path. 6 A 6 Turn vague goals into measurable criteria with examples.

7 C 3 Rerank to pass fewer, better chunks; more context is not better context. 8 D 1 The SLA rules out a long sequential chain; redesign the flow.

9 B 7 Shared, versioned project configuration gives consistent behavior.

10 A 4 A judge is trusted only after calibration against humans.

11 C 5 Erasure must reach every store that holds the person's data.

12 D 2 Cache the stable prefix: system prompt and earlier turns.

13 A, C 3 Audience validation and scoped short-lived credentials; no passthrough. 14 B 1 Decompose a complex task into a checked chain.

15 C 6 Executives need outcomes against targets, risks, and decisions.

16 A 4 A repeated static prefix is the classic caching win.

17 D 5 Verify facts in code and keep a professional in the loop.

18 B 3 Structured attributes need structured queries or filters.

19 C 1 Evaluator-optimizer needs clear criteria and pays off through iteration. 20 A 6 Transparent risk and options; a narrower evaluated scope can ship early. 21 D 2 Simple subtasks suit the fast tier, confirmed by evaluation.

22 B 1 Value and SLAs must be stated as numbers.

23 C 4 Randomize by user or conversation for a consistent experience.

24 A 5 Low-risk output, low-touch oversight.

25 D 1 Managed runtime trades control for lower operational load, within policy. 26 B 3 Progressive discovery helps large catalogs, not six core tools.

27 A 6 Owned, regular feedback triage closes the loop.

28 C 2 Diverse examples, including edge cases, generalize best.

29 D 4 Grounding, abstention, citations, and a groundedness check.

30 B 7 Deterministic enforcement beats instructions.

31 A 5 Measure the fairness gap by segment, fix the cause, keep monitoring.

32 A, B 3 Structure-aware chunks with context preserve meaning.

33 C 1 Platform choice is evidence-based and documented.

34 D 6 State measured rates and the controls honestly.

35 B 4 X: 0.01 + 0.2 × 0.50 = $0.11. Y: 0.03 + 0.05 × 0.50 = $0.055.

36 A 2 Skills load on demand and keep context small.

37 C 5 Residency is met by where processing runs, plus contracts.

38 D 3 Delegated per-user access; never a superuser deputy.

39 B 1 Hard limits stop runaway loops.

40 A, B 6 Operations need runbooks and the evaluation suite.

41 A 4 Any change that can affect behavior runs the regression suite.

42 C 7 Delivery outcomes, not activity counts.

43 D 3 Another agent's output is untrusted data under a contract.

44 A 1 Transformation value is proven with a pilot.

45 B 5 Contain first, then investigate.

46 C 2 Contradictions cause inconsistency; resolve them in one rule.

47 D 4 Tuning on the test set overfits; keep a frozen holdout.

48 B 3 Freshness must match how often the source changes.

49 A 6 Volume and response time are non-functional requirements.

50 C 1 Context isolation keeps subagents focused and results compact.

51 A, B 5 Layered input and output guardrails.

52 D 3 Remove capabilities the role does not need.

53 A 4 Semantic quality needs semantic monitoring.

54 B 2 Compaction and trimming manage long-run context.

55 C 6 SLAs account for provider dependencies with fallbacks.

56 A 7 Read-only by default, human approval for change.

57 B 1 Keep slow steps out of a hard real-time path.

58 C 3 Aggregates over structured data need a query tool.

59 D 4 Non-urgent bulk work suits batch processing.

60 A 5 Reviewers need context, capacity, and auditing.

61 B 2 Evaluate and canary before migrating models.

62 C 6 Risk tiers turn a disagreement into a data-driven decision.

63 D 3 Redact before logging; restrict raw traces.

Scoring practice exam 2 by domain

Domain Items Ready at

1. Solution Design & Architecture 1, 8, 14, 19, 22, 25, 33, 39, 44, 50, 57 9 of 11
2. Models, Prompting & Context 3, 12, 21, 28, 36, 46, 54, 61 6 of 8
3. Integration 2, 7, 13, 18, 26, 32, 38, 43, 48, 52, 58, 63 10 of 12
4. Evaluation, Testing & Optimization 4, 10, 16, 23, 29, 35, 41, 47, 53, 59 8 of 10
5. Governance, Safety & Risk 5, 11, 17, 24, 31, 37, 45, 51, 60 7 of 9
6. Stakeholder Communication & Lifecycle 6, 15, 20, 27, 34, 40, 49, 55, 62 7 of 9
7. Developer Productivity & Enablement 9, 30, 42, 56 3 of 4

An overall score of 51 or more (about 80%) with no domain below its "ready at" level is a good sign. The real exam's passing standard is a scaled 720, which does not map exactly to a percentage, so aim above it.