6 Model Selection and Optimization (16.8%) · CCDV-F

What the exam tests: LLM fundamentals, the engineering basics behind SDK integration, choosing the right Claude model for quality, latency, and cost, and managing tokens and caching. This is the second-heaviest domain.

LLM fundamentals (5.2%)

Tokens. Models read and write tokens, sub-word pieces of text. Prices, context limits, and rate limits are all counted in tokens. Use the token counting endpoint (POST /v1/messages/ count_tokens) for exact counts; tokenizers differ across model generations, so re-measure after an upgrade.

Context window. The maximum tokens of input plus output a request can hold. Current Claude models offer up to 1M tokens. Bigger is not free: every token is billed and long contexts dilute attention. Next-token generation. The model predicts one token at a time. Output is generated, not retrieved, which is why it can be fluent and wrong. Sampling and non-determinism. Sampling parameters (temperature, top_p, top_k) historically controlled randomness. Even at low temperature, outputs are not guaranteed identical across calls. Note: on recent models these parameters are removed, and from API version 2026-10-21 sending them returns a 400. Design for variability with validation and evals instead of relying on determinism. Prompting techniques. Zero-shot (instructions only), single-shot (one example), and multishot or few-shot (several examples). Examples are one of the most reliable ways to steer format and style.

Model options. Extended thinking lets Claude reason before answering. Adaptive thinking lets Claude decide how much to think. Effort levels (low, medium, high, xhigh, max) trade depth for tokens and latency. Fast mode is a research-preview option on some Opus models that generates output up to about 2.5 times faster at premium pricing.

Technical fundamentals (6.1%)

The guide groups the engineering basics behind integration here.

SDKs wrap REST. Official SDKs exist for Python, TypeScript, Java, Go, Ruby, C#, and PHP. They add retries, typed errors, streaming helpers, and pagination over the same HTTP API. Timeouts. SDK default timeout is 10 minutes; units differ by language (seconds in Python, milliseconds in TypeScript). Use streaming for long generations.

Websockets and server-sent events. Claude's API streams over server-sent events (one-way, server to client, over HTTP). Websockets are two-way connections you might use between your own backend and browser to relay a stream. Know the difference.

Rate limits. Limits apply to requests per minute and input and output tokens per minute, per model and per organization tier. Watch the anthropic-ratelimit-* response headers and back off on 429.

Model selection and tradeoffs (2.7%)

The current families, as of October 2026 (first-party API prices per million tokens): Model ID

claudefable-5-1$10 $50 Most demanding reasoning and long-horizon agentic work Claude Opus

5.5

claudeopus-5-5$4 $20 Complex coding, agents, deep analysis; the default high-capability choice Claude Son‐

net 5.5

claudesonnet-5-5$2 $10 Balanced speed and capability for everyday coding, agents, enterprise work Claude

Haiku 5.5

claudehaiku-5-5$0.1

$0.50 High-volume, low-latency tasks: classification, routing, extraction, subagents (prices apply up to 100K-token prompts; $0.50 and $2.50 above)

Prices change often; check Anthropic pricing before relying on them. The exam is more likely to test the pattern than exact figures: Opus for hardest tasks, Sonnet for the balanced default at scale, Haiku for speed and volume.

How to choose:

Start with the most capable model that fits the budget, and build an eval.

Try lower effort on that model before switching to a smaller model; it often holds quality with fewer tokens.

Step down to a smaller model per route where evals show quality holds.

Judge cost per completed task, not per request. A cheaper model that needs three retries is not cheaper.

Breaking changes across releases. New models can change request rules, not just quality. Recent examples: assistant-message prefill removed, fixed thinking budgets replaced by adaptive

thinking and effort, forced tool_choice (any or a named tool) rejected on the newest models, and sampling parameters removed. This is why you pin versions and test migrations behind evals.

Cost and token management (2.8%)

Track usage. Log usage on every response and aggregate by feature, user, and model. Build a simple cost model: requests per day times average input and output tokens times price. Prompt caching is the biggest free win for repeated context. Mark a stable prefix with cache_control; later requests that share that exact prefix read it from cache.

Item Value

Default TTL 5 minutes, refreshed on each hit; 1-hour TTL optional Cache write price 1.25 times base input (5-minute) or 2 times (1-hour)

Cache read price About 0.1 times base input (lower on some newest models)

Breakpoints Up to 4 cache_control markers per request

Minimum cacheable prefix Model-dependent, roughly 512 to 4,096 tokens; shorter prefixes silently do not cache Render order tools, then system, then messages

Caching is a prefix match: any byte change invalidates everything after it. Put stable content first (tools, system prompt, reference documents) and volatile content (timestamps, the user's question) last. A timestamp at the top of the system prompt silently kills your cache. Verify with usage.cache_read_input_tokens.

Figure 6. Prompt caching is a prefix match: stable content first, volatile content last

Cache check-pointing. In multi-turn conversations, move a breakpoint to the latest turn so each request reuses the whole prior conversation. Add intermediate breakpoints in very long turns so each lookup stays within range of an earlier entry.

Other levers, roughly in order of how often they are the right answer:

Prompt caching for repeated prefixes. tools system prompt reference documents earlier turns new question cache_control breakpoint

Cached prefix: identical bytes every request (read at about 0.1x input price) Changes each call rendered in this order: tools, then system, then messages

Any byte change in the prefix (a timestamp at the top of the system prompt) invalidates everything after it.

MODEL SELECTION AND OPTIMIZATION (16.8%)

Message Batches API (50% off) for latency-tolerant work.

Trim inputs: prune old tool output, retrieve fewer and better chunks.

Right-size max_tokens and request concise output.

Lower effort where quality holds.

Route simple traffic to a smaller model.

Worked example 6.1: building a cost model

The exam will not ask you to memorize prices, but it will expect you to reason about where cost comes from. Practice with real numbers (Claude Sonnet 5.5 at $2 input and $10 output per million tokens, October 2026). Scenario. A support bot handles 100,000 questions a day. Each request carries a 30,000-token policy manual, a 500-token question, and returns about 400 tokens.

Without caching:

Input: 30,500 tokens times 100,000 = 3.05 billion tokens. At $2 per million, that is $6,100 a day.

Output: 400 times 100,000 = 40 million tokens. At $10 per million, $400 a day. Total: about $6,500 a day.

With prompt caching on the manual (cache reads on Sonnet 5.5 cost $0.10 per million, and continuous traffic keeps the 5-minute cache warm so writes are rare):

Cached manual: 30,000 times 100,000 = 3 billion tokens at $0.10 per million = $300. Uncached question: 500 times 100,000 = 50 million tokens at $2 per million = $100. Output unchanged: $400.

Total: about $800 a day, roughly an 88% saving with no quality change.

Notice that after caching, output is now the largest cost. The next lever is shorter answers ("answer in at most three sentences unless asked for detail"), not a smaller model.

Worked example 6.2: when does caching pay off?

With the 5-minute TTL, a cache write costs 1.25 times the base input price and each read about 0.1 times. Compare two requests sharing a prefix:

Uncached: 1 + 1 = 2.0 units.

Cached: 1.25 (write) + 0.1 (read) = 1.35 units.

So caching pays for itself on the second request within the TTL. With the 1-hour TTL the write is 2 times base, so you need at least three requests to break even (2 + 0.1 + 0.1 = 2.2 versus 3.0). Rule of thumb for TTL choice:

Over an hour Caching usually does not help; consider whether the prefix needs to be sent at all

Worked example 6.3: caching in code, and the silent invalidator response = client.messages.create(

model="claude-sonnet-5-5",
max_tokens=1024,
system=[

{"type": "text", "text": "You are Acme's policy assistant. Answer only from the manual."},

{"type": "text", "text": POLICY_MANUAL, # 30,000 stable tokens "cache_control": {"type": "ephemeral"}}, # breakpoint after the manual ],

messages=[{"role": "user", "content": question}], # varies per request )

u = response.usage

print(u.cache_creation_input_tokens, u.cache_read_input_tokens, u.input_tokens) First request: cache_creation_input_tokens is about 30,000 and reads are 0. Later requests within the TTL: reads are about 30,000 and creation is 0.

Now suppose a developer "improves" the system prompt:

{"type": "text", "text": f"Today is {datetime.now():%Y-%m-%d %H:%M}. You are Acme's policy assistant..."}

The timestamp changes every minute and sits before the breakpoint, so every request writes a new cache entry and none is ever read. Cost goes up, not down, because writes cost more than plain input. The fix: move the date into the user message, after the breakpoint, or drop the minutes.

Other silent invalidators: tool definitions in a different order per request, JSON serialized with unsorted keys, a per-user name in the system prompt, switching models (caches are per model), and changing effort or thinking settings mid-conversation.

Worked example 6.4: model selection cases

Scenario Recommendation Reasoning
Classify 2 million short tickets a day into
12 categories

Haiku High volume, simple task, latencysensitive; verify accuracy on an eval Agent that refactors a large legacy code‐

base over hours
Opus (or Fable for the hardest
long-horizon work), effort high
Complex multi-step reasoning; errors
are costly
MODEL SELECTION AND OPTIMIZATION (16.8%)
Scenario Recommendation Reasoning
General-purpose internal assistant for
5,000 employees
Sonnet, effort medium Balanced quality and cost at scale
Research subagents that read and sum‐
marize documents for an Opus lead
Haiku or Sonnet workers Bulk reading is cheaper on a smaller
model; the lead does synthesis
Real-time voice assistant with a strict
latency budget
Haiku or Sonnet at low effort,
streaming
Time to first token matters more than
depth
Monthly board-report analysis, accur‐
acy critical, volume tiny
Opus at high effort Cost is negligible at low volume; quality
dominates

To compare the price gap concretely, take the ticket classifier: 2 million tickets of 300 input and 5 output tokens each.

Haiku 5.5: 600 million input tokens at $0.10 per million plus 10 million output at 0.50 = about * *65 a day**.

Sonnet 5.5: 600 million at $2 plus 10 million at 10 = about * *1,300 a day**. If Haiku meets the accuracy bar, that is a 20 times saving. If it misses by 4 points and each misroute costs an agent 3 minutes, the cheaper model may cost more overall. Always cost the whole task, not just the API call.

Worked example 6.5: effort before model swaps

A team runs an Opus-tier coding agent at high effort and wants to cut cost by 40%. Two experiments on their 100-task eval: Configuration Tasks solved Relative cost

Opus, effort high (baseline) 82 1.00
Opus, effort medium 80 0.62
Sonnet, effort high 71 0.55

(Figures are illustrative.) Lowering effort on the same model kept nearly all the quality for most of the saving, and kept one cache namespace. This is why the recommended order is: free wins (caching, batching, trimming) first, then effort, then model changes.

Worked example 6.6: counting tokens before sending

count = client.messages.count_tokens(
model="claude-sonnet-5-5",
system=SYSTEM_PROMPT,
messages=[{"role": "user", "content": big_document}],
)
if count.input_tokens > 180_000:
chunks = split_document(big_document) # or summarize first

Use the counting endpoint rather than a generic tokenizer, because Claude's tokenizer differs from others and has changed across generations. Never silently truncate input; chunk it or tell the user.

Scenario drills

Caching is on, but cache_creation_input_tokens is high on every request and reads are zero. Name three possible causes.

A team doubles max_tokens from 1,000 to 2,000. Will the bill double?

A summarization job runs nightly on 500,000 documents with a shared 8,000-token instruction block. Name two levers that stack. A product manager wants "the cheapest model" for a medical-triage assistant. How do you respond?

After a model upgrade, the same prompts use about 20% more tokens. What is a likely explanation? Why can a multi-model cascade (try Haiku, escalate to Opus) cost more than expected? Answers:

A changing value before the breakpoint (timestamp, user name, request ID), tools in varying order or unsorted JSON, and requests spaced beyond the TTL (also: a prefix below the minimum cacheable length gives zero writes, not high writes). No. You pay for tokens generated; the cap only matters if answers were being truncated. The Batch API (50% off) plus prompt caching on the shared instruction block. Model choice follows the quality bar first; for safety-critical work, pick the model that passes a rigorous eval, then optimize cost within that.

A new tokenizer; re-baseline with the token counting endpoint and update cost models. Escalated requests pay twice, caches do not carry across models, and misjudged escalations add retries.

Common traps

"Use the smallest model regardless of quality." Cost alone is never the whole requirement. Lowering max_tokens to save money. It only truncates; you pay for tokens actually generated. Expecting identical output from identical prompts.

Putting a per-request value early in the cached prefix.

MODEL SELECTION AND OPTIMIZATION (16.8%)

Building a multi-model cascade before testing one strong model at lower effort. Caches are per model, so cascades also lose cache reuse.

Check yourself

A support app sends the same 30,000-token policy manual with every question. What is the cheapest fix?

Caching is enabled but cache_read_input_tokens is always 0. Name two likely causes. Which model family suits a router that classifies 2 million tickets a day?

Why does lowering max_tokens not reduce cost on answers that were already short? What is the difference between server-sent events and websockets?

Answers: Prompt caching with a breakpoint after the manual. A changing value early in the prefix (such as a timestamp), or a prefix below the minimum cacheable length; also requests more than 5 minutes apart. Haiku. You pay for tokens generated, not for the cap. SSE is one-way server-to-client over HTTP; websockets are two-way.