8 Troubleshooting and optimization · CCAO-F

Exam domain 7: Troubleshooting and Optimization (10%)

When output is poor, diagnose the cause before you rewrite anything: most problems trace back to missing context, an unclear task, the wrong tool, or an overloaded conversation. This domain tests fixing underperforming prompts, adjusting to feedback, and making workflows more efficient.

Symptom, cause, fix

Symptom Likely cause Fix

Generic, bland output Missing context, audience or examples Add background, the reader, and a sample of what good looks like

Too many instructions at once; conflicts Simplify; split into steps; remove contradictions between prompt, Project instructions and files

Forgets earlier details

in a long chat

Context window filling up; topic drift Summarize and start a fresh chat; move standing facts into the Project

Contradicts current

policy

Outdated knowledge file in the Project Replace the file; remove old versions Too slow or uses up

Model too light; no room to reason Move up a tier; ask it to think step by step; enable

extended thinking

Same correction

needed every time

Fix lives in chat, not in configuration Put it in Project instructions, memory or a Skill Inconsistent results

across a team

Everyone prompts differently Shared Project or Skill with standard instructions and examples

A troubleshooting routine

Figure 7. A repeatable troubleshooting routine

Reread your own prompt as if you were a new colleague. Is the task unambiguous? Check the inputs. Did Claude get the right, current files? Did it read them correctly? Check the setup. Right feature, right model, Project instructions not conflicting? Change one thing at a time so you know what fixed it.

Ask Claude to diagnose. "Here is my prompt and your output. The output missed X. What in my prompt caused that, and how should I rewrite it?"

Make the fix permanent in the Project, memory or a Skill.

Optimizing a workflow

Measure. Time per task, number of revision rounds and error rate, before and after. Reduce rework. Fold recurring edits into instructions and templates.

Right-size models. Lighter models for volume, heavier for judgment.

Batch similar work. One prompt for 20 product descriptions in a table beats 20 separate chats.

Retire what does not work. If review time cancels out drafting time, redesign or drop the use case.

Output not right?

Reread your prompt

as a new colleague would

Check the inputs

right, current files?

Check the setup
feature, model,
instructions

Change one thing

so you know what fixed it

Ask Claude to
diagnose
prompt + output + the miss

Make the fix
permanent
Project, memory or Skill

Asking Claude to help debug your prompt

Claude is good at diagnosing why a prompt underperformed. A reliable template: Here is my prompt: [prompt]. Here is your output: [output]. The output was supposed to [goal] but instead [problem]. Explain what in my prompt likely caused this, then rewrite the prompt to fix it. Keep everything that worked.

Treat its diagnosis as a hypothesis, test the rewrite, and keep whichever version performs better.

Worked examples

Worked example 8.1: The ignored constraint

Situation. Fatima's prompt is 600 words long. Buried in the middle is "Use Canadian spelling." Outputs keep using American spelling.

Diagnosis. The instruction is easy to miss among many others, and nothing reinforces it. Fix. Move format and spelling rules into a short "Output rules" list at the end of the prompt, and put "Use Canadian spelling (colour, centre, organize)" in the Project instructions with examples. Retest: correct in five out of five outputs.

Worked example 8.2: The drifting summary

Situation. A weekly report Project produced great summaries for a month; now they are vaguer and miss key metrics.

Investigation.

Prompt unchanged? Yes.

Knowledge files? A new metrics file was added, but the old one was not removed. The two use different column names.

Instructions? Still reference the old column names.

Fix. Remove the old file, update instructions to the new column names, and add: "Always report these five metrics: ..." Retest against last week's data.

Lesson. Check the setup, not just the prompt, when quality changes over time.

Worked example 8.3: Right-sizing for speed

Situation. A support team's ticket-tagging workflow is slow and keeps hitting usage limits. It runs on the most capable model with extended thinking on.

Change one thing at a time.

Troubleshooting and optimization

Change Speed Accuracy on a 50-ticket sample
Baseline: top tier, extended thinking Slow 49/50
Turn off extended thinking Faster 49/50
Move to the balanced tier Faster still 48/50

Move to the fastest tier with three examples per tag Fastest 47/50 Decision. The team picks the fastest tier with examples and adds a weekly sample review. They keep the top tier for escalated tickets that need nuanced replies.

Worked example 8.4: Fixing hallucinated features

Situation. Sales collateral drafted by Claude sometimes describes product features that do not exist.

Fixes, in order.

Ground: add the product facts sheet to the Project and instruct, "Mention only features listed in the facts sheet."

Allow uncertainty: "If a requested feature is not listed, write 'Feature not confirmed: check with product.'"

Verify: add a reviewer step that checks every feature claim against the facts sheet. Result. Invented features stop appearing in sampled outputs. The review step stays, because grounding reduces but does not eliminate the risk.

Worked example 8.5: Batching for efficiency

Situation. An e-commerce coordinator writes 60 product descriptions a week, one chat per product.

Optimization. Upload a spreadsheet of products and attributes. Ask Claude to return a table with one description per row, following the house template, in batches of 20. Review each batch, then fold common edits back into the instructions. Weekly time falls from about five hours to under two, including review.

Diagnostic decision guide

Work down the list until you find the cause.

Is the task clear? If a colleague would ask questions, Claude needs that information too. Are the inputs right? Correct files, current versions, readable format.

Is the context clean? Long or off-topic chats degrade results; start fresh with a summary. Is the setup consistent? Instructions, knowledge and prompt do not conflict.

Is the tool right? Research for current facts, code execution for numbers, Artifacts for reusable output.

Is the model right? Move up for reasoning, down for volume.

Is the expectation realistic? Some tasks need a human, a specialist tool or a technical solution.

Scenario walkthroughs

Scenario A. Claude's analysis of a spreadsheet gives a different total from Excel. Analysis. Ask Claude to recompute with code and show its calculation. Check for hidden rows, filters or text-formatted numbers in the source.

Scenario B. Outputs are good for one team member and poor for another using the same Project. Analysis. Compare their prompts. The weaker prompts likely lack context. Share prompt templates or build the missing context into the instructions. Scenario C. Claude refuses or hedges on a legitimate business request.

Analysis. Add context that makes the legitimate purpose clear, such as your role and why you need it. If the request touches a genuinely restricted area, follow policy and escalate rather than trying to get around safeguards.

Scenario D. A scanned PDF yields garbled summaries.

Analysis. The scan may not be readable as text. Try a clearer copy or a text version, or ask Claude to describe what it can read before summarizing.

Scenario E. A workflow saves time but staff have stopped reviewing outputs carefully. Analysis. This is automation complacency. Reinstate structured review, use sampling, and track errors that reach customers.

Key points

Diagnose before rewriting; change one variable at a time.

Repeated fixes belong in configuration, not in chat.

Optimize for total time including review, not just drafting speed.

Common traps

Repeating an instruction louder instead of making it explicit and giving an example. Changing several things at once, so you never learn what fixed the problem.

Correcting the same issue in every chat instead of fixing the configuration.

Troubleshooting and optimization

Check yourself

Why change only one thing at a time when troubleshooting?

A Project's quality dropped after a month with no prompt changes. Where do you look? What is the best fix when Claude invents product features?

Name two ways to make a slow, simple workflow faster.

What is automation complacency, and how do you counter it?

Answers. (1) So you know which change fixed the problem. (2) Knowledge files and instructions, especially new or outdated files. (3) Ground outputs in a facts sheet, allow "not confirmed" answers, and verify claims in review. (4) Any two of: a lighter model, turning off extended thinking, sharing only relevant content, batching. (5) Reviewers relaxing because outputs are usually good; counter it with structured review, sampling and error tracking.

Exercise

Take an output you were unhappy with. Use the symptom table to name the cause, apply one fix, and compare.