Multi-Instance and Multi-Pass Review

Official Exam Guide Objectives

Task 4.6: Design multi-instance and multi-pass review architectures

Knowledge of

• Self-review limitations: a model retains reasoning context from generation, making it less likely to question its own decisions in the same session
• Independent review instances (without prior reasoning context) are more effective at catching subtle issues than self-review instructions or extended thinking
• Multi-pass review: splitting large reviews into per-file local analysis passes plus cross-file integration passes to avoid attention dilution and contradictory findings

Skills in

• Using a second independent Claude instance to review generated code without the generator's reasoning context
• Splitting large multi-file reviews into focused per-file passes for local issues plus separate integration passes for cross-file data flow analysis
• Running verification passes where the model self-reports confidence alongside each finding to enable calibrated review routing

What You Need to Know

A model asked to review what it has just written is not approaching the work neutrally — it still holds the reasoning that produced it, and reasoning it has already justified is reasoning it is inclined to defend. That is not a defect to prompt around. It is a property to design around.

The Self-Review Limitation

Within one session, the chain that produced the output is still present: why each approach was taken, why a finding was rated where it was, why particular values were selected. Asked to review, the model reads its own justifications alongside the output, and confirmation is the path of least resistance.

An independent instance — a separate invocation carrying none of that history — has only the artefact to go on. It evaluates the code, the findings, the extraction as they stand, with no prior commitment to explain away. That absence of context is precisely what makes it better at finding what the first pass missed.

The exam tests this directly. Offered a set of ways to improve review quality, the option that scores is the one that moves the review to a separate instance. Instructing the same session to take more care, or giving it more room to think, both leave the justifications in place — and those are the problem.

// Anti-pattern: self-review in the same session
const generation = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [
    { role: "user", content: "Write a function to process orders" },
    { role: "assistant", content: generatedCode },
    { role: "user", content: "Now review your code for bugs" }
    // Model retains its reasoning — less likely to find its own mistakes
  ]
});

// Correct: independent review instance
const review = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [
    {
      role: "user",
      content: `Review this code for bugs, security issues, and edge cases:\n\n${generatedCode}`
    }
    // Fresh instance — no prior reasoning context
  ]
});

# Anti-pattern: self-review in the same session
generation = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=[
        {"role": "user", "content": "Write a function to process orders"},
        {"role": "assistant", "content": generated_code},
        {"role": "user", "content": "Now review your code for bugs"},
        # Model retains its reasoning — less likely to find its own mistakes
    ],
)

# Correct: independent review instance
review = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": (
                "Review this code for bugs, security issues, and edge cases:"
                f"\n\n{generated_code}"
            ),
        }
        # Fresh instance — no prior reasoning context
    ],
)

Multi-Pass Review Architecture

Put a multi-file PR, a broad audit or a long extraction through one pass and attention dilution sets in. It announces itself the same way every time:

• Close, specific commentary on some files and summaries on others
• Plain defects passed over somewhere in the middle of the set
• The same construct raised as a problem in one file and accepted without comment in another

Splitting the work into passes that each answer one kind of question resolves it:

Pass 1: Per-file local analysis. One file per invocation, reviewed on its own terms. Because nothing else is competing for the budget, the depth of the tenth file matches the depth of the first.

Pass 2: Cross-file integration. With the per-file work complete, a further pass receives those findings and looks for what only exists between files: data flowing between modules, API usage that diverges across services, dependency conflicts, and disagreements among the findings themselves.

// Pass 1: Per-file analysis
const perFileFindings = await Promise.all(
  files.map(file =>
    client.messages.create({
      model: "claude-sonnet-5",
      max_tokens: 4096,
      messages: [{
        role: "user",
        content: `Review this file for local issues (bugs, security, logic errors):\n\n${file.content}`
      }]
    })
  )
);

// Pass 2: Cross-file integration
const integrationReview = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [{
    role: "user",
    content: `Given these per-file findings, identify cross-file issues:\n` +
      `- Data flow inconsistencies between modules\n` +
      `- Contradictory patterns flagged in different files\n` +
      `- API contract violations across service boundaries\n\n` +
      `Findings:\n${JSON.stringify(perFileFindings)}`
  }]
});

import asyncio
import json


# Pass 1: Per-file analysis. Each call gets the whole attention budget for
# one file, which is the point — a single pass spreads it across all of them.
async def review_file(file: dict):
    return await async_client.messages.create(
        model="claude-sonnet-5",
        max_tokens=4096,
        messages=[
            {
                "role": "user",
                "content": (
                    "Review this file for local issues (bugs, security, logic "
                    f"errors):\n\n{file['content']}"
                ),
            }
        ],
    )


per_file_findings = await asyncio.gather(*(review_file(f) for f in files))

# Pass 2: Cross-file integration
integration_review = await async_client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": (
                "Given these per-file findings, identify cross-file issues:\n"
                "- Data flow inconsistencies between modules\n"
                "- Contradictory patterns flagged in different files\n"
                "- API contract violations across service boundaries\n\n"
                f"Findings:\n{json.dumps(per_file_findings)}"
            ),
        }
    ],
)

Each symptom maps onto a part of this. Uneven depth disappears because no file shares its pass. Cross-file problems get found because one pass exists to look for nothing else. Contradictions surface rather than shipping, because comparing the findings is now an explicit step.

Why Larger Context Windows Do Not Fix This

One distractor recurs: move to a higher-tier model with a larger context window. It follows naturally — if 14 files will not fit comfortably, supply more room. But capacity was never what failed. Attention distributes unevenly across whatever is present, and a larger window changes how much can be present rather than how evenly it is treated. The fall-off simply begins later. Only giving a file a pass of its own guarantees it a full share.

Confidence-Based Routing

Where a finding is uncertain, the model can attach its own confidence to it, which makes routing possible:

• High confidence findings: go straight to the developer
• Low confidence findings: divert to a person before anyone acts on them
• Threshold calibration: decide where that line sits by measuring reported confidence against known-correct answers

{
  "finding": "Potential race condition in order processing",
  "severity": "major",
  "confidence": 0.65,
  "reasoning": "The lock acquisition pattern appears correct but the unlock timing depends on an async callback whose ordering I cannot fully verify.",
  "route": "human_review"
}

What that number reports is the model's sense of its own certainty, which is not the same thing as its accuracy. The two only become related through measurement: run examples whose answers you already know, observe how the reported confidence tracks correctness, and set the routing threshold from what you observe.

That measurement is the whole distinction the exam draws — between a raw score, which is uncalibrated and unfit to automate against, and a calibrated threshold validated on labelled data. Automating decisions on the former is an anti-pattern.

Putting It All Together

A production review architecture combines all three concepts:

1. Generation: the first instance produces the code, extraction or analysis
2. Per-file review: each unit is examined by an instance holding nothing but that unit
3. Integration review: a further instance looks only for what sits between units
4. Confidence routing: whatever falls below the threshold reaches a person
5. Calibration loop: labelled data keeps that threshold honest as the system changes

It costs more than reviewing once, and the question is what a missed issue costs by comparison. Where the review gates a CI/CD pipeline, financial extraction or compliance analysis, what escapes it goes downstream — and the extra passes are cheap against that.

Deep Dive

Official guidance: fresh context is the mechanism, not a heuristic

Anthropic's own best-practices documentation states the principle this lesson teaches as settled guidance, not a tip: "A fresh context improves code review since Claude won't be biased toward code it just wrote." This is offered as one of several "quality-focused workflows" enabled by running multiple sessions — the exam's "independent instance" requirement is a direct restatement of this documented pattern, not an inference from first principles.

Sourcecode.claude.com › best-practicesfetched 2026-07-30

The Writer/Reviewer session pattern

The concrete architecture the docs describe: "use a Writer/Reviewer pattern... You can do something similar with tests: have one Claude write tests, then another write code to pass them." One session (or instance) generates the artefact; a second, separately-initialised session reviews it; the first then addresses the feedback. This is a two-instance loop, not a three-stage pipeline — the reviewer's only job is to evaluate, never to have also produced the thing it is evaluating.

Sourcecode.claude.com › best-practicesfetched 2026-07-30

Adversarial review sees only the diff and the criteria

Before treating work as done, the docs recommend a review step that deliberately withholds the generating reasoning: "A reviewer running in a fresh subagent context sees only the diff and the criteria you give it, not the reasoning that produced the change, so it evaluates the result on its own terms." This is the precise mechanism behind why self-review under-performs — it isn't that the same model is "worse," it's that it has extra information (its own justification) that a genuinely fresh reviewer never receives, and that extra information biases it toward confirmation.

Sourcecode.claude.com › best-practicesfetched 2026-07-30

Verification by a second opinion that tries to refute the result

The docs describe a specific pattern for high-stakes verification: "a verification subagent or a dynamic workflow that checks its own findings has a fresh model try to refute the result, so the agent doing the work isn't the one grading it." The framing — trying to refute, not merely re-confirm — is stronger than a neutral second look; it's an adversarial stance that surfaces the subtle issues a confirmatory pass would skate past.

Sourcecode.claude.com › best-practicesfetched 2026-07-30

Subagent context isolation is the architectural primitive underneath

The independent-instance pattern rests on the same isolation property that makes subagents useful generally: "subagents use their own isolated context windows, and only send relevant information back" to the coordinator. An independent review instance is this same primitive applied to review specifically — it never inherits the generating session's conversation history, so it cannot be biased by reasoning it never saw.

Sourceclaude.com › building-agents-with-the-claude-agent-sdkfetched 2026-07-30

LLM-as-judge is explicitly the weakest verification method

The Agent SDK guidance ranks verification methods and places model-judges-model near the bottom: "Verification via LLM-as-judge... is generally not a very robust method, and can have heavy latency tradeoffs." This matters directly for confidence-based routing in this task statement — a raw, self-reported confidence score is a form of the model judging itself, which is exactly the weak pattern this guidance warns against. It is one more reason routing thresholds must be calibrated against labelled data rather than trusted as reported.

Sourceclaude.com › building-agents-with-the-claude-agent-sdkfetched 2026-07-30

Calibration method: evaluate the end state with a rubric, starting small

Anthropic's multi-agent research system describes the calibration workflow this task statement's confidence-routing skill depends on. First, judge outcomes, not process: "Instead of judging whether the agent followed a specific process, evaluate whether it achieved the correct final state." Second, use a structured rubric rather than free-form grading: "an LLM judge that evaluated each output against criteria in a rubric: factual accuracy, citation accuracy, completeness, source quality, and tool efficiency." Third, start small — "about 20 queries representing real usage patterns" is enough to see the impact of a change — and keep a human in the loop regardless of automation, because "people testing agents find edge cases that evals miss."

Sourceanthropic.com › multi-agent-research-systemfetched 2026-07-30

Quick Reference

Exam Traps

Practice Scenario

Build Exercise

Build a Multi-Pass Code Review System

Difficulty: Advanced (3/4)

60 minutes

1. Create a single-pass review prompt and run it against a 10-file mock PR — document instances of inconsistent depth, missed issues, and contradictory findings

Why: Establishing the single-pass baseline demonstrates the three symptoms of attention dilution: inconsistent depth across files, missed bugs in the middle of the review, and contradictory findings flagging the same pattern differently in different files.

You should see: Detailed feedback on some files (typically first and last) but superficial comments on others, at least one obvious bug missed in a middle file, and at least one contradictory finding where the same code pattern is flagged as problematic in one file but approved in another.

2. Implement per-file local analysis: iterate over each file with a focused review prompt that examines only that file for bugs, security issues, and logic errors

Why: Per-file analysis ensures every file receives consistent, focused attention. Each invocation examines only one file, eliminating the attention dilution that causes inconsistent depth and missed bugs in single-pass reviews.

You should see: Consistent review depth across all 10 files. Bugs that were missed in the single-pass review should now be caught, especially those in the middle files. Each review should be focused and thorough.

3. Implement a cross-file integration pass: feed all per-file findings into a separate prompt that checks for data flow inconsistencies, contradictory findings across files, and API contract violations

Why: Per-file analysis catches local issues but misses cross-file concerns: data flow between modules, consistent API usage, and contradictions in per-file findings. The integration pass is a separate invocation that receives all findings and checks for systemic issues.

You should see: A synthesis output identifying cross-file issues that no single-file review could catch: data passed between modules in incompatible formats, contradictory findings from per-file reviews, and API contracts violated across service boundaries.

4. Add confidence scoring to each finding (0.0-1.0) and implement routing: high confidence findings go directly to the developer, low confidence findings go to a human review queue

Why: Confidence-based routing directs limited human reviewer attention to the findings that need it most. The exam distinguishes raw uncalibrated confidence from calibrated thresholds validated against labelled sets.

You should see: Each finding annotated with a confidence score, reasoning for the score, and a routing decision (direct_report or human_review). The routing threshold should separate clear-cut findings from uncertain ones.

5. Use a separate Claude instance (fresh session, no prior context) to review a subset of the generated findings and compare its assessment to the original confidence scores for calibration

Why: Independent review instances approach output fresh without the bias of I chose this because reasoning. This step calibrates confidence thresholds by comparing self-reported confidence against independent assessment, the method the exam identifies as the correct approach.

You should see: A calibration dataset showing the relationship between reported confidence scores and independent verification results. Some high-confidence findings may be overturned, revealing calibration gaps that adjust your routing thresholds.

Sources

• Claude Certified Architect Foundations Exam Guide — Task Statement 4.6 — Anthropic
• Prompt Engineering Overview — Anthropic
• Building with Claude API (Skilljar) — Anthropic
• Claude Code Best Practices — Anthropic
• Building Agents with the Claude Agent SDK — Anthropic
• How We Built Our Multi-Agent Research System — Anthropic

Appendix A — Build Exercise Step Hints

Progressive hints revealed by the "Stuck? Get a nudge" control on each step.

Step 1. Create a single-pass review prompt and run it against a 10-file mock PR — document instances of inconsistent depth, missed issues, and contradictory findings

Why: Establishing the single-pass baseline demonstrates the three symptoms of attention dilution: inconsistent depth across files, missed bugs in the middle of the review, and contradictory findings flagging the same pattern differently in different files.

You should see: Detailed feedback on some files (typically first and last) but superficial comments on others, at least one obvious bug missed in a middle file, and at least one contradictory finding where the same code pattern is flagged as problematic in one file but approved in another.

const files = generateMockPR(10); // 10 files with known bugs
const singlePassResult = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [{
    role: "user",
    content: `Review this PR for bugs, security issues, and logic errors:\n\n${files.map(f => `--- ${f.name} ---\n${f.content}`).join("\n\n")}`
  }]
});

// Document attention dilution symptoms
const perFileDepth = files.map(f => assessReviewDepth(singlePassResult, f.name));

files = generate_mock_pr(10)  # 10 files with known bugs
joined = "\n\n".join(f"--- {f['name']} ---\n{f['content']}" for f in files)
single_pass_result = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": (
                f"Review this PR for bugs, security issues, and logic errors:\n\n{joined}"
            ),
        }
    ],
)

# Document attention dilution symptoms
per_file_depth = [assess_review_depth(single_pass_result, f["name"]) for f in files]

Step 2. Implement per-file local analysis: iterate over each file with a focused review prompt that examines only that file for bugs, security issues, and logic errors

Why: Per-file analysis ensures every file receives consistent, focused attention. Each invocation examines only one file, eliminating the attention dilution that causes inconsistent depth and missed bugs in single-pass reviews.

You should see: Consistent review depth across all 10 files. Bugs that were missed in the single-pass review should now be caught, especially those in the middle files. Each review should be focused and thorough.

const perFileFindings = await Promise.all(
  files.map(file =>
    client.messages.create({
      model: "claude-sonnet-5",
      max_tokens: 4096,
      messages: [{
        role: "user",
        content: `Review this file for bugs, security issues, and logic errors:\n\n${file.content}`
      }]
    })
  )
);

// Compare depth and bug detection against single-pass results
const improved = files.map((f, i) => ({
  file: f.name,
  singlePassFindings: countFindings(singlePassResult, f.name),
  perFileFindings: countFindings(perFileFindings[i])
}));

async def review_file(file: dict):
    return await async_client.messages.create(
        model="claude-sonnet-5",
        max_tokens=4096,
        messages=[
            {
                "role": "user",
                "content": (
                    "Review this file for bugs, security issues, and logic "
                    f"errors:\n\n{file['content']}"
                ),
            }
        ],
    )


per_file_findings = await asyncio.gather(*(review_file(f) for f in files))

# Compare depth and bug detection against single-pass results
improved = [
    {
        "file": f["name"],
        "single_pass_findings": count_findings(single_pass_result, f["name"]),
        "per_file_findings": count_findings(per_file_findings[i]),
    }
    for i, f in enumerate(files)
]

Step 3. Implement a cross-file integration pass: feed all per-file findings into a separate prompt that checks for data flow inconsistencies, contradictory findings across files, and API contract violations

Why: Per-file analysis catches local issues but misses cross-file concerns: data flow between modules, consistent API usage, and contradictions in per-file findings. The integration pass is a separate invocation that receives all findings and checks for systemic issues.

You should see: A synthesis output identifying cross-file issues that no single-file review could catch: data passed between modules in incompatible formats, contradictory findings from per-file reviews, and API contracts violated across service boundaries.

const integrationReview = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [{
    role: "user",
    content: `Given these per-file review findings, identify cross-file issues:\n` +
      `- Data flow inconsistencies between modules\n` +
      `- Contradictory patterns flagged differently in different files\n` +
      `- API contract violations across service boundaries\n\n` +
      `Per-file findings:\n${JSON.stringify(perFileFindings, null, 2)}`
  }]
});

integration_review = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": (
                "Given these per-file review findings, identify cross-file issues:\n"
                "- Data flow inconsistencies between modules\n"
                "- Contradictory patterns flagged differently in different files\n"
                "- API contract violations across service boundaries\n\n"
                f"Per-file findings:\n{json.dumps(per_file_findings, indent=2)}"
            ),
        }
    ],
)

Step 4. Add confidence scoring to each finding (0.0-1.0) and implement routing: high confidence findings go directly to the developer, low confidence findings go to a human review queue

Why: Confidence-based routing directs limited human reviewer attention to the findings that need it most. The exam distinguishes raw uncalibrated confidence from calibrated thresholds validated against labelled sets.

You should see: Each finding annotated with a confidence score, reasoning for the score, and a routing decision (direct_report or human_review). The routing threshold should separate clear-cut findings from uncertain ones.

const reviewPrompt = `Review this file. For each finding, include:\n` +
  `- finding: description of the issue\n` +
  `- severity: critical/major/minor\n` +
  `- confidence: 0.0-1.0 score\n` +
  `- reasoning: why you are confident or uncertain\n\n`;

function routeFinding(finding) {
  return {
    ...finding,
    route: finding.confidence >= 0.80 ? "direct_report" : "human_review"
  };
}

review_prompt = (
    "Review this file. For each finding, include:\n"
    "- finding: description of the issue\n"
    "- severity: critical/major/minor\n"
    "- confidence: 0.0-1.0 score\n"
    "- reasoning: why you are confident or uncertain\n\n"
)


def route_finding(finding: dict) -> dict:
    # The threshold is a placeholder until calibration (step 5) sets it from
    # labelled data — a self-reported score is not a routing signal on its own.
    return {**finding, "route": "direct_report" if finding["confidence"] >= 0.80 else "human_review"}

Step 5. Use a separate Claude instance (fresh session, no prior context) to review a subset of the generated findings and compare its assessment to the original confidence scores for calibration

Why: Independent review instances approach output fresh without the bias of I chose this because reasoning. This step calibrates confidence thresholds by comparing self-reported confidence against independent assessment, the method the exam identifies as the correct approach.

You should see: A calibration dataset showing the relationship between reported confidence scores and independent verification results. Some high-confidence findings may be overturned, revealing calibration gaps that adjust your routing thresholds.

// Independent review — fresh instance, no prior context
const independentVerification = await Promise.all(
  sampleFindings.map(finding =>
    client.messages.create({
      model: "claude-sonnet-5",
      max_tokens: 4096,
      messages: [{
        role: "user",
        content: `Review this code and assess whether this finding is valid:\n\nCode:\n${finding.code}\n\nFinding: ${finding.description}\nSeverity: ${finding.severity}\n\nIs this finding valid? Explain your assessment.`
      }]
    })
  )
);

// Build calibration curve
const calibration = buildCalibrationCurve(sampleFindings, independentVerification);
// Adjust routing threshold based on calibration data

# Independent review — fresh instance, no prior context
async def verify(finding: dict):
    return await async_client.messages.create(
        model="claude-sonnet-5",
        max_tokens=4096,
        messages=[
            {
                "role": "user",
                "content": (
                    "Review this code and assess whether this finding is valid:\n\n"
                    f"Code:\n{finding['code']}\n\n"
                    f"Finding: {finding['description']}\n"
                    f"Severity: {finding['severity']}\n\n"
                    "Is this finding valid? Explain your assessment."
                ),
            }
        ],
    )


independent_verification = await asyncio.gather(*(verify(f) for f in sample_findings))

# Build calibration curve
calibration = build_calibration_curve(sample_findings, independent_verification)
# Adjust routing threshold based on calibration data

Appendix B — Interactive Study Prompts

Two prompts to paste into Claude. B1 drills the judgement the exam actually measures; B3 reviews the code you wrote for the Build Exercise above. The exam simulator between them is the interactive quiz on this page.

B1. Concept Check — Discrimination Drill

B2. Exam Simulator

B3. Build Coach — Code Review

The Build Exercise and its hint ladder are already on this page. This prompt is for the one thing the page cannot do: review the code you actually wrote.