20 Questions — Domain 3 : Integration Quiz

Expand any question to reveal the correct answer and explanation.

1. 1 A technical lead needs to provide a suite of database inspection tools to three different internal teams: one using a custom Claude API integration, one using Claude Code, and one using the Claude Desktop application. According to the CCAR-P framework, which integration mechanism is most appropriate?

   Consider which protocol is designed for the 'build once, expose everywhere' requirement.

   Model Context Protocol (MCP)

   MCP is the preferred choice when a single capability must be discoverable and reused across multiple client surfaces and teams.

   • ✗ Direct API Integration

     While technically functional, this creates redundant maintenance overhead as each client surface would require a custom-built integration.

   • ✗ Agent-to-Agent Delegation

     This pattern is intended for systems that must maintain independent context and reasoning, not for sharing a common toolset across interfaces.

   • ✗ Monolithic Context Injection

     Front-loading all tools into a single context window is inefficient for multi-team reuse and risks model attention dilution.

2. 2 An architect observes that an agent is frequently misrouting requests between two tools with overlapping descriptions, leading to increased token usage and errors. What is the recommended 'structural fix' for this capability bloat?

   Focus on the architectural change that minimizes the number of decisions the model has to make at once.

   Split the overloaded agent into scoped subagents.

   Specializing agents into narrow domains of responsibility improves tool-selection accuracy and reduces the risk of cognitive overload.

   • ✗ Add a routing classifier prompt to the system instructions.

     Relying on prompt-based enforcement is a probabilistic approach that doesn't address the underlying architectural issue of tool overlap.

   • ✗ Increase the context window to improve the model's 'attention'.

     Expanding the context window is a 'trap answer' that often worsens attention dilution rather than fixing poor tool mapping.

   • ✗ Implement a retry loop to catch misrouting errors.

     Retry loops are reactive band-aids that increase latency rather than proactively solving the structural selection failure.

3. 3 When designing a RAG pipeline for a massive corpus of legal documents, an architect chooses a large chunk size (entire sections) over a small chunk size (single sentences). What is the primary architectural trade-off of this decision?

   Think about how including more surrounding text affects the specificity of a search result.

   It preserves cross-references but dilutes relevance scoring.

   Larger chunks maintain vital context and internal references but make it harder for the embedding model to pinpoint highly specific facts.

   • ✗ It improves precision but increases retrieval latency.

     Larger chunks typically degrade precision because they include more 'noise' relative to a specific query.

   • ✗ It reduces token costs but loses semantic grounding.

     Larger chunks actually increase token costs per retrieval because more text is injected into the prompt.

   • ✗ It ensures $100\%$ accuracy at the cost of document recall.

     No chunking strategy guarantees perfect accuracy, and larger chunks generally favor context over specific recall.

4. 4 In a production Claude deployment, why is using a single shared service account for all MCP tool invocations considered a high-risk security gap?

   Consider the impact on accountability and the principle of least privilege.

   It erases per-user access boundaries and audit trails.

   Shared accounts create 'confused deputy' risks where an agent might perform actions the actual user is not authorized to take.

   • ✗ It causes the MCP server to exceed token rate limits.

     While rate limits are an operational concern, the primary professional-level risk is the violation of least-privilege principles.

   • ✗ It prevents the model from accessing environment variables.

     Environment variables are typically accessible to the server process regardless of the specific account used for tool calls.

   • ✗ It requires hard-coding credentials into the CLAUDE.md file.

     Credentials should never be hard-coded; the account type and storage location are distinct architectural issues.

5. 5 A RAG system begins returning confident but incorrect answers immediately after a document index refresh. The model version and latency remain unchanged. What is the most likely root cause?

   Map the symptom to the specific component in the architecture that was recently modified.

   A broken re-index or mismatched embeddings feeding poor context.

   When symptoms appear specifically after a data refresh, the failure is almost certainly in the retrieval/indexing pipeline, not the model.

   • ✗ The model's temperature is set too high for the new data.

     Temperature affects variance, but a sudden shift in grounding quality post-refresh points to a data retrieval failure.

   • ✗ The new documents exceeded the model's context window.

     RAG systems are designed to handle data larger than the context window by retrieving only relevant chunks; window limits aren't the primary factor here.

   • ✗ The model's weights have drifted due to the new index.

     Claude's weights are static; they do not change or drift based on the contents of an external RAG index.

6. 6 What is the primary benefit of using a 'Progressive Discovery' strategy over a 'Monolithic Context' strategy when integrating 50+ enterprise tools?

   Think about how much 'noise' the model has to ignore if every possible tool is listed in every message.

   It improves attention allocation and reduces token costs.

   Loading only relevant tool definitions on demand keeps requests lean and prevents the model from becoming overwhelmed by irrelevant options.

   • ✗ It guarantees deterministic tool execution order.

     Execution order is managed via prompt structure or tool-choice settings, not by the discovery method itself.

   • ✗ It allows tools to share a single global memory space.

     Tools generally operate independently; progressive discovery is about managing the model's available options, not shared state.

   • ✗ It eliminates the need for Model Context Protocol (MCP) servers.

     Progressive discovery can be (and often is) implemented specifically through MCP to manage large tool catalogs.

7. 7 An architect is designing a multi-step tool chain where a 'search' tool finds a file and a 'move' tool relocates it. How should the information be passed between these tools to ensure reliability?

   Consider the best way to uniquely identify an object in a programmatic system.

   The first tool should output a unique machine-readable identifier (ID).

   Using explicit IDs (e.g., file_id) prevents ambiguity and errors that occur when passing unstructured text or raw URLs between agents.

   • ✗ The agent should summarize the search results and pass the summary.

     Summarization can drop critical metadata needed by the second tool, leading to 'lost in the middle' or hallucination effects.

   • ✗ The move tool should independently re-verify the search path.

     While safe, this creates unnecessary latency and doubles the operational cost of the integration.

   • ✗ Both tools should share the same global environment variable.

     Global variables are difficult to manage in parallel multi-agent systems and don't provide the necessary traceability.

8. 8 Under what specific condition should a 're-ranking' stage be added to a RAG pipeline?

   Think about a scenario where the initial search returns too many results that look relevant but aren't.

   When first-stage retrieval has high recall but low precision.

   A re-ranker filters a broad set of candidates to ensure only the most relevant chunks reach the prompt, which is essential for large, noisy corpora.

   • ✗ When the model's context window is less than 100k tokens.

     Re-ranking is a retrieval quality optimization, not a direct fix for context window capacity.

   • ✗ When the cost per input token exceeds the budget.

     Adding a re-ranking model actually increases total costs because it introduces an additional processing step.

   • ✗ When the system requires real-time streaming output.

     Re-ranking adds latency to the initial processing phase, which can actually delay the start of streaming.

9. 9 A developer is configuring a shared MCP server in a team environment. Where should they store the server-level authentication tokens to follow CCAR-P best practices?

   Search for the method that balances team sharing with credential security.

   In environment variables referenced within .mcp.json.

   This allows for secure credential management while keeping the project configuration version-controlled without committing secrets.

   • ✗ Directly in the CLAUDE.md file for easy discoverability.

     Storing secrets in a markdown file committed to the repository is a major security violation.

   • ✗ In a hard-coded string inside the tool's schema description.

     This exposes credentials in every model request, burning tokens and creating a massive security leak.

   • ✗ In the user's personal ~/.claude.json file only.

     While secure, this prevents the team from sharing the tool configuration efficiently across a project.

10. 10 Which scenario triggers the 'Confused Deputy' problem in an agentic integration?

    Think about a situation where one entity incorrectly uses its power to help another unauthorized entity.

    An agent uses broad system permissions to act on behalf of a restricted user.

    If tool endpoints skip identity checks, a model can be tricked into using its higher-level access to bypass user-level security.

    • ✗ Two subagents attempt to write to the same file simultaneously.

      This is a race condition or concurrency error, not a confused deputy security vulnerability.

    • ✗ A model hallucinates a tool call that does not exist in the schema.

      This is a capability or grounding failure, not a permission escalation problem.

    • ✗ An MCP server fails to respond within the 30-second timeout window.

      This is a transient network or performance failure.

11. 11 You are building a system that analyzes 500-page insurance policies. If the entire document fits within the context window, when would RAG still be the wrong choice?

    Look for a mechanism that rewards reusing a stable, large piece of information.

    When prompt caching can eliminate re-processing costs for repeated queries.

    If the whole doc fits and is reused, caching the entire prefix is cheaper and more accurate than the added complexity of RAG.

    • ✗ When the task requires high-volume pattern matching.

      RAG is often used specifically to handle high volumes, though accuracy-latency trade-offs must be evaluated.

    • ✗ When the document contains many high-resolution images.

      Images are generally handled via multimodal capabilities, which would still require effective context management or RAG for very large sets.

    • ✗ When the user requires low-latency streaming responses.

      RAG can sometimes lower latency by reducing the total prompt size, but it increases the complexity of the initial retrieval step.

12. 12 In a multi-agent system, what is the 'Task' tool primarily used for?

    Think about the mechanism that allows one agent to 'create' another one to do a specific job.

    Spawning and delegating work to specialized subagents.

    The Task tool is the native mechanism for a coordinator to invoke subagents, provided 'Task' is in the allowedTools list.

    • ✗ Managing environment variables for MCP servers.

      Environment variables are handled at the configuration level (mcp.json), not via an agent tool.

    • ✗ Intercepting tool calls to apply safety guardrails.

      Tool call interception is a programmatic hook in the SDK, not a tool called by the model itself.

    • ✗ Calculating the total token cost of a batch request.

      Token usage is typically reported by the API or a monitoring layer, not a model-invoked tool.

13. 13 What is the primary difference between MCP 'Tools' and MCP 'Resources'?

    Distinguish between taking an action and simply viewing a directory or summary of information.

    Tools take actions or perform queries; Resources expose content catalogs and data hierarchies.

    Exposing data as Resources allows the agent to 'see' what is available without making repetitive exploratory tool calls.

    • ✗ Tools are for internal use; Resources are for external third-party integrations.

      Both can be internal or external; the distinction is their function (action/query vs. data discovery).

    • ✗ Tools have strict JSON schemas; Resources use unstructured markdown exclusively.

      Both are structured parts of the protocol designed to give the model context in a predictable way.

    • ✗ Tools require human-in-the-loop validation; Resources never do.

      Validation requirements depend on the risk of the action, which is more common for Tools, but not an absolute distinction.

14. 14 A developer receives an error from an MCP tool. To enable an 'intelligent recovery' by the coordinator agent, what must the tool return?

    Think about what information you would need to fix a mistake you just made.

    A structured response with isError: true and specific failure metadata.

    Providing context like 'failure type' and 'attempted query' allows the coordinator to decide whether to retry, change parameters, or escalate.

    • ✗ A generic 'Operation Failed' message to save on token costs.

      Uniform, generic errors hide valuable context and prevent the agent from making informed recovery decisions.

    • ✗ An empty result set to signal a successful but null query.

      Silently suppressing errors (returning empty results as success) is an anti-pattern that leads to 'hallucinations of absence'.

    • ✗ A natural language apology including the model's internal stack trace.

      Models don't have internal stack traces in the traditional sense, and unstructured prose is harder for the agent to parse reliably.

15. 15 How does 'Chunk Overlap' specifically improve RAG performance?

    Think about a sentence that starts on page 1 and ends on page 2.

    It ensures concepts split across section boundaries are captured in shared context.

    Like roof shingles, overlapping chunks prevent crucial context from 'slipping through the cracks' at the end of one segment and the start of the next.

    • ✗ It reduces the total number of chunks stored in the vector database.

      Overlap actually increases the number of chunks and total storage required by duplicating text near the boundaries.

    • ✗ It forces the model to ignore the beginning of every prompt.

      Overlap has no effect on the model's instruction following; it is a data preparation technique for better retrieval.

    • ✗ It eliminates the need for semantic embedding models.

      Overlap works alongside embeddings to ensure the retrieved segments are contextually complete.

16. 16 When an agent is configured with 50+ different API connectors, it often chooses the wrong one. What is the most effective architectural fix?

    Focus on a solution that proactively prevents error by limiting options.

    Implement dynamic scoping to expose only relevant tools based on user intent.

    Reducing the number of decisions (the decision space) at execution time is more effective than simply improving tool descriptions.

    • ✗ Provide all 50+ tools in the system prompt but use larger bold text for headers.

      Visual formatting does not significantly impact a model's ability to navigate massive schemas; cognitive overload remains the issue.

    • ✗ Switch to a monolithic API that handles all 50 connectors internally.

      This prevents the agent from inspecting specific parameter requirements, which often leads to more opaque errors.

    • ✗ Add a few-shot example for every single connector in the catalog.

      Including 50+ examples would consume a massive portion of the context window and likely trigger the 'lost in the middle' effect.

17. 17 What should an architect do with an MCP tool annotation that says readOnlyHint=true?

    Consider how you would treat a sign that says 'No Entry' on a door that is already open.

    Treat it as untrusted metadata and verify the server's identity separately.

    Self-reported labels from third-party servers should be treated as suggestions; security policies should be based on system-level trust.

    • ✗ Automatically bypass all user confirmations for the tool.

      Trusting self-reported metadata can lead to security breaches if a malicious server falsely labels a destructive tool as read-only.

    • ✗ Assume the MCP protocol's internal sandbox will enforce the restriction.

      MCP defines the communication protocol; it does not inherently sandbox the execution of third-party tools on the host.

    • ✗ Merge the tool into the CLAUDE.md file to make it 'trusted'.

      The location of the description (CLAUDE.md vs. MCP) does not change the trustworthiness of the underlying tool code.

18. 18 How does 'Observability at Scale' differ between a single-model app and a multi-agent system?

    Think about a relay race where the final runner drops the baton�who is really at fault?

    Failures in multi-agent systems often manifest symptomatically several steps downstream.

    Tracing must cover the entire path (retrieved chunks, subagent tool calls, coordinator logic) to identify where the chain actually broke.

    • ✗ Multi-agent systems do not require latency monitoring because the subagents are parallelized.

      Parallelization helps, but total pipeline latency remains a critical SLA that must be tracked.

    • ✗ Only the final response needs to be logged for audit purposes in multi-agent systems.

      Auditors require reconstruction of the full payload, including intermediate tool calls and subagent reasoning.

    • ✗ Multi-agent systems use deterministic logging which eliminates the need for trace IDs.

      These systems are probabilistic and non-linear, making unique request IDs and end-to-end tracing more important than ever.

19. 19 A support agent needs to retrieve customer order details. A tool returns 40+ fields per order, but only 5 are relevant to the synthesis step. What should the architect do?

    Focus on a solution that actively removes 'noise' without involving the model's reasoning.

    Trim the tool output to include only relevant fields before appending to history.

    This optimizes the context window and prevents 'attention dilution' caused by irrelevant data (noise).

    • ✗ Leave the full output in history to preserve provenance and auditability.

      While audit logs should keep the full data, the *active context* provided to the model should be lean to maintain performance.

    • ✗ Request the model to summarize the tool output after every call.

      Summarization can introduce hallucinations and costs additional tokens compared to programmatic trimming.

    • ✗ Switch to a larger model tier that can handle the increased token count.

      Upgrading models is an expensive 'trap answer' that doesn't solve the core inefficiency of the context design.

20. 20 Which RAG strategy is most effective for a corpus partitioned by clean categories like 'Region' or 'Product Line'?

    Consider the most efficient way to narrow down a search before you even start reading the content.

    Pre-filter by metadata before performing semantic search.

    Filtering by metadata (e.g., region=US) reduces the search space, improving both precision and retrieval speed.

    • ✗ Use a larger model to perform a global search across all partitions simultaneously.

      Even capable models perform better when provided with focused, relevant context rather than a 'global' dump of irrelevant data.

    • ✗ Rely exclusively on semantic embeddings to sort through the partitions.

      Embeddings are probabilistic; using deterministic metadata filters first provides a much stronger guarantee of relevance.

    • ✗ Increase the retrieval count ($k$) until all regions are represented in the prompt.

      Stuffing context with irrelevant regions Dilutes the model's attention and increases costs without improving answer quality for the target region.