Context Engineering for AI Agents: Architecture, Memory, Retrieval and 10 Templates

01

Context engineering vs prompt engineering

Prompt engineering defines the instruction for a task: the goal, constraints, examples, output format, and acceptance criteria. Context engineering designs the information environment in which that instruction runs. It decides which sources, memories, records, tool results, policies, and intermediate state enter the model's working context—and which material stays out.

The distinction becomes important when an assistant evolves into a workflow or agent. A single user can paste the right background into a one-off prompt. A production system must retrieve the right evidence for thousands of changing requests, respect permissions, preserve state across steps, and prevent stale or untrusted material from silently controlling an action.

The two disciplines reinforce each other. A precise prompt cannot compensate for missing evidence, while perfect retrieval cannot rescue an unclear task. Treat the prompt as the operating contract and context as the controlled working set assembled for one execution.

• Prompt engineering: what the model should do and how the result will be judged
• Context engineering: what the model may know, use, remember, and trust
• Workflow engineering: what tools and actions the surrounding system permits
• Evaluation: whether the complete system succeeds on representative cases

02

The seven layers of an effective context stack

A useful context stack separates stable instructions from changing task data. The first layer is the system policy: identity, scope, non-negotiable rules, and escalation boundaries. The second is the user goal. The third contains authoritative domain knowledge. The fourth contains request-specific records. The fifth carries examples and schemas. The sixth contains live tool state. The seventh records a concise execution history.

Do not merge all seven layers into an undifferentiated block. Label each source by owner, date, trust level, and purpose. When instructions conflict, the system should know which layer wins. When facts conflict, it should preserve the disagreement and ask or escalate instead of selecting whichever sentence appears last.

• Operating policy and permission boundary
• Current user request and success criteria
• Authoritative policies, documentation, and reference knowledge
• Case-specific records, files, and retrieved passages
• Approved examples, templates, rubrics, and output schemas
• Current tool results, errors, and action availability
• Compact state: decisions, completed work, open questions, and next step

03

Select context by usefulness, authority, and freshness

More context is not automatically better. Every additional passage consumes attention, increases latency, and creates another opportunity for conflict or distraction. Retrieve material because it can change the answer or action—not because it is available. A short authoritative policy can be more valuable than fifty semantically similar documents.

Rank candidate context on five dimensions: relevance to the current decision, authority of the source, freshness, permission to use it, and uniqueness. Remove duplicated passages and boilerplate. Keep source identifiers attached to facts so the final answer can cite evidence and a reviewer can trace the conclusion.

Define a retrieval threshold from evaluation rather than intuition. If the system often misses a necessary document, broaden recall. If it includes noisy neighbors, improve metadata, filters, chunking, ranking, or query generation before simply increasing the number of passages.

04

Design retrieval for decisions, not keyword similarity

A retrieval query should reflect the decision and the missing evidence. For a support request, retrieve the exact product version, customer entitlement, known incident, and approved resolution. For research, retrieve sources that establish each claim. For an agent action, retrieve the authenticated user's authority, destination, constraints, and current object state.

Use metadata filters before semantic ranking when the request includes clear boundaries such as customer, region, product, date, language, document status, or confidentiality class. Prefer current approved documents over drafts. Return a small set with source title, date, owner, trust level, and exact passage instead of disconnected text fragments.

Retrieval-augmented generation is only one part of context engineering. The application must also decide when retrieval is unnecessary, when a database lookup should replace semantic search, when a person must supply missing information, and when no available evidence supports an answer.

05

Manage memory without preserving every conversation

Memory should store information that remains useful beyond the current turn. Separate durable profile facts, task state, episodic history, and temporary working notes. Give each memory an owner, source, timestamp, sensitivity level, expiration rule, and correction path. Do not turn an entire transcript into permanent memory by default.

Before recalling a memory, check whether it applies to the current person, project, date, and scope. Present uncertain memories as candidates, not facts. If the user corrects a stored value, update or supersede it rather than keeping two equally trusted versions.

For long sessions, compress completed steps into a structured handoff containing goal, confirmed facts, decisions, artifacts, failures, pending questions, and next action. Compression should remove conversational repetition while preserving identifiers and constraints that affect future work.

06

Keep untrusted content separate from instructions

Webpages, emails, uploaded documents, retrieved notes, and tool outputs can contain text that looks like a command. Treat this material as data to inspect. Put it in clearly delimited fields, preserve its source, and never allow it to change permissions or the authenticated user's request.

Prompt wording is not a complete security boundary. The surrounding application must enforce authorization, tool allow-lists, parameter validation, destination checks, secret isolation, output handling, and human approval for consequential actions. One misleading passage should not be sufficient to make the system disclose data or perform an unauthorized operation.

07

Compress context while preserving decision-critical facts

Compression is useful when documents or conversations exceed the practical working set. Begin by defining what future steps need: dates, names, identifiers, requirements, decisions, exceptions, evidence, and unresolved conflicts. Then summarize around those fields rather than asking for a generic short version.

Keep provenance through compression. A summary should point back to the original source and distinguish direct facts from interpretation. For high-impact work, retain the exact passages that support the final decision even when surrounding material is compressed.

Use progressive disclosure: start with a compact index, retrieve detailed passages only when a step needs them, and archive completed intermediate material outside the model context. This reduces cost without forcing the system to forget the state that determines correctness.

08

Evaluate the complete context pipeline

An evaluation should test retrieval and assembly, not only the final wording. Create cases with missing evidence, stale sources, duplicated documents, conflicting policies, permission restrictions, irrelevant high-similarity passages, long histories, tool failures, and malicious-looking instructions inside data.

Measure whether the system selected the required sources, excluded forbidden material, cited evidence correctly, preserved user scope, used current state, asked when information was insufficient, and stopped before unauthorized actions. Track latency and cost beside task success because oversized context may improve one case while making the workflow impractical.

Review failures by layer. A wrong answer can come from a weak request, poor retrieval query, missing metadata, incorrect ranking, destructive compression, stale memory, tool error, or an ambiguous output contract. Fixing the responsible layer is more effective than repeatedly lengthening the system prompt.

• Context precision: how much selected material was genuinely useful
• Context recall: whether every decision-critical item was present
• Grounded accuracy: whether claims match supplied evidence
• Permission adherence: whether inaccessible data remained excluded
• State continuity: whether completed work and open issues survived
• Efficiency: tokens, latency, tool calls, and cost per accepted result

09

A practical implementation plan

Start with one recurring workflow and document its context contract. List the decisions, required facts, approved sources, permissions, freshness rules, memory needs, tool state, and output evidence. Build a small evaluation set before selecting retrieval or memory technology.

Implement the narrowest reliable pipeline: deterministic filters, a small retrieved set, structured source labels, explicit missing-data behavior, and a compact output schema. Add memory only after the workflow demonstrates which information genuinely needs to persist. Add autonomous actions only after permissions, validation, and approval paths work.

Monitor real failures and update the evaluation set. A context system is maintained like data infrastructure: sources change, documents age, user permissions move, and tasks evolve. Assign ownership for the context contract, retrieval quality, memory policy, and evaluation threshold.