Context Engineering: An Architecture Guide to Sessions and Memory for Stateful Agents

Fetch context: Memory, RAG, recent events, and task metadata
        ↓
Prepare context: the framework assembles the full request on the hot path
        ↓
Invoke the LLM and tools: iterate while appending model and tool outputs
        ↓
Upload context: send durable information to an asynchronous persistence pipeline

Sessions Hold the State of the Current Conversation

Events and state have different jobs

A Session is a self-contained record for one user and one continuous conversation. It contains events appended in time order and mutable working state. Events record user inputs, agent responses, tool calls, and tool outputs. State holds structured temporary data required by the current task, such as cart contents, confirmed filters, or intermediate calculations.

A production agent runtime is commonly stateless after a request finishes, so Session history must be persisted. In-memory storage is suitable for development; production needs a reliable database or managed Session service and must preserve deterministic event ordering.

Framework differences should stay behind the application boundary

Agent frameworks do not define Session, Event, and state in the same way. Google ADK exposes an explicit Session containing an Event list and separate state. LangGraph treats mutable state as the Session, with message history and other working data in one state object.

The framework maps its internal events to the request format required by a model API. For example, Gemini requests use a Content list with role and parts. Business logic should depend on a stable internal abstraction while the framework performs the conversion, preventing a provider-specific message schema from spreading through the application.

Multi-agent systems should share semantic state

In a multi-agent system, each agent may keep a private history and return a task result to a parent agent, or the agents may share richer Session state. A2A messages are useful for structured tasks and results, but they do not by themselves solve history-schema translation across frameworks.

Cross-framework collaboration benefits from a framework-agnostic Memory layer. It stores processed summaries, entities, and facts in common structures such as strings or dictionaries. Session keeps raw events; Memory provides a shared knowledge layer that heterogeneous agents can read.

Why Long Conversations Need Compaction

Long context raises four costs at once

Session history may be read from a central store and sent to the model on every turn. Without management, the context limit, API bill, response speed, and answer quality all come under pressure.

The aim of compaction is to shrink history to a model-usable range while retaining facts, decisions, and constraints needed by the current task. The stored history and the view sent to the model can be separated: compaction may change only the view for one call, or persist a reusable summary for later turns.

Three Session compaction strategies

• Keep the last N turns: Use a sliding window and send only the most recent turns. It is simple and low-latency, and fits conversations where old material has little value.
• Token-based truncation: Count from the newest message backward until a token budget is reached. This directly controls payload size, but it can abruptly discard an important earlier fact.
• Recursive summarization: Replace older messages with an LLM-generated summary and combine it with recent verbatim messages. This preserves more of the task narrative, but adds a model call and requires summary-quality checks.

Expensive work such as recursive summarization should run in the background, and its result should be persisted so that each turn does not repeat the computation. The system should also record which source events a summary covers, preventing already-compacted text from being sent to the model again without reason. Compaction can be triggered by turn count, token count, time, or an event such as task completion.

How Memory Extracts Durable Information from Conversations

The boundary between Memory and Session

A Session is the immediate work surface of one conversation. Memory stores information extracted from a conversation or another source for future interactions. The two systems reinforce each other: Sessions provide the main raw material for Memory generation, while Memory helps compact Sessions and reduces the amount of raw history passed between turns.

A Memory generally contains content and metadata. Content should use framework-agnostic strings, dictionaries, or JSON where possible. Metadata can record a memory ID, owner, source, time, labels, and governance information. Memories describe information that happened or was explicitly expressed; predictive guesses should not be treated as user facts.

RAG and Memory have different responsibilities

RAG and Memory both supply information to context, but their sources, isolation levels, and write patterns differ. RAG fits shared, stable external facts. Memory fits personalized information scoped to a user and updated through interaction.

A Memory Manager is more than a vector-similarity database. It must also extract, deduplicate, resolve conflicts, update, and invalidate information. RAG gives an agent domain facts, while Memory gives it continuity with a user. Both can participate in the same Context Engineering loop.

Memory types and organization patterns

By information content, Memory can be declarative or procedural. Declarative Memory answers “what” and contains facts, history, and user profiles. Procedural Memory answers “how” and contains reusable workflows, tool-call sequences, and problem-solving strategies.

Three common organization patterns are useful. Collections store multiple independent events or observations as a searchable set. A Structured User Profile keeps stable facts such as names, preferences, and account details in a continuously updated profile. A Rolling Summary consolidates the user-agent relationship into one evolving natural-language document and is often used to control the token count of a long Session.

For storage, vector databases retrieve natural-language atomic facts by semantic similarity. Knowledge graphs represent entities and relationships and support structured relational queries. A hybrid design combines graph structure with vector representations to support both relationship reasoning and semantic search.

Memory can also be classified by its creation path. Explicit memories come from a direct “remember this” request. Implicit memories are extracted from conversation, so their trigger and confidence need separate governance. Internal Memory is managed inside the agent framework and is convenient for starting out. External Memory uses a dedicated service for generation, storage, and retrieval and can scale independently. Multimodal inputs can be converted into textual insights, or the media itself can be retained as memory content; the latter raises additional storage, retrieval, and model-interface requirements.

Memory Generation Requires Extraction, Consolidation, and Provenance

The four-stage memory-generation pipeline

Memory generation resembles an LLM-driven ETL pipeline, but the model also helps judge the content:

1. Ingestion: Accept conversation history, direct memories, or another raw source.
2. Extraction & Filtering: Extract meaningful information according to predefined topics. If no information matches, create no Memory.
3. Consolidation: Compare new information with existing memories and decide how to handle duplicates, conflicts, and change through update, creation, or deletion.
4. Storage: Persist the final memory in a vector database, knowledge graph, or another durable store for future retrieval.

Consolidation handles updates, conflicts, and forgetting

Without consolidation, Memory becomes a noisy log full of duplicate and contradictory entries. A consolidation workflow commonly retrieves old memories similar to the new information and then asks an LLM to choose an operation:

• UPDATE: Change an existing memory with new or corrected information.
• CREATE: Create an independent memory when the insight is novel and unrelated.
• DELETE / INVALIDATE: Remove or invalidate an old memory when it is wrong, irrelevant, or no longer useful.
• FORGET: Prune stale or low-confidence content using freshness, confidence, or TTL.

User preferences and state change over time. The system needs to let newer information supersede older information and retain enough lineage to explain that change. Deletion must also account for derivation: when one Memory depends on several sources and a user revokes one source, the system needs to find and process the affected derived memories.

Provenance sets the trust level of a Memory

A Memory's source and evolution history influence consolidation priority and the weight it should receive during inference. Common sources deserve separate assessments of trust and stability:

When sources conflict, the system can prefer the more trusted source, the newer source, or corroboration across multiple data points. Memory Poisoning is another provenance problem: malicious input can inject false facts or instructions into long-term storage. Validation, sanitization, and access control are needed before committing content to durable Memory.

When to Retrieve Memory and How to Place It in Context

Rank relevance, recency, and importance together

Memory Retrieval must find useful information within a latency budget. Ranking only by vector similarity can surface an old or trivial memory that is semantically close. A stronger ranking model combines at least three dimensions:

Query rewriting, reranking, and specialized retrievers can improve accuracy, but extra LLM calls and ranking work add latency. When results are stable and do not become stale quickly, caching can offset part of the cost. A high-quality Memory Corpus usually matters more than sophisticated retrieval tricks.

Proactive retrieval and tool-based retrieval

Proactive retrieval loads Memory at the start of every turn. It fits stable information such as a user profile, but it makes turns that do not need Memory pay the retrieval cost. Reactive retrieval, also called Memory-as-a-Tool, gives the Agent a query tool and lets it decide when Memory is needed. This reduces unnecessary retrieval, but adds a tool-decision call and may fail when the Agent does not know that a relevant memory exists.

The tool description can state what kinds of information are available, helping the Agent decide more accurately. Either strategy needs test sets for missing memories, irrelevant memories, and latency.

Boundaries for system-instruction and dialogue injection

Stable global information that should be present on every turn can be appended to system instructions. This keeps dialogue history clean and gives the Memory high instruction authority; it also creates a risk that the Agent will force every topic to relate to the profile. The framework must support dynamic system-prompt construction and handle multimodal memories that do not fit naturally into text-only instructions.

Short-lived or task-specific memories can be injected into conversation history or returned as tool output. This is flexible, but it can add token noise and cause the model to mistake a Memory for something actually said in the original dialogue. The injection should identify source, perspective, and trust level. A hybrid strategy often balances stable profiles with temporary context.

Procedural Memory Records How to Execute

Declarative and procedural Memory solve different problems

Most commercial Memory systems are optimized for declarative information: user preferences, history, and “what” facts. Procedural Memory stores “how to do it” by extracting reusable playbooks from successful interactions.

Procedural Memory also has extraction, consolidation, and retrieval stages, but its consolidation target is a workflow. It absorbs successful methods, patches existing steps, and removes outdated or ineffective procedures. Retrieval returns an executable plan for a complex task rather than data for a factual answer. Procedural Memory may therefore need a different schema and evaluation standard from declarative Memory.

The boundary between procedural Memory and fine-tuning

Fine-tuning is an offline training process that changes model weights, often with a long cycle and a broad effect. Procedural Memory enables fast online adaptation by injecting an appropriate playbook during inference without changing the model weights. It fits updateable procedures and strategies, while still requiring versioning, provenance, rollback, and security review.

Production Safety, Reliability, and Evaluation

Production constraints for Sessions and Memory

Production Session governance must cover privacy, security, data integrity, lifecycle, and performance. Every access should authenticate and authorize the user or tenant, with ACLs enforcing strict isolation. PII should be redacted before Session data is written. Sessions need TTL, archive, and deletion policies, and events must be appended in deterministic order.

Session sits on the hot path of every response, so read and write operations, network transfer, and context compaction need explicit budgets. Memory generation usually requires LLM calls and database writes, so a dedicated service should process it asynchronously. The Agent pushes raw data, the service extracts and consolidates it in the background, the result is persisted, and the Agent retrieves it during a later interaction.

At high concurrency, the system must handle races when the same Memory is updated at the same time. Transactions or optimistic locking can help, while queues absorb spikes. Transient errors need exponential-backoff retries; persistent failures should go to a dead-letter queue. Global deployments need storage-level multi-region replication while preserving one transactionally consistent logical view.

Three evaluation layers

Memory evaluation should answer three separate questions: Is the system remembering the right content? Can it retrieve that content when needed? Does using it help the Agent complete the task?

Start with a manually maintained golden set. Then iterate through a baseline, failure analysis, prompt or retrieval tuning, and re-evaluation. Generation and consolidation may run in the background, but throughput, failure rate, and high-concurrency stability also need measurement.

A minimal implementation checklist

1. Define the boundary between Session, Event, State, and Memory, including which information is valid only for the current task.
2. Record user or tenant scope, source, time, freshness, confidence, and deletion paths for every Memory.
3. Start with a sliding window or token truncation for Session control, then add recursive summarization when quality requirements justify it.
4. Put Memory generation, consolidation, and cleanup in a non-blocking background pipeline, and record lineage between source events and derived memories.
5. Choose system instructions, dialogue injection, or Memory-as-a-Tool for each Memory type, and label the source explicitly.
6. Run continuous regression over quality, retrieval, task success, latency, throughput, and security cases to measure the practical value of each architecture change.

Context Engineering connects request assembly, the immediate state of a Session, and the long-term governance of Memory in one engineering loop. Session provides low-latency continuity for the current conversation; Memory provides persistence and personalization across conversations. Reliability comes from information selection, traceable data lifecycles, and continuous evaluation.