Agent memory vs RAG vs tools: which one your agent actually needs

Every agent project eventually hits the same three-way argument. The agent forgot what the user said two turns ago, so someone proposes memory. It gave a wrong answer about the customer's contract, so someone proposes RAG. It can't actually do the thing the user asked for, so someone proposes more tools. Then the team spends a quarter building all three, and the agent is still stuck.

This piece is for the Head of AI or lead engineer trying to sequence that work. Memory, retrieval, and tools solve different problems. Picking the wrong one is the fastest way to burn a quarter. The short version: use tools when the agent needs to do or read something live, use RAG when the agent needs to reason over a bounded corpus of text, use memory when the agent needs continuity across sessions. Most production agents need tools first, some retrieval second, and memory only when the product genuinely spans sessions.

Here's each one on its own terms, then a side-by-side, then how to sequence the work.

How agent memory works

Agent memory is state that persists beyond a single request. In practice it splits into two things people conflate. Short-term memory is the conversation history inside the context window — the last N turns the model can see. Long-term memory is anything the agent writes to external storage and reads back later: user preferences, prior decisions, facts about entities the agent has interacted with before.

Long-term memory is usually a vector store, a key-value store, or a relational table sitting behind a retrieval step. On each turn, the agent (or the framework around it) decides what to write and what to read. General agent frameworks like LangGraph ship memory as one component among many; memory-first frameworks like Letta are built around it. Most production teams end up rolling something narrower against Postgres or Redis because the general abstractions don't fit their entity model.

Memory sounds obvious until you try to run it in production. Every write is a decision about what matters. Every read risks pulling in stale or contradictory facts. And the audit question — why did the agent believe this? — gets harder the more you write. Memory is powerful, but it's the layer that most often introduces bugs you can't reproduce.

How RAG works

Retrieval-augmented generation is a pattern for grounding model output in a corpus of text the model wasn't trained on. The agent takes a query, retrieves relevant chunks from a store (usually vector-indexed, sometimes hybrid with keyword search), and passes those chunks into the prompt as context. The model then answers using that context rather than parametric knowledge alone.

RAG's job is to make the model's output faithful to a specific body of text — internal documentation, product specs, past support tickets, policy documents. It's a read-only pattern over content. It doesn't take action, it doesn't reflect the current state of a live system, and it doesn't persist anything across sessions on its own. When people say "we added RAG," they usually mean they wired a vector database into the prompt path.

RAG works well when the answer lives in text and the text is bounded and reasonably static. It breaks in familiar ways: retrieval misses, chunk boundaries that split the answer, embeddings that don't match the query's phrasing, and — the failure mode people underestimate — the corpus being subtly out of date. We wrote about the practical trade-offs in RAG vs tool calling; the summary is that RAG is a good fit narrower than most teams assume.

How tools work

Tools are functions the agent can invoke to read live state or take action against a real system. A tool call is a structured request — name, arguments, expected schema — that runs against your product, a third-party API, or an internal service. The result comes back into the agent's context, and the model decides what to do next.

Tools are the only one of the three that touches the live system. Memory reads history. RAG reads text. Tools read and write the actual product. If the user asks "cancel this subscription and issue a pro-rata refund," no amount of memory or retrieval will get that done — you need a tool that hits the billing system as the authenticated user. This is why we've argued that most production agents need tool calling before retrieval: the moment an agent has to do something, retrieval alone stops being enough.

Tools also have the highest production cost. Every tool needs a schema, a stable contract, auth that runs as the right identity, error handling, rate-limit awareness, and observability. Get the tool descriptions wrong and the model picks the wrong one. Get the auth boundary wrong and security review kills the launch. Tools are where agents earn their keep and where projects most often stall.

Side by side

The pattern that falls out of the table: tools carry the most weight and the most cost, RAG is a narrower text-grounding pattern, and memory is a session-continuity pattern that's often over-scoped early.

When to choose agent memory

Reach for memory when the product genuinely spans sessions and the agent's usefulness depends on remembering what happened before. A coding assistant that learns your codebase conventions over weeks. A customer-support agent that recalls the user's prior tickets without re-asking. A sales agent that remembers which objections a prospect raised in the last call.

The signal you actually need memory: users repeat themselves and get annoyed, or the agent makes decisions that would be obviously better if it knew one specific fact from a prior session. If the answer is "we could just pass it in the prompt each turn," you don't need memory yet — you need a slightly better prompt or a lookup tool.

Skip memory when the interaction is single-session, when the facts change often enough that persisting them creates staleness bugs, or when you can't answer why did the agent believe this to a security review. Memory adds a debugging surface most teams underestimate.

When to choose RAG

Reach for RAG when the answer lives in a bounded body of text that the model doesn't know about, and the agent's job is to answer questions or summarise rather than act. Internal documentation search. Policy Q&A. Support agents grounded in a product's help centre. Legal or compliance assistants working over a specific corpus.

The signal you need RAG: users are asking questions whose answers exist somewhere in writing, and the current failure mode is the model making things up or citing outdated general knowledge. If the failure mode is instead "the model can't do the thing," RAG won't help — you need tools.

Skip RAG when the ground truth lives in a live system rather than a document. A common trap: teams RAG their API docs so the agent can "understand the API," then wonder why the agent still can't call it. The docs describe the API; the tool is the API. If you want the agent to act, expose the capability as a tool, not as retrievable text.

When to choose tools

Reach for tools when the agent needs to read live state or take action. Which is almost always, in production, for anything more useful than a chatbot. Booking, cancelling, updating, querying current data, triggering a workflow, pulling a report — all tools. Tools are also the right answer when the underlying data changes often enough that a retrieval index would be stale by the time it was queried.

The signal you need tools: the user's request maps to a verb that already exists somewhere in your product's UI. If a human employee would do this by clicking through your product, an agent needs a tool that does the same thing through the API.

The reason most agent projects stall on tools rather than memory or RAG is structural. SaaS products were built for the UI. The APIs typically expose only a small fraction of what the UI can actually do, while agents need to reach the rest. We've written the long version in why agent projects stall. The short version: memory and RAG are usually solvable with the systems you have; tools are usually not.

How Pontil fits

Pontil sits in the tools layer of the agent stack. We generate agent tools from your existing codebase — not from an OpenAPI spec you don't have, and not by rewriting your APIs. The generated tools run on a managed runtime that executes as the authenticated user, so permissions and audit trails hold. Maintenance is automated as your product changes, which is the part that quietly breaks bespoke connector projects at portfolio scale.

If your agent project is stuck on memory or retrieval quality, Pontil isn't the answer. If it's stuck because the agent can't actually reach what your product does — the far more common case — the Pontil product is built for exactly that shape of problem. The three patterns aren't in competition. They compose: tools for action and live reads, RAG for text-grounded answers, memory for session continuity. Pontil handles the layer most teams underestimate.

What we'd choose

Start with tools. Every production agent needs them, and the tools layer is where projects most often stall — not because the pattern is unclear, but because building and maintaining tools at scale is expensive without the right infrastructure.

Add RAG when a specific class of user question is failing and the answer genuinely lives in text you control. Keep the corpus narrow. Measure retrieval quality, not just end-to-end accuracy.

Add memory last, and only when you can name the specific session-spanning behaviour it unlocks. If you can't finish the sentence "the agent needs to remember X so that Y," you don't need memory yet.

The teams that get this right treat the three as complementary layers, not as a menu. The teams that stall usually built memory and RAG first because they're cheaper to prototype, then discovered the tools layer was the actual bottleneck a quarter in.