Don't Make Your LLM a Data Pipe: The Layer-First Pattern for Context Management

Here's a smell I've been seeing more and more in production agent systems: Tool A fetches data, the LLM receives the full payload, then the LLM passes it directly to Tool B. The model isn't making a decision about the content — it's a glorified pipe.

This pattern is everywhere. A wildfire monitoring agent calls get_wildfire_data() which returns 847 GeoJSON features (~500KB), the LLM holds that in context, then passes it to render_map(). A RAG pipeline fetches 30 documents, dumps them into the prompt, then asks the model to summarize. A data enrichment agent calls three APIs in sequence, and every intermediate payload flows through the LLM's context window.

Allan Bogh at RidgeText published a sharp breakdown of this problem, framed around a Mapbox mapping application. But the fix — a pattern he calls "layer-first" composition — applies far beyond maps. It's a general-purpose architecture for keeping LLM context lean by storing intermediate data server-side.

The Problem: LLM as a Data Pipe

Here's the naive flow that's burning tokens and context space:

get_wildfire_data() → 500KB GeoJSON → LLM holds it → render_map(that GeoJSON) → done

The LLM saw 125,000 tokens of geometry coordinates it was never going to reason about. It didn't decide which fires to display or how to style them. It was a courier.

This gets worse as you add layers. A real mapping application might stack wildfire perimeters, active hotspots, hiking trails, weather overlays, and evacuation zones. Each one adds 50–500KB of GeoJSON. Before you know it, your context window is a firehose of coordinates, and the model is hallucinating or truncating because it can't see the forest for the trees.

The same dynamic plays out in non-spatial applications. Every time your agent fetches data, holds it, and passes it to another tool without interrogating it, you're paying token costs for zero reasoning.

The Layer-First Pattern

The fix is deceptively simple: each data-fetching tool stores its result server-side and returns only a lightweight acknowledgment to the LLM. A compositor layer drains the queue later.

Here's what the LLM actually sees instead of 125,000 tokens:

[
  { "name": "retrieve_wildfire_layer", "result": { "status": "queued", "layerId": "wildfires-0", "featureCount": 847 } },
  { "name": "retrieve_trail_layer", "result": { "status": "queued", "layerId": "trail-1", "featureCount": 1 } },
  { "name": "generate_map", "result": { "mapUrl": "https://..." } }
]

~150 tokens. The GeoJSON lives in server memory — an in-process Map keyed by session ID with a 30-minute TTL. The LLM orchestrates what to fetch and in what order (insertion order determines rendering order, exactly like Mapbox layer stacking), but it never touches the actual data payload.

The render step is deterministic and testable in isolation:

async function generateMap(options: MapOptions): Promise<string> {
  const layers = drainLayerQueue(); // consume and clear
  const base = await fetchMapboxBase(options);
  const composed = await compositeLayers(base, layers);
  return await uploadToStorage(composed);
}

Because the rendering pipeline is decoupled from the LLM's tool-calling loop, you can swap the renderer — static tiles today, headless Mapbox GL JS via Playwright tomorrow — without touching the LLM interface at all.

Why This Matters Beyond Maps

Bogh's post is framed around mapping, but the signal is universal: Tool A fetches data → LLM receives it → LLM passes it directly to Tool B. If the LLM isn't making a decision based on the content, it shouldn't be holding the content at all.

Apply this to:

Multi-source data enrichment. Queue each source server-side. A compositor joins them on a shared key (station ID, user ID, document hash). The LLM gets a structured summary, never the intermediate payloads.

Log analysis with multiple passes. Fetch the logs once. Store them. An error-count tool and a root-cause-analysis tool read from the same stored dataset independently. The LLM gets two small structured answers instead of the full log dump twice.

Any ETL pipeline where the output is what matters. The LLM decides what to pull and describes the result. The intermediate state belongs in your pipeline, not in the context window.

Tradeoffs Worth Knowing

This pattern isn't free. There are real tradeoffs:

The biggest constraint is that you're explicitly choosing not to let the LLM inspect the raw data. If your use case requires the model to make granular decisions based on individual records (re-rank search results, classify each document), this pattern isn't for you. But if the LLM's job is to decide what data to gather and in what order, while the actual processing happens server-side, this is your pattern.

For the persistence concern: the source implementation uses an in-process Map with 30-minute TTL, which is fine for single-turn interactions. For multi-turn conversations, persist the layer queue to a database table and rehydrate it with follow-up commands. The interface doesn't change — just the backend.

What This Means for Agent Tool Design

The layer-first pattern suggests a broader design principle for agent tools: return metadata, not data. If your tool call returns a payload that the model is expected to immediately hand off to another tool, you've designed the wrong abstraction.

The better shape is:

1. The fetch tool stores data server-side, returns a lightweight handle (layer ID + feature count + status).
2. The compose tool reads from the stored handle, processes the data, and returns the final artifact.
3. The LLM only ever sees handles and summaries.

This isn't a new idea in software architecture — it's basically a producer-consumer queue with the LLM as the orchestrator. But it's surprising how many agent systems skip this and dump raw payloads into context because it's the easy path.

The most useful question you can ask about your agent's tool chain right now: Where is my LLM acting as a data pipe instead of a reasoning engine? Each answer is an opportunity to cut context usage by an order of magnitude.

The full post and implementation details are on RidgeText's blog. Go read it — it's short, and the code samples are well worth the click.