Back

LangChain and LangGraph middleware: seven small lessons

16 min read
On this page

This post is about middleware in LangChain agents.

An agent runs in a loop. It calls the model. It maybe calls a tool. It repeats until it has an answer. Middleware is how you step into that loop and run your own code.

I’ll go through seven small lessons. Each one adds one idea. By the end you have a single file you can run, one lesson at a time.

The agent here is built with createAgent from LangChain. Under the hood that is LangGraph, so this is LangGraph middleware too.

What middleware is

Think of the agent loop as a few fixed points:

  • The agent starts.
  • The model is called.
  • The model replies.
  • A tool runs.
  • The agent ends.

Here is that loop as a picture:

   user message
        │
        ▼
      MODEL  ── the agent calls the model
        │
        ▼
   need a tool?
     │       │
    yes      no
     │       │
     ▼       ▼
    TOOL   answer
     │
     └─► loop back to MODEL

Middleware lets you hook into those points. You write the code once. It runs on every call.

What people use it for:

  • Logging what the agent is doing.
  • Timing the model call.
  • Retrying when something fails.
  • Catching tool errors.
  • Stopping the agent early.
  • Swapping the model per request.

The hooks

A middleware is an object with named functions. These are the ones we use:

  • beforeAgent — runs once, when the agent starts.
  • beforeModel — runs before every model call.
  • afterModel — runs after every model call.
  • afterAgent — runs once, when the agent ends.
  • wrapModelCall — wraps the model call. You call the model yourself.
  • wrapToolCall — wraps a tool call. Same idea.

This is where each hook fires in the loop:

   user message
        │
        ▼
   beforeAgent              runs once, at the start
        │
        ▼
   ┌─► beforeModel          before every model call
   │       │
   │       ▼
   │     MODEL              the model answers
   │       │
   │       ▼
   │   afterModel           after every model call
   │       │
   │       ▼
   │   need a tool?
   │     │       │
   │    yes      no
   │     │       │
   │     ▼       ▼
   └── TOOL   afterAgent    runs once, at the end
                  │
                  ▼
                answer

wrapModelCall wraps the MODEL box. wrapToolCall wraps the TOOL box.

Three more things show up later:

  • stateSchema — adds your own fields to the agent’s state.
  • contextSchema — read-only data you pass in per request.
  • jumpTo — exit the loop early.

You only add the hooks you need. The rest you leave out.

What you need

  • Node.js 20.6 or newer. It runs ES modules and reads .env on its own. Check with node --version.
  • An OpenRouter API key from openrouter.ai, starting with sk-or-v1-.
  • Three npm packages. Next step.

Setup

mkdir middleware-demo && cd middleware-demo
npm init -y
npm pkg set type=module
npm install langchain @langchain/openrouter zod

We use OpenRouter so one key covers every model. It also lets you switch providers later, which lesson 6 leans on.

Shared pieces

Every lesson shares the same model, two tools, and two tiny helpers.

import { ChatOpenRouter } from "@langchain/openrouter";
import {
  AIMessage,
  createAgent,
  createMiddleware,
  HumanMessage,
  tool,
  ToolMessage,
} from "langchain";
import * as z from "zod";

// Paste your key here. Fine for a script on your own machine.
process.env.OPENROUTER_API_KEY = "sk-or-v1-paste-your-key-here";

// Model ids on OpenRouter are "provider/model".
const MODEL = new ChatOpenRouter({
  model: "openai/gpt-4o-mini",
  temperature: 0,
  maxTokens: 300,
});

// A tool that works.
const getWeather = tool(async ({ city }) => `It's 22°C and sunny in ${city}.`, {
  name: "get_weather",
  description: "Get the weather for a city",
  schema: z.object({ city: z.string() }),
});

// A tool that always fails, for lesson 4.
const flakyTool = tool(
  async () => {
    throw new Error("Database is down!");
  },
  {
    name: "get_stock_price",
    description: "Get a stock price",
    schema: z.object({ ticker: z.string() }),
  },
);

const lastText = (result) => result.messages.at(-1).content;
const title = (t) =>
  console.log(`\n${"=".repeat(60)}\n ${t}\n${"=".repeat(60)}`);

Now the lessons.

Lesson 1: a plain agent

No middleware yet. This is the baseline.

async function lesson1() {
  title("LESSON 1: Plain agent (no middleware)");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "Weather in Paris?" }],
  });

  console.log("Answer:", lastText(result));
}

The agent calls get_weather on its own, then answers. Nothing of ours runs in between. That changes in lesson 2.

Lesson 2: your first middleware

The node-style hooks. They run before and after each step.

  • beforeModel can also change the state. Return an object and it gets merged in.
  • Return nothing and the state is left alone.
const loggingMiddleware = createMiddleware({
  name: "LoggingMiddleware",

  beforeAgent: () => {
    console.log("🚀 [beforeAgent] Agent starting");
  },

  beforeModel: (state) => {
    console.log(
      `➡️  [beforeModel] Sending ${state.messages.length} messages to LLM`,
    );
    // Add a message before the model runs.
    return { messages: [new HumanMessage("Answer in one sentence.")] };
  },

  afterModel: (state) => {
    const last = state.messages.at(-1);
    const tools = last.tool_calls?.map((t) => t.name) ?? [];
    console.log(
      `⬅️  [afterModel] LLM replied. Tool calls: [${tools.join(", ")}]`,
    );
  },

  afterAgent: () => {
    console.log("🏁 [afterAgent] Agent finished");
  },
});

async function lesson2() {
  title("LESSON 2: Logging with node-style hooks");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [loggingMiddleware],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "Weather in Tokyo?" }],
  });
  console.log("Answer:", lastText(result));
}

You pass middleware as an array. The hooks fire on every loop.

Lesson 3: wrap the model call

wrapModelCall is different. You get the request and a handler.

  • Call handler(request) to run the model.
  • Do work before and after that call.

This is where timing, retries, and caching go.

   wrapModelCall(request, handler)

        run your code         (before)
             │
             ▼
        handler(request)      ← runs the real MODEL
             │
             ▼
        run your code         (after)
const timingAndRetryMiddleware = createMiddleware({
  name: "TimingAndRetry",

  wrapModelCall: async (request, handler) => {
    const maxRetries = 3;

    for (let attempt = 1; attempt <= maxRetries; attempt++) {
      const start = Date.now();
      try {
        const response = await handler(request); // the real model call
        console.log(
          `⏱️  Model call took ${Date.now() - start}ms (attempt ${attempt})`,
        );
        return response;
      } catch (err) {
        console.log(`⚠️  Attempt ${attempt} failed: ${err.message}`);
        if (attempt === maxRetries) throw err;
      }
    }
  },
});

async function lesson3() {
  title("LESSON 3: wrapModelCall (timing + retry)");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [timingAndRetryMiddleware],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "Weather in Berlin?" }],
  });
  console.log("Answer:", lastText(result));
}

If the model throws, we try again. After three tries we give up and re-throw.

Lesson 4: wrap the tool call

wrapToolCall is the same shape, but for tools.

A tool can fail. If you let it throw, the agent crashes. Catch it instead and hand the model a normal message. Then the model can recover.

const toolErrorMiddleware = createMiddleware({
  name: "ToolErrorHandler",

  wrapToolCall: async (request, handler) => {
    const { name, args, id } = request.toolCall;
    console.log(`🔧 Calling tool "${name}" with`, args);

    try {
      return await handler(request); // the real tool call
    } catch (err) {
      console.log(`💥 Tool "${name}" failed: ${err.message}`);
      // Return a ToolMessage so the model keeps going.
      return new ToolMessage({
        content: `Tool error: ${err.message}. Tell the user to try later.`,
        tool_call_id: id,
      });
    }
  },
});

async function lesson4() {
  title("LESSON 4: wrapToolCall (handling tool errors)");

  const agent = createAgent({
    model: MODEL,
    tools: [flakyTool],
    middleware: [toolErrorMiddleware],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "What's the price of AAPL?" }],
  });
  console.log("Answer:", lastText(result));
}

flakyTool always throws. The agent still answers, politely.

Lesson 5: custom state and jumpTo

Two new ideas here.

  • stateSchema adds your own field to the state. Here it’s a counter.
  • jumpTo: "end" leaves the loop early.

This middleware caps the agent at two model calls.

   beforeModel checks the counter
            │
      reached the limit?
        │          │
       no         yes
        │          │
        ▼          ▼
     MODEL      jumpTo: "end"  ─►  skip the model, go to the end
     runs
const callLimitMiddleware = createMiddleware({
  name: "CallLimit",

  stateSchema: z.object({
    modelCallCount: z.number().default(0),
  }),

  beforeModel: {
    canJumpTo: ["end"], // you must declare where you may jump
    hook: (state) => {
      const LIMIT = 2;
      if (state.modelCallCount >= LIMIT) {
        console.log(`🛑 Limit of ${LIMIT} model calls reached, stopping.`);
        return {
          messages: [new AIMessage("Sorry, I hit my call limit.")],
          jumpTo: "end",
        };
      }
      // Returning an object updates the state.
      return { modelCallCount: state.modelCallCount + 1 };
    },
  },
});

async function lesson5() {
  title("LESSON 5: Custom state + jumpTo");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [callLimitMiddleware],
  });

  const result = await agent.invoke({
    messages: [
      {
        role: "user",
        content: "Weather in Paris, then London, then Rome, one at a time.",
      },
    ],
  });
  console.log("Model calls used:", result.modelCallCount);
  console.log("Answer:", lastText(result));
}

Notice the hook is now { canJumpTo, hook }, not a plain function. You need that form when you want to jump.

Lesson 6: runtime context and a dynamic model

contextSchema is read-only data you pass in per request. Think user id, plan, or tier.

Inside wrapModelCall you can read that context and pick the model. Free users get a cheap one. Pro users get a smart one.

const cheapModel = new ChatOpenRouter({
  model: "openai/gpt-4o-mini",
  maxTokens: 300,
});
const smartModel = new ChatOpenRouter({
  model: "openai/gpt-4o",
  maxTokens: 300,
});

const dynamicModelMiddleware = createMiddleware({
  name: "DynamicModel",

  contextSchema: z.object({
    userTier: z.enum(["free", "pro"]),
  }),

  wrapModelCall: async (request, handler) => {
    const tier = request.runtime.context.userTier;
    const isPro = tier === "pro";
    const model = isPro ? smartModel : cheapModel;
    const modelName = isPro ? "openai/gpt-4o" : "openai/gpt-4o-mini";
    console.log(`🎛️  User tier "${tier}", using ${modelName}`);

    return handler({
      ...request,
      model,
      systemPrompt: isPro
        ? "You are a detailed, premium assistant."
        : "You are a brief assistant. Answer in one sentence.",
    });
  },
});

async function lesson6() {
  title("LESSON 6: Context + dynamic model/prompt");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [dynamicModelMiddleware],
  });

  for (const userTier of ["free", "pro"]) {
    const result = await agent.invoke(
      { messages: [{ role: "user", content: "Weather in Madrid?" }] },
      { context: { userTier } }, // runtime context
    );
    console.log(`[${userTier}] Answer:`, lastText(result));
  }
}

You pass the context as the second argument to invoke. Because this is OpenRouter, you could even point smartModel at anthropic/claude-3.5-sonnet and switch providers the same way.

Lesson 7: order matters

You can use many middleware at once. The order is a rule, not a guess.

  • before* hooks run in order: A → B → C.
  • after* hooks run in reverse: C → B → A.
  • wrap* hooks nest like layers: A wraps B wraps C wraps the model.

As a picture:

   middleware: [A, B, C]

   before hooks run top-down:    A → B → C → MODEL
   after  hooks run bottom-up:   MODEL → C → B → A

   wrap hooks nest like layers:

        enter A
          enter B
            enter C
              MODEL runs
            exit C
          exit B
        exit A

Here are three tracers so you can watch it happen.

const makeTracer = (label) =>
  createMiddleware({
    name: `Tracer${label}`,
    beforeModel: () => console.log(`  before ${label}`),
    afterModel: () => console.log(`  after  ${label}`),
    wrapModelCall: async (req, handler) => {
      console.log(`  wrap ${label} ▶ enter`);
      const res = await handler(req);
      console.log(`  wrap ${label} ◀ exit`);
      return res;
    },
  });

async function lesson7() {
  title("LESSON 7: Middleware execution order");

  const agent = createAgent({
    model: MODEL,
    tools: [],
    middleware: [makeTracer("A"), makeTracer("B"), makeTracer("C")],
  });

  await agent.invoke({ messages: [{ role: "user", content: "Say hi" }] });
}

Run it and read the output. The first middleware in the array is the outermost layer.

The whole thing: one file

Everything above in one file. Save it as tutorial.mjs. Paste your key at the top.

// tutorial.mjs
import { ChatOpenRouter } from "@langchain/openrouter";
import {
  AIMessage,
  createAgent,
  createMiddleware,
  HumanMessage,
  tool,
  ToolMessage,
} from "langchain";
import * as z from "zod";

// Paste your key here. Fine for a script on your own machine.
process.env.OPENROUTER_API_KEY = "sk-or-v1-paste-your-key-here";

// ---------------------------------------------------------------------
//  Shared: model + tools + helpers
// ---------------------------------------------------------------------
const MODEL = new ChatOpenRouter({
  model: "openai/gpt-4o-mini",
  temperature: 0,
  maxTokens: 300,
});

const getWeather = tool(async ({ city }) => `It's 22°C and sunny in ${city}.`, {
  name: "get_weather",
  description: "Get the weather for a city",
  schema: z.object({ city: z.string() }),
});

const flakyTool = tool(
  async () => {
    throw new Error("Database is down!");
  },
  {
    name: "get_stock_price",
    description: "Get a stock price",
    schema: z.object({ ticker: z.string() }),
  },
);

const lastText = (result) => result.messages.at(-1).content;
const title = (t) =>
  console.log(`\n${"=".repeat(60)}\n ${t}\n${"=".repeat(60)}`);

// ---------------------------------------------------------------------
//  LESSON 1: Plain agent, no middleware
// ---------------------------------------------------------------------
async function lesson1() {
  title("LESSON 1: Plain agent (no middleware)");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "Weather in Paris?" }],
  });

  console.log("Answer:", lastText(result));
}

// ---------------------------------------------------------------------
//  LESSON 2: Node-style hooks
// ---------------------------------------------------------------------
const loggingMiddleware = createMiddleware({
  name: "LoggingMiddleware",

  beforeAgent: () => {
    console.log("🚀 [beforeAgent] Agent starting");
  },

  beforeModel: (state) => {
    console.log(
      `➡️  [beforeModel] Sending ${state.messages.length} messages to LLM`,
    );
    return { messages: [new HumanMessage("Answer in one sentence.")] };
  },

  afterModel: (state) => {
    const last = state.messages.at(-1);
    const tools = last.tool_calls?.map((t) => t.name) ?? [];
    console.log(
      `⬅️  [afterModel] LLM replied. Tool calls: [${tools.join(", ")}]`,
    );
  },

  afterAgent: () => {
    console.log("🏁 [afterAgent] Agent finished");
  },
});

async function lesson2() {
  title("LESSON 2: Logging with node-style hooks");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [loggingMiddleware],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "Weather in Tokyo?" }],
  });
  console.log("Answer:", lastText(result));
}

// ---------------------------------------------------------------------
//  LESSON 3: wrapModelCall (timing + retry)
// ---------------------------------------------------------------------
const timingAndRetryMiddleware = createMiddleware({
  name: "TimingAndRetry",

  wrapModelCall: async (request, handler) => {
    const maxRetries = 3;

    for (let attempt = 1; attempt <= maxRetries; attempt++) {
      const start = Date.now();
      try {
        const response = await handler(request);
        console.log(
          `⏱️  Model call took ${Date.now() - start}ms (attempt ${attempt})`,
        );
        return response;
      } catch (err) {
        console.log(`⚠️  Attempt ${attempt} failed: ${err.message}`);
        if (attempt === maxRetries) throw err;
      }
    }
  },
});

async function lesson3() {
  title("LESSON 3: wrapModelCall (timing + retry)");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [timingAndRetryMiddleware],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "Weather in Berlin?" }],
  });
  console.log("Answer:", lastText(result));
}

// ---------------------------------------------------------------------
//  LESSON 4: wrapToolCall (handling tool errors)
// ---------------------------------------------------------------------
const toolErrorMiddleware = createMiddleware({
  name: "ToolErrorHandler",

  wrapToolCall: async (request, handler) => {
    const { name, args, id } = request.toolCall;
    console.log(`🔧 Calling tool "${name}" with`, args);

    try {
      return await handler(request);
    } catch (err) {
      console.log(`💥 Tool "${name}" failed: ${err.message}`);
      return new ToolMessage({
        content: `Tool error: ${err.message}. Tell the user to try later.`,
        tool_call_id: id,
      });
    }
  },
});

async function lesson4() {
  title("LESSON 4: wrapToolCall (handling tool errors)");

  const agent = createAgent({
    model: MODEL,
    tools: [flakyTool],
    middleware: [toolErrorMiddleware],
  });

  const result = await agent.invoke({
    messages: [{ role: "user", content: "What's the price of AAPL?" }],
  });
  console.log("Answer:", lastText(result));
}

// ---------------------------------------------------------------------
//  LESSON 5: Custom state + jumpTo
// ---------------------------------------------------------------------
const callLimitMiddleware = createMiddleware({
  name: "CallLimit",

  stateSchema: z.object({
    modelCallCount: z.number().default(0),
  }),

  beforeModel: {
    canJumpTo: ["end"],
    hook: (state) => {
      const LIMIT = 2;
      if (state.modelCallCount >= LIMIT) {
        console.log(`🛑 Limit of ${LIMIT} model calls reached, stopping.`);
        return {
          messages: [new AIMessage("Sorry, I hit my call limit.")],
          jumpTo: "end",
        };
      }
      return { modelCallCount: state.modelCallCount + 1 };
    },
  },
});

async function lesson5() {
  title("LESSON 5: Custom state + jumpTo");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [callLimitMiddleware],
  });

  const result = await agent.invoke({
    messages: [
      {
        role: "user",
        content: "Weather in Paris, then London, then Rome, one at a time.",
      },
    ],
  });
  console.log("Model calls used:", result.modelCallCount);
  console.log("Answer:", lastText(result));
}

// ---------------------------------------------------------------------
//  LESSON 6: Context + dynamic model/prompt
// ---------------------------------------------------------------------
const cheapModel = new ChatOpenRouter({
  model: "openai/gpt-4o-mini",
  maxTokens: 300,
});
const smartModel = new ChatOpenRouter({
  model: "openai/gpt-4o",
  maxTokens: 300,
});

const dynamicModelMiddleware = createMiddleware({
  name: "DynamicModel",

  contextSchema: z.object({
    userTier: z.enum(["free", "pro"]),
  }),

  wrapModelCall: async (request, handler) => {
    const tier = request.runtime.context.userTier;
    const isPro = tier === "pro";
    const model = isPro ? smartModel : cheapModel;
    const modelName = isPro ? "openai/gpt-4o" : "openai/gpt-4o-mini";
    console.log(`🎛️  User tier "${tier}", using ${modelName}`);

    return handler({
      ...request,
      model,
      systemPrompt: isPro
        ? "You are a detailed, premium assistant."
        : "You are a brief assistant. Answer in one sentence.",
    });
  },
});

async function lesson6() {
  title("LESSON 6: Context + dynamic model/prompt");

  const agent = createAgent({
    model: MODEL,
    tools: [getWeather],
    middleware: [dynamicModelMiddleware],
  });

  for (const userTier of ["free", "pro"]) {
    const result = await agent.invoke(
      { messages: [{ role: "user", content: "Weather in Madrid?" }] },
      { context: { userTier } },
    );
    console.log(`[${userTier}] Answer:`, lastText(result));
  }
}

// ---------------------------------------------------------------------
//  LESSON 7: Middleware execution order
// ---------------------------------------------------------------------
const makeTracer = (label) =>
  createMiddleware({
    name: `Tracer${label}`,
    beforeModel: () => console.log(`  before ${label}`),
    afterModel: () => console.log(`  after  ${label}`),
    wrapModelCall: async (req, handler) => {
      console.log(`  wrap ${label} ▶ enter`);
      const res = await handler(req);
      console.log(`  wrap ${label} ◀ exit`);
      return res;
    },
  });

async function lesson7() {
  title("LESSON 7: Middleware execution order");

  const agent = createAgent({
    model: MODEL,
    tools: [],
    middleware: [makeTracer("A"), makeTracer("B"), makeTracer("C")],
  });

  await agent.invoke({ messages: [{ role: "user", content: "Say hi" }] });
}

// ---------------------------------------------------------------------
//  Runner
// ---------------------------------------------------------------------
const lessons = {
  1: lesson1,
  2: lesson2,
  3: lesson3,
  4: lesson4,
  5: lesson5,
  6: lesson6,
  7: lesson7,
};

const arg = process.argv[2] ?? "1";
if (arg === "all") {
  for (const fn of Object.values(lessons)) await fn();
} else if (lessons[arg]) {
  await lessons[arg]();
} else {
  console.log("Usage: node tutorial.mjs <1-7|all>");
}

Running it

Pick a lesson by number. Or run them all.

node tutorial.mjs 1      # one lesson
node tutorial.mjs all    # every lesson

Lesson 2 prints the hook log around the answer. Lesson 4 fails a tool and still answers. Lesson 7 shows the order, plain as day.

About that hardcoded key

A key pasted into a script on your own machine is fine. A key pasted into a script that ends up in a git repo is not. Bots scan public GitHub for keys.

Before you commit, drop the line that sets the key, and run like this:

node --env-file=.env tutorial.mjs 1

Put OPENROUTER_API_KEY=sk-or-v1-... in a .env file. Add .env to .gitignore. ChatOpenRouter reads the key from the environment on its own.

Where to go from here

  • Stack real middleware. Logging on the outside, retries inside, a call limit in front.
  • Use beforeModel to trim old messages when the history gets long.
  • Use wrapToolCall to cache slow tools, not just catch errors.
  • Use contextSchema to pass a user id, then log per user.
  • Write your own hook. If you can log it, you can change it.

That’s the whole idea. Your code, running around the agent’s loop, one hook at a time.

Share:

Keep reading