LangChain and Qdrant with OpenRouter in Node.js

In the last post I built a small chat app on OpenRouter. It worked, but the model only knew what it was trained on. It had no idea what was in my notes folder. This post fixes that. By the end you’ll have two scripts: one that reads a folder of text files into a database, and one that answers questions using them.

The technique has a name, retrieval-augmented generation, or RAG. It sounds fancier than it is. You find the few paragraphs that are relevant to the question, paste them into the prompt, and let the model read them before it answers. That’s the whole trick. The rest of this post is about the finding part.

The three pieces

OpenRouter you already know if you read the last post. One key, any model. What’s new here is that it also serves embedding models, so both halves of this project go through the same account and show up on the same bill.

Qdrant is a database built for vectors. You give it lists of numbers with some text attached. Later you hand it another list of numbers and ask for the closest matches. It’s open source and runs in Docker with one command.

LangChain is the glue. It gives you one way of talking to embedding models, chat models and vector databases, so “split this text, store it, search it” is a few lines instead of a few pages. It also ships a text splitter, which turns out to matter more than I expected.

What an embedding is

An embedding is a list of numbers that stands for the meaning of a piece of text. “How do I restart the Pi?” and “The Raspberry Pi is not responding” end up with similar numbers even though they share almost no words. So if you turn every paragraph of your notes into numbers once, you can find the relevant paragraphs for any question by comparing numbers instead of matching keywords. That comparison is what Qdrant is good at.

What you need

• Node.js 22 or newer. Check with node --version.
• Docker, to run Qdrant locally. Docker Desktop is fine. If you’d rather not, Qdrant has a hosted free tier. More on that at the end.
• An OpenRouter API key. Same key as last time, starts with sk-or-v1-. You’ll want a little credit on the account this time because embedding models aren’t free. It’s cheap though. Indexing my whole notes folder cost less than a cent.
• Five npm packages. Listed in step 2.

Step 1: Start Qdrant

docker run -p 6333:6333 -p 6334:6334 \
  -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
  qdrant/qdrant

Leave that running in its own terminal. The -v part saves the data to a qdrant_storage folder so it survives restarts. Open localhost:6333/dashboard and you get a small web UI where you can poke at your collections. It’s empty for now.

Step 2: Set up the project

mkdir notes-search && cd notes-search
npm init -y
npm pkg set type=module
npm install @langchain/openrouter @langchain/openai @langchain/qdrant @langchain/textsplitters @langchain/core

Five packages is more than last time, so here’s what each one is for:

• @langchain/openrouter has the chat model class, ChatOpenRouter. It already knows OpenRouter’s URL, so there’s nothing to configure.
• @langchain/openai is for OpenAIEmbeddings. There’s no OpenRouter embeddings class yet, so we point the OpenAI one at OpenRouter. Same trick as the last post.
• @langchain/qdrant talks to Qdrant. It pulls in the Qdrant client for you.
• @langchain/textsplitters cuts long text into chunks.
• @langchain/core is the shared base the others depend on. You won’t import from it directly here.

The pieces

Before the full scripts, here are the parts they’re made of. Each one is a few lines.

Turn text into numbers

import { OpenAIEmbeddings } from "@langchain/openai";

const embeddings = new OpenAIEmbeddings({
  model: "openai/text-embedding-3-small",
  apiKey: "sk-or-v1-your-key-here",
  configuration: { baseURL: "https://openrouter.ai/api/v1" },
});

const vector = await embeddings.embedQuery("How do I restart the Pi?");
console.log(vector.length); // 1536
console.log(vector.slice(0, 3)); // [ 0.0213, -0.0187, 0.0442 ]

The baseURL and apiKey are what make this hit OpenRouter instead of OpenAI. The model ID has the openai/ prefix because that’s OpenRouter’s naming. Run it and you get 1536 numbers back. You’ll never look at them directly, but it’s nice to see them once.

Split the notes into chunks

You can’t embed a whole file as one vector and expect good results. The meaning gets averaged into mush, and you’d end up pasting the whole file into the prompt anyway. So you cut it up first.

import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 500,
  chunkOverlap: 50,
});

const chunks = await splitter.createDocuments([noteText], [{ source: "homelab.md" }]);
console.log(chunks[0]);
// Document {
//   pageContent: "# Home server\n\nThe Raspberry Pi 4 in the hallway ...",
//   metadata: { source: "homelab.md", loc: { lines: { from: 1, to: 9 } } },
// }

“Recursive” means it tries to split on paragraph breaks first, then sentences, then words, so chunks end at natural places when they can. The chunkOverlap makes each chunk repeat the last 50 characters of the one before it, so a sentence that gets cut in half shows up whole in one of the two. The second argument is metadata that gets attached to every chunk from that text. I use it to remember which file a chunk came from.

Each chunk is a Document: some pageContent and a metadata object. Everything else in LangChain passes these around.

Store them in Qdrant

import { QdrantVectorStore } from "@langchain/qdrant";

const vectorStore = await QdrantVectorStore.fromDocuments(chunks, embeddings, {
  url: "http://localhost:6333",
  collectionName: "notes",
});

That one call does a lot. If the notes collection doesn’t exist yet it creates it, and it works out the vector size by embedding the word “test” and counting. Then it sends every chunk to OpenRouter to be embedded, and writes the vectors plus the text and metadata into Qdrant. Refresh the dashboard and the collection is there.

Find the relevant chunks

const results = await vectorStore.similaritySearch("What's the Pi-hole address?", 3);

for (const document of results) {
  console.log(document.metadata.source, "->", document.pageContent.slice(0, 60));
}

similaritySearch embeds the question with the same model, asks Qdrant for the three closest vectors, and gives you back the chunks as Document objects. There’s also similaritySearchWithScore if you want to see how close each match was.

Ask the model

import { ChatOpenRouter } from "@langchain/openrouter";

const chatModel = new ChatOpenRouter("openai/gpt-4o-mini", {
  apiKey: "sk-or-v1-your-key-here",
});

const reply = await chatModel.invoke([
  { role: "system", content: "Answer in one sentence." },
  { role: "user", content: "What is a vector database?" },
]);

console.log(reply.text);

Same three roles as last time. reply.text is the answer as a plain string. If you leave out apiKey, ChatOpenRouter reads the OPENROUTER_API_KEY environment variable instead.

Pick an embedding model

The chat model is the same choice as last time, so I won’t repeat the table. The embedding model is new. A few from OpenRouter’s list:

Prices are from when I wrote this. The live list is at openrouter.ai/collections/embedding-models.

One thing that bit me: vectors from different models don’t mix. Once a collection has been filled with one model, every search against it has to use that same model. If you want to switch, delete the collection and index again.

The whole thing: two scripts

Indexing happens once and asking happens many times, so this is two files. Each has the config at the top and each is complete on its own.

First, make a notes folder with a couple of text files in it. Here’s one of mine, notes/homelab.md, so the example output below makes sense:

# Home server

The Raspberry Pi 4 in the hallway cupboard runs Pi-hole. It's at
192.168.1.2 and the admin page is at http://192.168.1.2/admin. Log in
with the password in the password manager under "pihole".

Backups run every night at 02:00 with restic to the Backblaze bucket.
To check the last one, ssh in as pi@192.168.1.2 and run `restic snapshots`.
The repo password is in the password manager too.

If the Pi stops responding, hold the power button for five seconds and
wait a minute. It takes a while to boot from the SD card. If it still
doesn't come up, the SD card is probably done. There's a spare in the
drawer with the cables, flashed in March.

The router is a Fritz!Box at 192.168.1.1. Pi-hole is set as the only DNS
server, so if the Pi is down the whole house loses internet. Switch the
DNS back to the router in the Fritz!Box settings until it's fixed.

ingest.js

Reads every .md and .txt file in the folder, splits them, and stores them in Qdrant.

// ingest.js
import { readdir, readFile } from "node:fs/promises";
import path from "node:path";
import { OpenAIEmbeddings } from "@langchain/openai";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { QdrantVectorStore } from "@langchain/qdrant";

// Paste your key here. Fine for a local script you keep to yourself.
const API_KEY = "sk-or-v1-paste-your-key-here";
const EMBEDDING_MODEL = "openai/text-embedding-3-small";
const QDRANT_URL = "http://localhost:6333";
const COLLECTION_NAME = "notes";
const NOTES_FOLDER = "./notes";

if (API_KEY === "sk-or-v1-paste-your-key-here") {
  console.error("Open ingest.js and paste your OpenRouter key into API_KEY.");
  process.exit(1);
}

const embeddings = new OpenAIEmbeddings({
  model: EMBEDDING_MODEL,
  apiKey: API_KEY,
  configuration: { baseURL: "https://openrouter.ai/api/v1" },
});

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 500,
  chunkOverlap: 50,
});

async function main() {
  const fileNames = (await readdir(NOTES_FOLDER)).filter(
    (fileName) => fileName.endsWith(".md") || fileName.endsWith(".txt"),
  );

  if (fileNames.length === 0) {
    console.error(`No .md or .txt files found in ${NOTES_FOLDER}.`);
    process.exit(1);
  }

  // One text and one metadata object per file, in the same order.
  const noteTexts = [];
  const noteMetadatas = [];
  for (const fileName of fileNames) {
    noteTexts.push(await readFile(path.join(NOTES_FOLDER, fileName), "utf8"));
    noteMetadatas.push({ source: fileName });
  }

  const chunks = await splitter.createDocuments(noteTexts, noteMetadatas);
  console.log(`Read ${fileNames.length} files, split into ${chunks.length} chunks.`);

  // Embeds every chunk through OpenRouter and writes it all to Qdrant.
  await QdrantVectorStore.fromDocuments(chunks, embeddings, {
    url: QDRANT_URL,
    collectionName: COLLECTION_NAME,
  });

  console.log(`Stored in Qdrant collection "${COLLECTION_NAME}".`);
}

main().catch((error) => {
  console.error(`Failed: ${error.message}`);
  process.exit(1);
});

ask.js

Takes a question on the command line, finds the closest chunks, and streams the answer.

// ask.js
import { stdout as output } from "node:process";
import { OpenAIEmbeddings } from "@langchain/openai";
import { QdrantVectorStore } from "@langchain/qdrant";
import { ChatOpenRouter } from "@langchain/openrouter";

// Same key and settings as ingest.js. The embedding model must match.
const API_KEY = "sk-or-v1-paste-your-key-here";
const EMBEDDING_MODEL = "openai/text-embedding-3-small";
const CHAT_MODEL = "openai/gpt-4o-mini";
const QDRANT_URL = "http://localhost:6333";
const COLLECTION_NAME = "notes";

if (API_KEY === "sk-or-v1-paste-your-key-here") {
  console.error("Open ask.js and paste your OpenRouter key into API_KEY.");
  process.exit(1);
}

const question = process.argv.slice(2).join(" ").trim();
if (question === "") {
  console.error('Usage: node ask.js "your question here"');
  process.exit(1);
}

const embeddings = new OpenAIEmbeddings({
  model: EMBEDDING_MODEL,
  apiKey: API_KEY,
  configuration: { baseURL: "https://openrouter.ai/api/v1" },
});

const chatModel = new ChatOpenRouter(CHAT_MODEL, { apiKey: API_KEY });

async function main() {
  // Connect to the collection ingest.js filled.
  const vectorStore = await QdrantVectorStore.fromExistingCollection(embeddings, {
    url: QDRANT_URL,
    collectionName: COLLECTION_NAME,
  });

  // The two chunks closest in meaning to the question.
  const matchingChunks = await vectorStore.similaritySearch(question, 2);

  // Paste them into the prompt, each labelled with the file it came from.
  const context = matchingChunks
    .map((chunk) => `[${chunk.metadata.source}]\n${chunk.pageContent}`)
    .join("\n\n");

  const messages = [
    {
      role: "system",
      content:
        "Answer the question using only the notes below. " +
        "If the notes don't cover it, say you don't know.\n\n" +
        `Notes:\n${context}`,
    },
    { role: "user", content: question },
  ];

  // Print the answer as it streams in.
  const stream = await chatModel.stream(messages);
  for await (const chunk of stream) {
    output.write(chunk.text);
  }
  output.write("\n");

  const sources = [...new Set(matchingChunks.map((chunk) => chunk.metadata.source))];
  console.log(`\nSources: ${sources.join(", ")}`);
}

main().catch((error) => {
  console.error(`Failed: ${error.message}`);
  process.exit(1);
});

Running it

$ node ingest.js
Read 2 files, split into 3 chunks.
Stored in Qdrant collection "notes".

$ node ask.js "how do I check the last backup?"
SSH into the Pi as pi@192.168.1.2 and run `restic snapshots`. Backups go
to the Backblaze bucket every night at 02:00.

Sources: homelab.md

$ node ask.js "the internet is down, what do I do?"
Check whether the Pi at 192.168.1.2 is responding, since Pi-hole is the
only DNS server. If it's down, hold the power button for five seconds and
wait a minute for it to boot. Until it's back, switch the DNS in the
Fritz!Box settings to the router itself.

Sources: homelab.md

$ node ask.js "what grinder setting for espresso?"
I don't know. The notes only give settings for the V60 (14) and the moka
pot (8), and say not to go below 6.

Sources: coffee.md, homelab.md

The second answer pulled from two different chunks of the same file and put them together. The third one is my favourite. The closest chunk really was the coffee note, but it doesn’t answer the question. The model was told to only use the notes, so it said it didn’t know instead of inventing a number.

What’s going on

ingest.js is a straight line. Read the files, split them into chunks with the filename attached as metadata, hand the lot to fromDocuments. That last call is the only one that talks to the network, and it talks to both OpenRouter and Qdrant.

ask.js starts by connecting to the existing collection with fromExistingCollection. It’s the same class as before, just told not to add anything. If you run it before ingest.js, it quietly creates an empty collection and every search comes back empty, so run them in order.

similaritySearch does the embedding of the question for you, which is why ask.js needs the same embeddings object as ingest.js. The matches get joined into one block of text with a [filename] label above each, and that whole block goes into the system message along with the instruction to stick to the notes. The question itself is the user message.

The streaming loop is the same shape as the last post. chatModel.stream gives you chunks, and chunk.text is the bit of text in each one.

The Sources line at the end is just the metadata from the matched chunks, with duplicates removed. It’s a small thing, but being able to see which file an answer came from makes it much easier to trust, or to go and fix the note when the answer is wrong.

One gotcha with ingest.js: running it twice stores everything twice. Qdrant doesn’t know the chunks are the same ones. The quick fix while you’re experimenting is to drop the collection first:

curl -X DELETE http://localhost:6333/collections/notes

About that hardcoded key

Everything I said last time still applies. A key pasted into a script on your own machine is fine. A key pasted into a script that ends up on GitHub is not. If you want to move it out, change the two API_KEY lines to:

const API_KEY = process.env.OPENROUTER_API_KEY;

Put the key in a .env file, add it to .gitignore, and run with node --env-file=.env ingest.js. ChatOpenRouter would have found that variable on its own, but the embeddings object still needs it passed in, so the one line covers both.

Where to go from here

• Swap CHAT_MODEL for a Claude or Gemini ID and see if the answers get better. The retrieval part doesn’t change at all.
• Bump the 2 in similaritySearch up once you have more than a couple of notes. Four or five chunks is a comfortable amount of context.
• Use similaritySearchWithScore and print the scores. It shows you when the top match isn’t really a match, which is when the model is most likely to guess.
• Pass a filter to similaritySearch to search inside one file only. Qdrant filters on the metadata, so metadata.source is already there to filter on.
• Point QDRANT_URL at a Qdrant Cloud cluster and add apiKey next to url in the config. The free tier is plenty for a notes folder and you can skip Docker.
• Load PDFs or web pages instead of Markdown. LangChain has document loaders for most things, and they all give you the same Document objects the splitter takes.
• The LangChain docs for the Qdrant store and ChatOpenRouter cover the rest.

That’s it. Two scripts, one database, one key.