RAG with LangChain, pgvector and Node.js

In the last RAG post I wrote everything by hand. The splitter, the SQL, the embedding calls, the prompt. That was a good way to learn what RAG is. It’s not code I’d want to keep growing in a real project though. Before that, in the Qdrant post, I used LangChain but added a second database for it.

This post puts the two together. Same notes, same Postgres table, same questions. LangChain does the splitting, the storing, the searching and the prompt. Postgres with pgvector stays the only database. It’s still one TypeScript file, about the same length as the hand-written one. The parts that were easy to get wrong are gone, and it does a bit more: it can print the distances, and it can search inside one note.

I’ll go through each LangChain piece on its own first, with the options I found useful. Then the whole file. If you didn’t read the last post, that’s fine. The short version of what RAG is comes next.

What RAG is, in short

Retrieval-augmented generation is three steps.

1. Find the few paragraphs of your own notes that relate to the question.
2. Put them into the prompt.
3. Ask the model to answer using only them.

Steps 2 and 3 are a prompt and one API call. Step 1 is where the database comes in.

To find paragraphs by meaning you use embeddings. An embedding is a list of numbers that stands for the meaning of a piece of text. Text with similar meaning gets similar numbers. So you embed every paragraph once and store the numbers. Later you embed the question and look for the closest stored numbers. pgvector is the part of Postgres that stores those lists and sorts rows by how close they are. The last post explains the distance maths if you want it.

What LangChain adds

LangChain gives every piece a standard shape, so you can swap one piece without touching the rest. These are the shapes in this post:

• A Document is some pageContent plus a metadata object. Chunks are documents. Search results are documents.
• An embeddings class turns text into vectors. OpenAIEmbeddings here.
• A vector store saves documents with their vectors and searches them. PGVectorStore here. It writes the SQL I wrote by hand last time.
• A retriever is a vector store wrapped up so it takes a question and returns documents.
• A prompt template is a prompt with holes in it.
• A chat model writes the answer. ChatOpenRouter here.
• .pipe() joins pieces into a chain that you invoke or stream as one thing.

Everything below is one of those seven things.

The three pieces

Postgres with pgvector. Same as last time. pgvector adds a vector column type and a few distance operators to Postgres. Your embeddings live in a normal table next to the text they came from. If your app already has Postgres, it’s one CREATE EXTENSION away, and LangChain runs that line for you.

OpenRouter. One key for both models: an embedding model and a chat model. LangChain has no OpenRouter embeddings class, so I point OpenAIEmbeddings at OpenRouter’s URL, the same trick as in the Qdrant post. For the chat model there is a proper ChatOpenRouter class.

LangChain. Five small packages. The one that matters most here is @langchain/pgvector. It’s fairly new. It replaces the pgvector store that used to live in @langchain/community, and the docs now say to use it for new projects.

What you need

• Node.js 22.18 or newer. Newer Node runs .ts files directly by stripping the types out, so there’s no build step and no tsconfig.json. Check with node --version.
• Docker, to run Postgres locally.
• An OpenRouter API key from openrouter.ai, starting with sk-or-v1-. Indexing the notes in this post costs a fraction of a cent.
• Five npm packages. Step 2.

Step 1: Start Postgres with pgvector

The pgvector project publishes a Postgres image with the extension already compiled in:

docker run --name rag-postgres -p 5432:5432 \
  -e POSTGRES_PASSWORD=postgres \
  -v "$(pwd)/pgdata:/var/lib/postgresql/data" \
  pgvector/pgvector:pg17

Leave that running in its own terminal. The -v keeps the data in a pgdata folder so it survives a restart. To look around later:

docker exec -it rag-postgres psql -U postgres

If you still have the container from the last post, you can reuse it. The script makes its own table.

Step 2: Set up the project

mkdir notes-rag-langchain && cd notes-rag-langchain
npm init -y
npm pkg set type=module
npm install @langchain/pgvector @langchain/openai @langchain/openrouter @langchain/textsplitters @langchain/core
npm install -D @types/node

What each package is for:

• @langchain/pgvector is the vector store. It pulls in the pg driver itself, so you don’t install that.
• @langchain/openai is for OpenAIEmbeddings.
• @langchain/openrouter is for ChatOpenRouter.
• @langchain/textsplitters cuts text into chunks.
• @langchain/core is the shared base. This time we do import from it: the prompt template, the output parser and the Document type live there.

@types/node is only there so your editor knows about process. Node strips the types and doesn’t check them, same as last time. Run npx tsc --noEmit if you want the check to be real.

The pieces

Here’s each part on its own before the whole file.

Turn text into numbers

import { OpenAIEmbeddings } from "@langchain/openai";

const embeddings = new OpenAIEmbeddings({
  model: "openai/text-embedding-3-small",
  apiKey: "sk-or-v1-your-key-here",
  configuration: { baseURL: "https://openrouter.ai/api/v1" },
});

const vector = await embeddings.embedQuery("When do I replace the chain?");
console.log(vector.length); // 1536

configuration.baseURL is what sends this to OpenRouter instead of OpenAI. The model ID has the openai/ prefix because that’s how OpenRouter names it.

There are two methods. embedQuery takes one string. embedDocuments takes an array and sends it in batches of batchSize, which is 512 by default. You won’t call either one yourself in this post. The vector store calls embedDocuments when you add chunks and embedQuery when you search. That is the point of handing it the object. It can’t use a different model for the two sides, which was the easiest mistake to make in the hand-written version.

One option worth knowing is dimensions. The text-embedding-3 models can return fewer numbers if you ask, for example dimensions: 512. Smaller vectors mean a smaller table and a faster index. If you set it, the table column has to match. More on that next.

Connect and make the table

import { PGVectorStore } from "@langchain/pgvector";

const vectorStore = await PGVectorStore.initialize(embeddings, {
  postgresConnectionOptions: {
    connectionString: "postgres://postgres:postgres@localhost:5432/postgres",
  },
  tableName: "chunks",
  columns: {
    idColumnName: "id",
    contentColumnName: "content",
    metadataColumnName: "metadata",
    vectorColumnName: "embedding",
  },
  distanceStrategy: "cosine",
  dimensions: 1536,
});

initialize opens a connection pool, runs CREATE EXTENSION IF NOT EXISTS vector, and creates the table if it’s missing. This is the table it makes:

CREATE TABLE IF NOT EXISTS chunks (
  "id" uuid NOT NULL DEFAULT gen_random_uuid() PRIMARY KEY,
  "content" text,
  "metadata" jsonb,
  "embedding" vector(1536)
);

Almost the same as the table from the last post. Two differences. The id is a uuid instead of a bigserial. And source isn’t its own column any more. It goes inside metadata, which is jsonb, along with anything else you attach to a chunk. That’s what makes filtering work later.

The options, one at a time.

postgresConnectionOptions goes straight to the pg Pool, so a connection string is enough. If your app already has a pool, pass it as pool instead and the store shares it.

tableName is the table. Add schemaName if it shouldn’t be in public.

columns renames the four columns. Leave it out and you get id, content, metadata and vector. I set them so the table looks like the one from last time.

distanceStrategy picks the operator the search query uses:

For text embeddings, cosine is the one you want. Same advice as last time.

dimensions sets the size of the vector column. If you leave it out the column is plain vector with no size. Searching still works, but you can’t put an HNSW index on it later. Set it, and keep it equal to what the embedding model returns.

When you’re done, vectorStore.end() closes the pool. Without it the process hangs at the end.

Cut the text into chunks

import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 300,
  chunkOverlap: 50,
});

const chunks = await splitter.createDocuments([note.text], [{ source: "bike.md" }]);
console.log(chunks.length); // 2
console.log(chunks[0]);
// Document {
//   pageContent: "# Bike\n\nThe chain is a KMC X11. Replace it once ...",
//   metadata: { source: "bike.md", loc: { lines: { from: 1, to: 5 } } },
// }

This replaces the twelve-line splitter from last time. “Recursive” means it tries to split on "\n\n" first, then "\n", then spaces, then single characters. It only moves to the next separator when a piece is still too long. Then it packs the pieces back together up to chunkSize. So paragraphs stay whole when they fit, and a paragraph that’s too long gets cut at a line break or a space instead of staying oversized. That was the one limitation of my hand-written version, and it’s handled here.

createDocuments takes an array of texts and an array of metadata objects, one per text. Every chunk from a text gets a copy of its metadata. It also adds loc, the line range the chunk came from. That’s free, and it ends up in the metadata column, so you can see later which lines of which file an answer came from.

On these notes I got 2 chunks per note, 4 in total. My hand-written splitter made 6 from the same text. If I’ve read the code right, the LangChain one only carries a previous piece over as overlap when that piece is 50 characters or shorter. Paragraphs are longer than that, so here there’s no overlap at all, and the chunks pack tighter. My version carried the last 50 characters over no matter what.

Two options I didn’t use. separators lets you change the list it tries. RecursiveCharacterTextSplitter.fromLanguage("markdown", { chunkSize: 300, chunkOverlap: 50 }) gives you a list that splits on headings first, which is better for real Markdown files. And if you already have Document objects, from one of LangChain’s loaders for example, splitDocuments(documents) splits them and keeps their metadata.

Store the chunks

await vectorStore.addDocuments(chunks);

That embeds every chunk through embedDocuments, then inserts the rows. Up to 500 rows go in one INSERT. You can change that with chunkSize in the store config, which is a confusing name because it has nothing to do with text chunks.

You can pass your own ids:

await vectorStore.addDocuments(chunks, {
  ids: chunks.map(() => crypto.randomUUID()),
});

They have to be uuids, because that’s the column type. There’s no upsert though. Inserting an id that’s already there is an error, not an update. So for re-indexing I do it the other way round, delete first, then add:

await vectorStore.delete({ filter: { source: "bike.md" } }); // by metadata
await vectorStore.delete({ ids: ["9b1deb4d-..."] }); // or by id

Deleting by filter is what the index command uses below. Each note removes its own old chunks before adding new ones. That’s a bit better than the TRUNCATE from last time, because re-indexing one note doesn’t touch the others.

Search

const results = await vectorStore.similaritySearch("which bin goes out on Friday?", 3);
// Document[]

const scored = await vectorStore.similaritySearchWithScore("which bin goes out on Friday?", 3);
// [Document, number][]

Both embed the question with the embeddings object you gave the store and run this query:

SELECT *, "embedding" <=> $1 AS "_distance"
FROM chunks
ORDER BY "_distance" ASC
LIMIT $2;

That’s the query from the last post, with the operator chosen by distanceStrategy. The number you get from similaritySearchWithScore is that _distance. For cosine it’s between 0 and 2. 0 means the same meaning, around 1 means unrelated. Lower is better.

If you’d rather have higher is better, set scoreNormalization: "similarity" in the store config. Then cosine gives you (2 - distance) / 2, a number between 0 and 1. I left it on distance so the numbers match the last post.

Filter by metadata

Both search methods take a third argument, a filter on the metadata column:

await vectorStore.similaritySearch(question, 3, { source: "flat.md" });

That becomes WHERE metadata ->> 'source' = $3 in the query. Plain equality is the common case. There are a few operators as well:

You can mix them in one object and they’re joined with AND. Anything you put in a chunk’s metadata when you split can be filtered on later. It’s the same “filtering is just SQL” point from last time, only now LangChain writes the WHERE.

Turn the store into a retriever

const retriever = vectorStore.asRetriever({ k: 3, filter: { source: "flat.md" } });
const relevantChunks = await retriever.invoke("which bin goes out on Friday?");

A retriever is the store with the search settings baked in. k is how many chunks to return. filter is the same filter as above. invoke takes the question and gives back documents. That’s similaritySearch again, so why bother? Because a retriever is a Runnable, the same shape as prompts and models, so it can sit inside a chain. And because the settings live in one place.

The one retriever option that does something different is searchType: "mmr":

const retriever = vectorStore.asRetriever({
  k: 3,
  searchType: "mmr",
  searchKwargs: { fetchK: 10, lambda: 0.5 },
});

MMR stands for maximal marginal relevance. It fetches fetchK chunks, then picks k of them that are close to the question but not close to each other. lambda is between 0 and 1 and says how much to care about that, with 1 meaning plain similarity. It helps when your notes say the same thing in several places and you’d otherwise get three near copies in the prompt. For two short notes it makes no difference, so I left it out of the file.

Prompt and chain

import { ChatOpenRouter } from "@langchain/openrouter";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

const chatModel = new ChatOpenRouter({
  model: "openai/gpt-4o-mini",
  apiKey: "sk-or-v1-your-key-here",
});

const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "Answer the question using only the notes below. If they do not " +
      "contain the answer, say you don't know.\n\n{context}",
  ],
  ["human", "{question}"],
]);

const answerChain = prompt.pipe(chatModel).pipe(new StringOutputParser());

const stream = await answerChain.stream({ context, question });
for await (const token of stream) {
  process.stdout.write(token);
}

The prompt is the same two messages as last time, but as a template. {context} and {question} are the holes. You fill them by passing an object to the chain. Only the template text is parsed for braces, so a note that happens to contain { is fine as a value. If you need a literal brace in the template itself, double it: {{.

pipe joins the three steps. The prompt turns the object into messages. The model turns messages into an AIMessage. StringOutputParser pulls the text out, so the chain gives you plain strings. stream gives them a few tokens at a time. invoke would give you the whole answer at once. RunnableSequence.from([prompt, chatModel, parser]) from @langchain/core/runnables builds the same chain, if you prefer a list to a row of pipes.

ChatOpenRouter takes the usual model options too, like temperature: 0. If you’d rather not add the package, ChatOpenAI from @langchain/openai with the same configuration.baseURL trick works, and the chain doesn’t change.

I kept the retriever outside the chain on purpose. You can put it inside, and the LangChain docs show how. But then the retrieved documents are gone by the time the answer comes back, and I want them for the Sources line.

The whole thing: one file

Everything above in one file, including the notes themselves so there’s nothing else to create. Save it as rag.ts and paste your key in at the top.

// rag.ts
import { PGVectorStore } from "@langchain/pgvector";
import { OpenAIEmbeddings } from "@langchain/openai";
import { ChatOpenRouter } from "@langchain/openrouter";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";
import type { Document } from "@langchain/core/documents";

// Paste your key here. Fine for a script on your own machine.
const API_KEY = "sk-or-v1-paste-your-key-here";
const DATABASE_URL = "postgres://postgres:postgres@localhost:5432/postgres";
const EMBEDDING_MODEL = "openai/text-embedding-3-small";
const EMBEDDING_DIMENSIONS = 1536;
const CHAT_MODEL = "openai/gpt-4o-mini";
const CHUNK_SIZE = 300;
const CHUNK_OVERLAP = 50;
const CHUNKS_TO_RETRIEVE = 3;

// The knowledge base. A real one would read a folder off disk.
const notes = [
  {
    source: "bike.md",
    text: `# Bike

The chain is a KMC X11. Replace it once the wear gauge drops past 0.75,
which so far has been about every 3,000 km. A worn chain chews through the
cassette, so swapping it early is the cheap option.

Tyre pressure is 5.5 bar front, 6 bar rear. The track pump in the cellar
reads about 0.3 bar high, so aim slightly over.

The rear brake rubs after the wheel comes out. Loosen the two caliper
bolts, squeeze the lever, tighten the bolts while it is still held.`,
  },
  {
    source: "flat.md",
    text: `# Flat

The boiler resets with the black button behind the little hatch. Pressure
should sit between 1 and 1.5 bar. If it drops below 1, top it up with the
grey filling loop underneath and stop as soon as it reaches 1.2.

Bins: paper on the first Monday of the month, plastic every other Tuesday,
general waste every Friday. The code for the bin store is 4417.

The router lives in the hallway cupboard. If the internet drops, unplug it
for thirty seconds. The wifi password is on a sticker underneath.`,
  },
];

// Turns text into vectors. Pointed at OpenRouter instead of OpenAI.
const embeddings = new OpenAIEmbeddings({
  model: EMBEDDING_MODEL,
  apiKey: API_KEY,
  configuration: { baseURL: "https://openrouter.ai/api/v1" },
});

// Writes the final answer.
const chatModel = new ChatOpenRouter({ model: CHAT_MODEL, apiKey: API_KEY });

// Cuts long text into chunks, trying paragraph breaks first.
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: CHUNK_SIZE,
  chunkOverlap: CHUNK_OVERLAP,
});

// The prompt. {context} and {question} get filled in later.
const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "Answer the question using only the notes below. If they do not " +
      "contain the answer, say you don't know.\n\n{context}",
  ],
  ["human", "{question}"],
]);

async function connect(): Promise<PGVectorStore> {
  // Creates the extension and the table if they are not there yet.
  return PGVectorStore.initialize(embeddings, {
    postgresConnectionOptions: { connectionString: DATABASE_URL },
    tableName: "chunks",
    columns: {
      idColumnName: "id",
      contentColumnName: "content",
      metadataColumnName: "metadata",
      vectorColumnName: "embedding",
    },
    distanceStrategy: "cosine",
    dimensions: EMBEDDING_DIMENSIONS,
  });
}

async function indexNotes(vectorStore: PGVectorStore): Promise<void> {
  let storedChunks = 0;

  for (const note of notes) {
    // Remove this note's old chunks first, so re-running does not pile up duplicates.
    await vectorStore.delete({ filter: { source: note.source } });

    // One Document per chunk, each carrying { source } in its metadata.
    const chunks = await splitter.createDocuments(
      [note.text],
      [{ source: note.source }],
    );

    // Embeds every chunk in one request and inserts the rows.
    await vectorStore.addDocuments(chunks);
    storedChunks += chunks.length;
  }

  console.log(`Indexed ${notes.length} notes into ${storedChunks} chunks.`);
}

async function searchChunks(
  vectorStore: PGVectorStore,
  question: string,
  sourceFilter?: string,
): Promise<void> {
  const filter = sourceFilter ? { source: sourceFilter } : undefined;

  const matches = await vectorStore.similaritySearchWithScore(
    question,
    CHUNKS_TO_RETRIEVE,
    filter,
  );

  for (const [chunk, distance] of matches) {
    const preview = chunk.pageContent.replace(/\s+/g, " ").slice(0, 45);
    console.log(`${distance.toFixed(2)}  ${chunk.metadata.source}  ${preview}...`);
  }
}

function formatChunks(chunks: Document[]): string {
  return chunks
    .map((chunk) => `[${chunk.metadata.source}]\n${chunk.pageContent}`)
    .join("\n\n");
}

async function answerQuestion(
  vectorStore: PGVectorStore,
  question: string,
  sourceFilter?: string,
): Promise<void> {
  const retriever = vectorStore.asRetriever({
    k: CHUNKS_TO_RETRIEVE,
    filter: sourceFilter ? { source: sourceFilter } : undefined,
  });

  // Embeds the question with the same model and runs the distance query.
  const relevantChunks = await retriever.invoke(question);

  if (relevantChunks.length === 0) {
    console.error('No chunks in the database. Run "node rag.ts index" first.');
    return;
  }

  // prompt -> chat model -> plain string, streamed.
  const answerChain = prompt.pipe(chatModel).pipe(new StringOutputParser());

  const stream = await answerChain.stream({
    context: formatChunks(relevantChunks),
    question,
  });

  for await (const token of stream) {
    process.stdout.write(token);
  }

  const sources = [...new Set(relevantChunks.map((chunk) => chunk.metadata.source))];
  console.log(`\n\nSources: ${sources.join(", ")}`);
}

async function main(): Promise<void> {
  if (API_KEY === "sk-or-v1-paste-your-key-here") {
    console.error("Open rag.ts and paste your OpenRouter key into API_KEY.");
    process.exit(1);
  }

  const [command, question, sourceFilter] = process.argv.slice(2);
  const usage =
    'Usage: node rag.ts index | node rag.ts search "question" [source] | node rag.ts ask "question" [source]';

  if (command !== "index" && command !== "search" && command !== "ask") {
    console.error(usage);
    process.exit(1);
  }
  if (command !== "index" && !question) {
    console.error(usage);
    process.exit(1);
  }

  const vectorStore = await connect();
  try {
    if (command === "index") {
      await indexNotes(vectorStore);
    } else if (command === "search") {
      await searchChunks(vectorStore, question, sourceFilter);
    } else {
      await answerQuestion(vectorStore, question, sourceFilter);
    }
  } finally {
    await vectorStore.end(); // closes the connection pool so the process exits
  }
}

main().catch((error) => {
  console.error(`Failed: ${error.message}`);
  process.exit(1);
});

Running it

Index once, then search or ask as often as you like:

$ node rag.ts index
Indexed 2 notes into 4 chunks.

$ node rag.ts search "which bin goes out on Friday?"
0.29  flat.md  Bins: paper on the first Monday of the month,...
0.50  flat.md  # Flat The boiler resets with the black butto...
0.55  bike.md  Tyre pressure is 5.5 bar front, 6 bar rear. T...

$ node rag.ts ask "which bin goes out on Friday?"
General waste goes out every Friday. The bin store code is 4417.

Sources: flat.md

$ node rag.ts ask "what pressure should it be?"
The tyres should be at 5.5 bar front and 6 bar rear. The boiler should sit
between 1 and 1.5 bar.

Sources: bike.md, flat.md

$ node rag.ts ask "what pressure should it be?" bike.md
5.5 bar in the front tyre and 6 bar in the rear. The track pump reads about
0.3 bar high, so aim a little over.

Sources: bike.md

$ node rag.ts ask "what is the alarm code at the office?"
I don't know. The notes don't mention an office alarm code.

Sources: flat.md, bike.md

The pressure question is why I added the filter argument. Both notes talk about pressure, one for tyres and one for the boiler. Without a filter the model gets both and answers both. With bike.md as the third argument the WHERE clause keeps the boiler out.

The last one is the same lesson as last time. There is no office in the notes, so the search still returned its three closest chunks, because something is always closest. The instruction to say “I don’t know” is what stops the model from handing you the bin store code instead.

You can look at what got stored:

docker exec -it rag-postgres psql -U postgres \
  -c "SELECT id, metadata, left(content, 40) FROM chunks"

The metadata column has both source and the loc line numbers the splitter added.

What’s going on

main reads three arguments: a command, a question, and an optional source. index needs only the command. search prints the closest chunks with their distances, and is there so you can look at retrieval on its own. ask does the full RAG. Both search and ask take a filename as the third argument, which becomes the metadata filter. Every path calls connect first and end last, so the pool closes and the process exits.

indexNotes loops over the notes. For each one: delete its old chunks, split, add. Compared to last time there’s no embed call and no INSERT. The store does both inside addDocuments.

searchChunks is similaritySearchWithScore and a console.log. I print the distance first because that number tells you whether a bad answer came from bad retrieval or from the prompt, and those have different fixes.

answerQuestion builds a retriever with k and the optional filter, gets the chunks, joins them into one block with a [filename] label above each, and streams the chain. The Sources line is the metadata from the matched chunks with duplicates removed.

The complete flow

Start to finish, here’s what happens when you type a question, and which LangChain call does each step.

User question

"which bin goes out on Friday?"

A plain string from process.argv.

→ Embedding

retriever.invoke calls embeddings.embedQuery on that string and gets 1536 floats. It’s the same embeddings object that indexed the notes, so it’s the same model. You can’t get this wrong by accident any more.

→ pgvector similarity search

PGVectorStore runs the <=> query with LIMIT 3, plus a WHERE on the metadata column if you gave a filter. Postgres sorts by distance and returns the top three rows. No AI in this step.

→ Relevant documents

The rows come back as Document objects, with pageContent and metadata. formatChunks turns them into one block of text:

[flat.md]
Bins: paper on the first Monday of the month, plastic every other
Tuesday, general waste every Friday. The code for the bin store is 4417.

The router lives in the hallway cupboard...

→ LLM

The prompt template puts that block into {context} and the question into {question}. ChatOpenRouter sends the two messages. The model isn’t searching anything. It’s reading a prompt that was built for it.

→ Final answer

General waste goes out every Friday. The bin store code is 4417.

StringOutputParser turns the streamed message chunks into strings, and the loop prints them.

Same two points as last time. The model is the last step and the least interesting one, and swapping it is one line. And the answer is decided before the model sees anything, by how the text was chunked and whether the right chunk made it into the top three.

What LangChain did for you

Here’s the hand-written version next to this one:

What you give up: you don’t see the SQL, and when something is slow or odd you have to go read the package source. I did that for this post and it was fine, the store is one file. What you get: fewer places to make the “wrong model on one side” mistake, and every piece can be swapped on its own.

Making it faster

Same as last time. The search compares the question against every row. On a few thousand chunks that’s milliseconds and exact, so leave it alone.

When the table gets big enough to feel slow, the store can add the HNSW index for you:

await vectorStore.createHnswIndex({ dimensions: EMBEDDING_DIMENSIONS });

It runs CREATE INDEX IF NOT EXISTS ... USING hnsw on the vector column, and picks the operator class from distanceStrategy, so "cosine" gets vector_cosine_ops. That was the easy thing to mismatch by hand. Because of the IF NOT EXISTS, you can call it once at the end of indexNotes and forget about it. There are m and efConstruction options for tuning, defaulting to 16 and 64, which are pgvector’s defaults too.

The two things to remember from last time still hold. HNSW is approximate, so it can miss a true nearest neighbour now and then. And pgvector can only index vectors up to 2000 dimensions, which is why dimensions on the embeddings class is worth knowing about.

Picking an embedding model

Prices are from when I wrote this. The live list is at openrouter.ai/collections/embedding-models.

Two constants change together: EMBEDDING_MODEL and EMBEDDING_DIMENSIONS. Vectors from different models don’t mix, so switching means dropping the table and running index again:

docker exec -it rag-postgres psql -U postgres -c "DROP TABLE chunks"

About that hardcoded key

Same as always. A key pasted into a script on your own machine is fine. A key pasted into a script that ends up in a git repo is not. Before you commit, change one line:

const API_KEY = process.env.OPENROUTER_API_KEY!;

Put it in a .env file, add that to .gitignore, and run with:

node --env-file=.env rag.ts index

ChatOpenRouter would find OPENROUTER_API_KEY on its own, but the embeddings object still needs it passed in, so the one line covers both.

Where to go from here

• Read notes off disk instead of the array at the top. fs.readdir plus fs.readFile, and nothing else changes.
• Try one of LangChain’s document loaders for PDFs or web pages. They all give you Document objects, and splitter.splitDocuments takes those straight in.
• Switch the splitter to fromLanguage("markdown") once your notes have real headings.
• Turn on searchType: "mmr" and see if the answers change on a bigger set of notes.
• Pass collectionName in the store config if you want several projects in one table. The store adds a collections table and filters by it.
• If you need exact matches, like product codes, the hybrid search post does that in plain SQL. I couldn’t find a hybrid option in the JS store, so that part stays hand-written.
• The PGVectorStore docs list every config option, and the pgvector README is still worth reading end to end.

That’s it. One table, one file, one key, and a lot less SQL.