AI & ML
Agent memory that commits the write before it tries to embed
Christ-loisele Atidegla DEV Community
2 views
I build on a connection that drops for hours at a time. That exposed an assumption nearly every agent memory library makes without stating it.
In most of them, remember() calls an embedding API, so the write fails when the connection does.
Reversing the order
It is an ordering decision, made once, that everything else follows from.
// The row lands first, unconditionally.
this.db.prepare(`INSERT INTO memories (...) VALUES (...)`).run(...);
// Then we try. This is allowed to fail.
await this.#tryEmbed([{ id: memoryId, content }]);
The write is durable before anything touches the network. If the embedder is unreachable, unconfigured or slow, the memory is still there. It gets its vector later, whenever there is a link.
Recall has to degrade, not break
Committing the write is half of it. If recall then requires vectors, the failure has moved rather than gone.
So retrieval runs whatever it can:
// Lexical always runs. It needs nothing.
if (mode !== 'lexical') {
const vectors = await this.embedder.embed([query]);
if (vectors && vectors[0]) {
// Semantic joins in when it can.
}
}
With embeddings present you get semantic search fused with full text search. With none, you get full text search alone. That is worse, and it is the same API returning the same shape.
So your code has no if (hasVectors) in it. That property matters more than the retrieval quality, because a fallback you handle explicitly is a second code path, and second code paths drift from the first.
Backfill is a first class operation
Memories written offline need to catch up, so that is a named operation:
const { embedded, remaining } = await memory.backfill();
It embeds everything without a vector for the current model, in batches, and reports what is left. memory.pending() tells you how many are waiting.
Two details worth stealing.
Backfill checks availability first and returns a reason instead of throwing. Being offline is the expected state, so { embedded: 0, remaining: 42, reason: 'no embedder' } is a normal answer.
Changing a memory's content deletes its vector immediately:
this.db.prepare('DELETE FROM embeddings WHERE memory_id = ?').run(memoryId);
An embedding describes text. If the text changed, the vector describes something that no longer exists, and nothing will ever tell you it is wrong. Deleting it puts the memory back in the backfill queue.
What this is not
Local search on npm is not an unfilled gap. Orama is a complete in-process search engine and RAG pipeline. The sqlite-vec bindings put vector search inside SQLite directly. Both are good and both are more capable than this at what they are for.
The difference is what they assume about the embedding. They treat it as present, or as somebody else's problem. This one is built for the case where it is neither.
With reliable connectivity you probably do not need this. If you have had a write fail because a model was unreachable, you know what it is for.
npm install hinterland
Zero dependencies, one SQLite file, Node 22.5+. hinterland.
The storage side has its own article, because node:sqlite removes almost all of the usual setup.
Read original: https://dev.to/catidegla/agent-memory-that-commits-the-write-before-it-tries-to-embed-nck
← Previous
One webhook for WhatsApp, Instagram and Messenger — the payload shape that lets you write a single parser
Next →
Why I Built Another yt-dlp GUI (And Why I Think It Was Worth It)
Related
One webhook for WhatsApp, Instagram and Messenger — the payload shape that lets you write a single parser
AI & ML
2
DEV Community
Meta Muse Is Here: What Meta Actually Shipped
AI & ML
2
DEV Community
Passkeys for Non-Technical Users
AI & ML
1
DEV Community
If I Had to Build an AI Customer-Support System Today, I Wouldn't Start With an Agent
AI & ML
0
Dev.to (EN Zone)
Comments0
No comments yet — be the first