How to Add RAG to Hermes Agent with DigitalOcean Knowledge Bases

Most RAG tutorials never prove that retrieval happened. They stand up a pipeline, ask the model a question, get a correct answer, and declare victory — without checking whether the model could have answered that question from general knowledge alone. If it could, the demo demonstrates nothing.

This tutorial takes the opposite approach. We write the test questions first, verify that the model fails them with no retrieval attached (that failure transcript is our negative control), and only then build the pipeline and re-run the questions. By the end, you’ll have three things: a working retrieval setup, evidence that it works, and a validation method you can reuse on any RAG stack — not just this one.

One clarification before anything else, because it’s the most common misconception about Hermes: Hermes is not missing memory. It ships with agent-curated memory files, FTS5 full-text search over past sessions, cross-session recall, and skills. If you followed Part 1, you’ve already seen it remember things across conversations. What Hermes does not ship with is retrieval over an external corpus you own — your runbooks, your internal docs, your product knowledge. Memory is about the conversation. Retrieval is about your documents. They’re different problems, and this tutorial solves the second one.

The concrete use case we’ll build: an on-call agent that answers “what does error FM-4419 mean, and who do I page?” from your actual runbooks, rather than pattern-matching off the error’s shape.

What you’ll end with:

• A DigitalOcean Knowledge Base indexed from a Spaces bucket
• A ~60-line Hermes plugin that calls the Knowledge Base Retrieve API
• A validated retrieval loop where every answer cites its source files
• A negative-control transcript proving your test corpus was genuinely unguessable

Everything — the sample corpus, the plugin source, and the test questions — is in the companion repo, so you can follow along or skip ahead.

A scope note: this setup is about getting retrieval correct on a working corpus. Scaling to millions of documents — high-throughput embedding, retrieval latency under load — is a different architecture problem, and a different article.

Prerequisites

• A DigitalOcean account
• Hermes installed and connected to DigitalOcean inference (covered in Part 1)
• A DigitalOcean personal access token with the genai:read scope
• Spaces access keys
• doctl installed and authenticated
• Working Python familiarity — you’ll edit four short files

Versions used in this tutorial: Hermes Agent v0.20.4 (2026.8.18), model slugs deepseek-v4-pro-0813 and qwen3.8-max on DigitalOcean Serverless Inference. Hermes moves fast and DigitalOcean retires older model slugs, so if a command below fails, check the troubleshooting section before assuming the tutorial is broken.

Three things that sound like solutions, and why they aren’t:

The context window doesn’t solve it. You can’t paste every runbook into every turn. Even when the corpus technically fits, stuffing context degrades both cost and the model’s attention to the parts that matter.

Web search doesn’t solve it. Your runbooks aren’t on the web. If they are, you have a different problem.

Hermes’ own memory doesn’t solve it. Memory recalls your conversations. It will remember that you discussed FM-4419 last Tuesday. It will not know what FM-4419 means if nobody ever told it.

What you actually want is an agent that decides for itself when a question needs the corpus, fetches only the relevant chunks, and tells you exactly which documents those chunks came from. That’s the loop we’re building.

The full chain looks like this:

Spaces (source docs)
  → Knowledge Base (chunk + embed + index)
    → Retrieve API (kbaas.do-ai.run)  ← the Hermes plugin calls this
      → Hermes agent loop
        → DigitalOcean Serverless Inference (the model)

The correction worth making before you build anything: Serverless Inference does not know Knowledge Bases exist. The retrieval and the completion are two entirely separate API calls. If your mental model is “I’ll point my inference endpoint at my Knowledge Base,” you will hit a wall, because there is nothing to point. The plugin we build in this tutorial is the thing that joins them: it fetches chunks from the Retrieve API and places them into the agent’s context, and the agent’s next inference call reasons over them.

When to use this approach — and when not to

Use the Retrieve API + plugin approach (this tutorial) when:

• You want raw chunks back, under your agent’s control — not a second model’s completion wrapped inside your tool call
• You need retrieval parameters (alpha, result count, formatting) pinned deterministically in code
• You’re running your own agent loop and care about how many tokens each retrieval injects per turn

Use the managed alternative instead — attach the Knowledge Base to an agent and call its endpoint with include_retrieval_info — when:

• You want a single managed endpoint that returns retrieval-augmented completions
• You aren’t running your own agent loop and don’t need chunk-level control

Stop at the MCP server (Option A, below) when:

• You want zero code and default retrieval behavior is good enough. This path genuinely works, and some readers should take it.

Worth naming the prior question this tutorial skips past: managed Knowledge Base, or your own vector store? Running Qdrant, pgvector, or Pinecone yourself buys control over the index — your choice of embedding model, your own chunking and hybrid-search implementation, portability across clouds — in exchange for owning the ingestion pipeline and the database behind it, whereas a DigitalOcean Knowledge Base handles chunking, embedding, hybrid retrieval, and optional reranking, and provisions and sizes the backing OpenSearch database automatically if you don’t supply one. We take the managed path here because the goal is validating retrieval correctness rather than operating a vector database — but the plugin pattern below doesn’t care what’s on the other end of the HTTP call, so if you’re already running pgvector on DO Managed Postgres, swap the URL and keep everything else.

This is the section most RAG tutorials skip, and it’s the reason most RAG demos prove nothing.

The design principle: if a competent model can answer your test question from general knowledge, your test is worthless. Ask a frontier model “what does HTTP 503 mean and what should I check?” and it will give you a good answer with no retrieval at all. Your pipeline could be completely broken and the demo would still look perfect.

So the corpus has to be unguessable. The workflow, in order:

1. Write your test questions first. Before you create a single document.
2. Run those questions against the model with no retrieval attached, and save the transcript. The model should fail — refuse, hedge, or ask where the information lives. This failure transcript is your negative control. If the model answers correctly here, your questions are guessable and you need to rewrite them.
3. Only then build the corpus and the pipeline, and re-run the same questions.

This three-step method isn’t specific to Hermes or DigitalOcean. It works on any RAG stack, and it’s the difference between “my demo produced a plausible answer” and “I have evidence retrieval happened.”

The sample corpus

The companion repo contains eight markdown files documenting a fictional payments platform with four services: atlas-ingest, ledger-core, payouts-service, and webhook-relay. The corpus is deliberately seeded with things no model can know:

• Invented service names and custom error codes. FM-4419 means something specific in this corpus and nothing anywhere else.
• A named on-call rotation (Riverbend) with a defined scope — which services it covers and, importantly, which it doesn’t.
• Version-specific behavior changes. A changelog entry documents that a lease timeout changed from 30s to 90s, and why that changed the meaning of a sustained error burst.
• Deliberate cross-references, so at least one test question can only be answered by combining two or three documents.
• A deliberate scope trap. payouts-service errors appear throughout the corpus, but the escalation doc states explicitly that they are not the Riverbend rotation’s responsibility. This tests whether the model reads what it retrieved or free-associates off the error prefix.

We won’t paste all eight files here — grab them from the repo. The structure matters more than the content: when you build a corpus for your own validation, steal the pattern (unguessable specifics, cross-references, a scope trap), not the files.

An eight-document corpus is deliberately small. The goal here is proving correctness. Scaling ingestion and retrieval to millions of documents introduces different constraints and deserves its own treatment.

Upload the corpus to Spaces

Create a Spaces bucket and upload the corpus under a folder prefix. Something like runbooks/ works well.

The prefix isn’t cosmetic. A Knowledge Base stores each document’s file_id as {bucket}/{object_key}, so a consistent prefix (runbooks/) is what lets you scope retrieval to a subset of the bucket later using metadata filters. If you ever expect multiple teams or document types in one bucket, decide your prefixes now.

Create the Knowledge Base and start indexing

In the control panel: Data Services → Knowledge Bases → Create. Pick a region and an embedding model, add your Spaces bucket as a data source, and start indexing.

Two things to know before you click create:

Indexing is a billed job that consumes tokens. Every document gets chunked and embedded, and you pay for the embedding tokens. For eight markdown files this is trivial; for a real corpus, know what you’re indexing before you index it. It’s also not instant — expect the indexing job to take a few minutes even for a small corpus.

Chunking strategy is set per data source, and changing it means deleting and re-adding the source — which triggers a full re-index and a fresh embedding bill. Choose deliberately up front. Section-based chunking (splits on markdown headers) and fixed-length chunking are the predictable-cost options, and section-based is the natural fit for runbook-style documents that already have meaningful header structure.

How to find your Knowledge Base UUID

You’ll need the KB’s UUID for every API call, and it’s not prominent in the GUI. Three places to get it:

1. The browser URL on the Knowledge Base detail page — the UUID is the path segment
2. doctl gradient knowledge-base list
3. The auto-generated snippet on the Retrieve Endpoint tab, which includes it inline

Export it now, along with your token:

export DO_API_TOKEN="dop_v1_..."   # PAT with genai:read — not a Spaces key
export KB_UUID="your-kb-uuid"

Before touching Hermes, prove retrieval works in isolation. If retrieval is broken, then you want to find out now, and not while debugging an agent loop with four other moving parts.

The fastest check is the control panel’s Retrieve tab on your Knowledge Base: run a test query with num_results: 5 and alpha: 0.5, and you’ll see the returned chunks with relevance scores and source metadata.

The Retrieve Endpoint tab goes one better: it generates a working curl command for your current KB and settings. The raw call looks like this:

curl -X POST "https://kbaas.do-ai.run/v1/$KB_UUID/retrieve" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $DO_API_TOKEN" \
  -d '{
    "query": "what does FM-4419 mean",
    "num_results": 5,
    "alpha": 0.5
  }'

The response shape is worth understanding, because the plugin depends on it:

{
  "results": [
    {
      "text_content": "FM-4419 indicates a single-shard lease expiry in ledger-core...",
      "metadata": {
        "item_name": "ledger-core-errors.md",
        "page_number": 1
      },
      "score": 0.87
    }
  ],
  "total_results": 5
}

The field that matters most is metadata.item_name. That’s the source filename, and it’s the entire basis for citation later: if it survives from this response through the plugin into the model’s context, the model can name its sources. If it gets dropped in formatting, citation is impossible no matter how you prompt.

About alpha, the least obvious parameter: it sets the balance between lexical and semantic retrieval. 0 is keyword-only, 1 is semantic-only, and anything in between runs hybrid search. DigitalOcean’s guidance is to treat hybrid as the default and start somewhere in the 0.5–0.7 range — pure semantic drifts away from the exact query, pure keyword misses synonyms.

Our test questions pull in both directions, which is the interesting part. “What does FM-4419 mean” is an exact-token lookup, exactly the kind of product-code query DO’s docs suggest tuning toward alpha: 0. “Who should I page if it keeps happening” is conversational and wants meaning-based matching. One corpus serving both query shapes is precisely the case hybrid exists for, so we pin 0.5 in the plugin rather than letting the model pick per call. If your corpus is dominated by identifiers — error codes, SKUs, ticket numbers — tune down. If it’s prose, tune up.

The gate: run your test questions against this endpoint directly. If the right chunks don’t come back here, no amount of agent engineering will fix it — fix the corpus or the chunking strategy first, then continue.

There’s a zero-code path and a ~60-line path. Honesty first: the zero-code path works, and some readers should take it and stop.

Option A: The DigitalOcean MCP server (no code)

DigitalOcean ships an MCP server that includes Knowledge Base tools, and Hermes registers MCP tools directly from config.yaml with no Python involved. If you set up an MCP server in Part 1, this is the same motion:

# ~/.hermes/config.yaml
mcp_servers:
  digitalocean:
    command: npx
    args: ["-y", "@digitalocean/mcp"]
    env:
      DIGITALOCEAN_ACCESS_TOKEN: ${DO_API_TOKEN}

Restart Hermes, and the Knowledge Base tools appear alongside its built-ins. This genuinely works. If you want default retrieval behavior and don’t need control over parameters or formatting, stop here.

Option B: A custom Hermes plugin (~60 lines)

Why you’d write the plugin anyway — and why it’s the one you’ll want in production:

• Control over what the model sees. You write the tool description, and the description is what determines when the model reaches for the tool. With MCP, you get whatever description the server ships.
• Pinned retrieval parameters. alpha and num_results stay constants in your code instead of being LLM-chosen per call. Letting the model tune retrieval makes runs nondeterministic — the same question can retrieve differently on different runs.
• Control over result formatting. You decide exactly how chunks enter the prompt, which is the main lever on token cost per turn.
• No extra process. The plugin is an in-process HTTP call, not a server you have to keep running.
• It’s a template. Any HTTP API becomes a Hermes tool by the same four-file pattern.

MCP is the fast path. The plugin is the production path. The rest of this tutorial builds the plugin.

The plugin is four files:

~/.hermes/plugins/do-knowledge-base/
├── plugin.yaml      # manifest — what this is, what it needs
├── __init__.py      # register() — wires schema to handler
├── schemas.py       # what the LLM sees
└── tools.py         # what runs

plugin.yaml: declaring the manifest and required credentials

name: do-knowledge-base
version: 1.0.0
description: Retrieval over a DigitalOcean Knowledge Base
provides_tools: true
requires_env:
  - name: DO_API_TOKEN
    description: DigitalOcean personal access token with the genai:read scope
    url: https://cloud.digitalocean.com/account/api/tokens
    secret: true
  - name: KB_UUID
    description: UUID of the target Knowledge Base
    url: https://cloud.digitalocean.com/gen-ai/knowledge-bases

The part worth highlighting is requires_env. Declare your credentials here — with descriptions and URLs — and Hermes prompts for them at install time and writes them to ~/.hermes/.env. Marking the token secret: true masks the input. This is strictly better than hand-rolling environment checks in your handler, and it’s what makes the plugin installable by someone who isn’t you.

schemas.py: why the tool description determines when the model calls your tool

The description field is the product. It’s the only thing the model reads when deciding whether your tool is relevant to the current question. Vague description, unused tool.

KNOWLEDGE_BASE_RETRIEVAL_SCHEMA = {
    "name": "knowledge_base_retrieval",
    "description": (
        "Search the team's operational runbooks for the payments platform "          # names the domain
        "(atlas-ingest, ledger-core, payouts-service, webhook-relay). "
        "Use this for questions about error codes in the FM-#### format, "            # names the query format
        "escalation and on-call routing, service configuration, and "
        "version-specific behavior changes. "
        "Do NOT use this for general programming questions or anything "               # negative space
        "unrelated to the payments platform. "
        "Always cite the source file names returned in the results."                   # citation instruction
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "The question or search terms to retrieve documents for",
            }
        },
        "required": ["query"],
    },
}

Every clause is there for a reason: the first names the domain, the second names the error-code format (so the model recognizes FM-4419 as in-scope), the third carves out the negative space (what not to use the tool for), and the last instructs citation.

Notice what the schema deliberately does not expose: alpha, num_results, filters. Those are constants in tools.py, not model-controlled parameters. This is the determinism argument from Option B made concrete.

tools.py: the handler and how to format retrieval results for the agent loop

Four rules from the Hermes plugin docs, each worth internalizing:

1. The handler signature is (args: dict, **kwargs) -> str
2. Always return a JSON string, never a dict
3. Never raise — catch everything and return error JSON instead
4. Accept **kwargs for forward compatibility with future Hermes versions

import json
import os
import urllib.request

_ALPHA = 0.5
_NUM_RESULTS = 5
_MAX_CHUNK_CHARS = 1200
_RETRIEVE_URL = "https://kbaas.do-ai.run/v1/{kb_uuid}/retrieve"


def knowledge_base_retrieval(args: dict, **kwargs) -> str:
    try:
        query = args["query"]
        url = _RETRIEVE_URL.format(kb_uuid=os.environ["KB_UUID"])
        payload = json.dumps({
            "query": query,
            "num_results": _NUM_RESULTS,
            "alpha": _ALPHA,
        }).encode()
        req = urllib.request.Request(
            url,
            data=payload,
            headers={
                "Content-Type": "application/json",
                "Authorization": f"Bearer {os.environ['DO_API_TOKEN']}",
            },
        )
        with urllib.request.urlopen(req, timeout=15) as resp:
            data = json.load(resp)

        results = data.get("results", [])
        if not results:
            return json.dumps({
                "results": [],
                "note": "No relevant documents found in the knowledge base "
                        "for this query. Do not answer from general knowledge; "
                        "say the runbooks do not cover it."
            })

        formatted = [
            {
                "source": r.get("metadata", {}).get("item_name", "unknown"),
                "page_number": r.get("metadata", {}).get("page_number"),
                "excerpt": r.get("text_content", "")[:_MAX_CHUNK_CHARS],
            }
            for r in results
        ]
        return json.dumps({"results": formatted})

    except Exception as e:
        return json.dumps({"error": f"knowledge base retrieval failed: {e}"})

The interesting part is not the HTTP call — it’s the formatting, because formatting is where citation is won or lost:

• Each chunk is mapped to {source, page_number, excerpt}, so item_name survives into the model’s context under an unambiguous key. This is the plumbing that makes “always cite your sources” achievable rather than aspirational.
• Excerpts are truncated at _MAX_CHUNK_CHARS. Every character here is injected into the prompt on every retrieval turn; this constant is your token-cost lever.
• Empty results return an explicit message, not an empty structure. A bare [] invites the model to fill the void from general knowledge. An explicit “the runbooks don’t cover this” instruction gives it something to say instead.

__init__.py: registering the tool

from .schemas import KNOWLEDGE_BASE_RETRIEVAL_SCHEMA
from .tools import knowledge_base_retrieval


def register(ctx):
    ctx.register_tool(
        "knowledge_base_retrieval",
        "do-knowledge-base",
        KNOWLEDGE_BASE_RETRIEVAL_SCHEMA,
        knowledge_base_retrieval,
    )

That’s it. ctx.register_tool(name, toolset, schema, handler) wires the schema the model sees to the handler that runs.

Tuning knobs: _ALPHA, _MAX_CHUNK_CHARS, and num_results

All three sit as constants at the top of tools.py on purpose. They jointly control answer quality and how many tokens you inject per turn: more results and longer excerpts give the model more to work with and cost more on every retrieval; alpha shifts precision between exact-match and semantic queries. They’re worth tuning per corpus — but a full optimization matrix is a different article. The defaults above are reasonable for a runbook-style corpus.

Install and enable

mkdir -p ~/.hermes/plugins/do-knowledge-base
# copy the four files into it
hermes plugins doctor ~/.hermes/plugins/do-knowledge-base --ci
hermes plugins enable do-knowledge-base

Run plugins doctor before enabling — it exercises the real discovery, parse, and registration path, so it catches structural problems while they’re still cheap. plugins enable then triggers the requires_env prompts from the manifest and writes your credentials to ~/.hermes/.env.

Confirm the plugin loaded

Start Hermes and check:

hermes
/plugins

You should see:

✓ do-knowledge-base v1.0.0 (1 tools, 0 hooks)

Common errors and fixes (from the actual build)

Every row in this table is a problem we actually hit while building this tutorial.

One of these deserves its own callout. The no endpoints available: request rejected pre-queue error looks alarming and looks tool-related — you’ll see it right after installing the plugin and assume the plugin broke something. The tell that it’s provider-level, not tool-level: it fires on every API call, including auxiliary ones like session title generation. The cause is almost always a retired model slug. DigitalOcean actively retires older slugs, so a config that worked last month can stop working — re-verify with doctl serverless-inference models list whenever this appears.

This is the payoff, and the completion of the method we opened with: questions first, negative control, then the pipeline.

The three test questions

1. Two-document join: “What does error FM-4419 mean, and who should I page if it keeps happening?” — answerable only by combining the error reference and the escalation doc.
2. Three-document chain: “Why is a sustained burst of FM-4419 treated as a real problem now, when it wasn’t before? What’s the current lease timeout?” — requires the changelog, the service doc, and the error reference.
3. The scope trap: “FM-3350 is firing a lot right now — should the Riverbend on-call handle it?” — the correct answer is no. FM-3350 is a payouts-service error, and the escalation doc explicitly places payouts outside Riverbend’s scope. A model that free-associates off the error-code format will confidently route it wrong.

The negative control: Hermes without the plugin

With the plugin disabled, we asked question 3. The agent searched its available surfaces — including the local filesystem — found nothing on FM-3350 or the Riverbend rotation, and refused to guess:

“a wrong guess on ‘who to page’ during an incident would be worse than useless.”

It then asked where the knowledge base actually lives.

This transcript demonstrates two things. First, the corpus design worked: the questions are genuinely unguessable, so any correct answer later can only have come from retrieval. Second — worth noting on its own — Hermes degrades honestly. Faced with a question it can’t ground, it refused and asked for the source rather than confabulating. That’s the failure mode you want in an on-call agent.

Results with the plugin enabled

With the plugin enabled, the same questions produce a visible tool sequence — tool_search → tool_describe → knowledge_base_retrieval — followed by grounded answers:

• Q1: correctly distinguishes FM-4419 (single-shard lease expiry, self-recovering) from its neighbor FM-4420 (quorum lost, Sev-1), and routes the page to the payments-riverbend Pageloop key rather than naming an individual — exactly what the escalation doc specifies.
• Q2: correctly explains the 30s→90s lease timeout change and why it reclassified sustained FM-4419 bursts, and confirms the current value against two independent documents.
• Q3 passes the trap: the agent declines to route a payouts error to Riverbend and names Payouts Eng as the owning rotation instead. It read what it retrieved; it didn’t pattern-match the error prefix.

The detail to look at in every one of these answers: each names its source files. That’s metadata.item_name surviving from the Retrieve API, through the plugin’s {source, ...} formatting, into the model’s context. Citation here isn’t a prompt trick — it’s a plumbing outcome. If the plugin dropped item_name in formatting, no instruction in the schema could recover it.

What “validated” actually means for a RAG pipeline

Not “it answered correctly.” The pass condition is three-part:

1. Tool disabled → the model refuses or fails. (Proves the questions are unguessable.)
2. Tool enabled → the model answers correctly and cites its sources. (Proves retrieval happened and provenance survived.)
3. The trap question routes correctly. (Proves the model reads retrieved content rather than free-associating.)

That definition is reusable on any RAG stack. If your validation can’t distinguish a working pipeline from a knowledgeable model, it isn’t validation.

Once retrieval is working, a tempting optimization appears: “the retrieved chunks are doing the heavy lifting now — I’ll route these turns to a cheaper model.”

DigitalOcean’s own Inference Router documentation advises against mid-loop switching, for three reasons that compound:

• Tool-calling formats differ between models. An agent loop parses tool calls in the format its model emits; switching models mid-loop is a good way to break that parsing at the worst possible moment.
• Prefix-based KV caching invalidates on a model switch, forcing full recomputation of the context you’ve been accumulating all conversation.
• Cached input tokens are cheaper than fresh ones — so losing the cache raises the cost of exactly the thing you were trying to optimize.

The correct cost lever is the one already in your plugin: retrieval parameters. Fewer, better-targeted chunks (_NUM_RESULTS, _MAX_CHUNK_CHARS, a well-chosen _ALPHA) shrink the prompt on every turn, with none of the failure modes. Pick one model for the loop; tune what you feed it.

Directions to take this once the base loop is validated:

• Metadata filters. The Retrieve API supports starts_with on item_name and wildcard on file_id — which is where the Spaces folder prefix from earlier pays off. Scope a query to one team’s folder in a shared corpus.
• Reranking. Enable it on the KB’s Settings tab for better ordering of retrieved chunks. Note it bills separately from query vectorization.
• Scheduled re-indexing. Runbooks change; a KB indexed once drifts stale. Schedule re-indexing to keep retrieval current.
• The plugin as a template. The four-file pattern — manifest, schema, handler, registration — turns any HTTP API into a Hermes tool. The KB retrieval plugin is a worked example, not a special case.
• Multiple Knowledge Bases. Either register one tool per KB (distinct descriptions let the model choose), or add a kb_name parameter that maps to UUIDs in the handler.

Growing beyond a small corpus — millions of documents, high-throughput embedding, retrieval latency under load — is a different architecture problem, out of scope here.

The chain: Spaces → Knowledge Base → Retrieve API → Hermes plugin → DigitalOcean Serverless Inference. The actual work: four short files, one manifest, one HTTP call.

The method is the part worth carrying forward: write unguessable test questions first, capture the negative control, then build — and don’t call a RAG pipeline validated until the tool-disabled run fails, the tool-enabled run answers with citations, and the trap question routes correctly.

For Hermes users, the broader point is that the plugin surface is the general answer to “how do I connect Hermes to X.” The Knowledge Base was today’s X.

Resources:

• Companion repo — corpus, plugin source, test questions
• Part 1: How to Run Hermes Agent with DigitalOcean
• DigitalOcean Knowledge Bases documentation
• Nous Research Discord: #plugins-skills-and-skins

Does Hermes Agent support RAG out of the box?

No. Hermes ships with memory — agent-curated memory files, full-text search over past sessions, cross-session recall — but memory covers your conversations, not an external document corpus you own. Retrieval over your own documents requires connecting Hermes to a vector store or a managed knowledge base, either through the DigitalOcean MCP server or a custom plugin like the one built above.

Do I need to run a vector database to use RAG with Hermes?

No. A DigitalOcean Knowledge Base handles chunking, embedding, hybrid retrieval, and optional reranking, and provisions the backing OpenSearch database for you. Running Qdrant, pgvector, or Pinecone yourself buys more control over the index in exchange for owning the ingestion pipeline and the database — the plugin pattern in this tutorial works against any of them, since it’s just an HTTP call.

How do I know my RAG pipeline is actually retrieving anything?

Test it against a corpus the model can’t answer from general knowledge. Write your questions first, run them with retrieval disabled and confirm the model fails, then enable retrieval and re-run. A pipeline is validated when the tool-disabled run refuses, the tool-enabled run answers correctly and cites its sources, and a deliberate trap question routes correctly instead of pattern-matching.

What should I set alpha to when retrieving from a Knowledge Base?

The alpha parameter sets the balance between keyword and semantic retrieval: 0 is keyword-only, 1 is semantic-only. DigitalOcean recommends hybrid as the default, starting in the 0.5–0.7 range. Tune lower for corpora dominated by exact identifiers like error codes or SKUs, and higher for conversational prose.

Should I switch to a cheaper model once retrieval is doing the work?

Not mid-loop. Tool-calling formats differ between models, so switching can break how your agent parses tool calls, and a model switch invalidates prefix-based KV caching — which raises the cost of the thing you were optimizing. Reduce retrieval parameters instead: fewer chunks and shorter excerpts shrink the prompt on every turn with none of the failure modes.