Which RAGAS metrics can you run without ground truth?
Short answer: three of ten run with no reference at all, one more can if you pick a
different class, and one is commonly miscategorised.
But the useful answer is the table, because "do I need ground truth" is really "which
columns do I have". Every RAGAS metric declares its required columns, and if one is
missing the metric does not degrade — it refuses to run.
The table
Ten metrics, checked against the source in
src/ragas/metrics/
rather than against tutorials:
| Metric | Required columns | Reference? |
|---|---|---|
| Faithfulness | user_input, response, retrieved_contexts | No |
| Response Relevancy | user_input, response | No |
| Aspect Critic / Rubrics | all optional — you define the rubric | No |
| Context Precision (default) | user_input, retrieved_contexts, reference | Yes |
Context Precision (WithoutReference variant) | user_input, response, retrieved_contexts | No |
| Context Recall | user_input, retrieved_contexts, reference | Yes |
| Context Entities Recall | reference, retrieved_contexts | Yes |
| Answer Correctness | user_input, response, reference | Yes |
| Factual Correctness | response, reference | Yes |
| Semantic Similarity | response, reference | Yes |
| Noise Sensitivity | user_input, response, reference, retrieved_contexts | Yes |
Three things in that table cost me real time.
1. Context Entities Recall needs reference, not reference_contexts
The name sounds like it compares entity sets between gold chunks and retrieved chunks.
It does not. ContextEntityRecall takes the reference answer, extracts entities from
it, and measures what fraction of those entities appear in the retrieved chunks:
Context Entity Recall = |entities(retrieved) ∩ entities(reference)| / |entities(reference)| Its declared columns are {"reference", "retrieved_contexts"}. So if you have gold
chunks (reference_contexts) but no reference answer, this metric will not run —
and plenty of write-ups, including an earlier version of my own field table, say the
opposite.
NonLLMContextPrecisionWithReference is the metric that actually uses
reference_contexts, if that is what you were reaching for.
2. Context Precision has a reference-free variant
The default ContextPrecision requires a reference. But RAGAS also ships:
from ragas.metrics.collections import LLMContextPrecisionWithoutReference scorer = LLMContextPrecisionWithoutReference(llm=llm) result = await scorer.ascore( user_input="What is RAG?", response="RAG retrieves documents, then generates an answer.", retrieved_contexts=["RAG combines retrieval with generation."], ) It judges whether each retrieved chunk was useful in arriving at your own generated
answer, instead of at a gold answer. That makes it usable before you have any labels —
which is exactly the situation most teams are in when they first try to evaluate
retrieval.
ContextUtilization subclasses the same class if you want the score named for what it
measures.
3. Aspect Critic's columns are all optional
Aspect Critic declares every column as :optional:
{"user_input:optional", "response:optional", "retrieved_contexts:optional", "reference:optional", "reference_contexts:optional"} Because the metric is whatever criterion you write. A "is the answer polite" rubric needs
only response. A "does the answer cite the retrieved policy section" rubric needs
response and retrieved_contexts. The dependency is in your prompt, not in the schema,
so no static table can tell you whether your rubric will work — read your own criteria.
What "no reference" actually costs you
Being able to run a metric is not the same as being able to answer your question.
Without a reference you can measure:
- Groundedness — Faithfulness: is every claim in the answer supported by the chunks
- Relevance — Response Relevancy: does the answer address the question
- Chunk usefulness —
LLMContextPrecisionWithoutReference: were the retrieved chunks actually needed to produce this answer
What you cannot measure without a reference:
- Recall — did the retriever find everything it should have? Context Recall and Context Entities Recall both need a target to compare against.
- Correctness — Answer Correctness, Factual Correctness, Semantic Similarity all compare against a gold answer.
- Noise robustness — Noise Sensitivity needs to know which claims are wrong, which requires the truth.
Notice the shape of that split: precision can be judged against your own output, recall
cannot. If your retrieval returned three good chunks out of ten that existed, nothing in
the answer tells you about the seven you missed. That is not a RAGAS limitation, it is
information you do not have.
So the honest framing is not "how do I avoid ground truth" but "what can I learn before I
have it". The answer: whether your answer is faithful and on-topic, and whether your
chunks were useful. That is enough to catch a broken prompt or a badly-tuned retriever
top-k, and it is not enough to claim your system is accurate.
Checking your own dataset
Before running anything, verify the two things that fail silently rather than throwing:
import json data = json.load(open("eval_set.json")) # 1. every parallel array must be the same length lengths = {k: len(v) for k, v in data.items() if isinstance(v, list)} if len(set(lengths.values())) != 1: print("ragged arrays — rows will shift:", lengths) # 2. contexts must be a list of strings, not a string bad = [i for i, c in enumerate(data.get("retrieved_contexts", [])) if not isinstance(c, list)] if bad: print("contexts not a list at rows:", bad) Neither of these raises. Both produce scores that look plausible and mean nothing.
If you want this without the code
I built a converter that reads a dataset in any of the five common shapes (RAGAS modern,
current and legacy, TruLens, LangSmith), converts it, and reports which of these ten
metrics your dataset can run right now — naming the exact fields each missing metric
still needs.
It runs entirely in the browser: ragasconverter.com.
No upload, no signup, no server.
The standalone page for this topic, if you only want the table:
ragasconverter.com/ragas-without-ground-truth
Column requirements above are read from
src/ragas/metrics/
and cross-checked against the
available metrics docs.
These declarations have changed between releases — pin your RAGAS version before trusting
any table, including this one.
