Enterprise analytics agents fail in ways that are hard to catch. A query might return plausible-looking numbers from the wrong table. A multi-step workflow might skip a validation step. An agent might hallucinate a column name that happens to exist in a different schema.

A new comparative study from KDD 2026 tested three evaluation paradigms on two analytics agents using frozen execution traces. The researchers wanted to know which approach catches what kinds of failures, how stable each framework is across repeated runs, and whether any single paradigm can cover both functional correctness and security risks.

The answer: no single framework does it all. Trace-based evaluation catches subtle functional errors. Outcome-based evaluation is simpler but noisier. LLM-judge frameworks surface failures human reviewers miss, but only when they have the right inputs. Security red-teaming finds a completely different set of problems.

The Three Evaluation Paradigms

The study compared three distinct approaches:

Trace-based LLM judging inspects the full execution trace: tool calls, intermediate outputs, state transitions, and final results. The evaluator sees what the agent did, not just what it produced.

Outcome-based LLM judging (text-only) evaluates final outputs without access to intermediate steps. The evaluator sees the query and the answer, but not the tool calls or reasoning chain.

Deterministic heuristics use rule-based checks: schema validation, query syntax checks, output format verification. No LLM, no ambiguity, but also no semantic understanding.

Each paradigm makes different trade-offs between information access, stability, and coverage.

Phase 1: Stability Across Repeated Runs

The researchers ran each evaluation framework multiple times on the same frozen traces to measure stability. Non-deterministic LLM judges can produce different verdicts on identical inputs, which breaks CI/CD pipelines and makes regression detection unreliable.

Results:

  • 97% of configurations met the stability threshold (defined as consistent verdicts across repeated runs)
  • Stability patterns were agent-independent, meaning the same framework behaved consistently across different agent architectures
  • Trace-based evaluation was slightly more stable than outcome-based evaluation, likely because richer context reduces ambiguity

The stability finding matters for production deployment. If your eval framework flips verdicts on the same trace, you cannot trust it to catch regressions. The 97% threshold suggests that with proper prompt engineering and temperature settings, LLM-based evals can be stable enough for CI.

Phase 2: Known Risk Detection

The team created 72 ground-truth test cases with labeled failures: wrong table joins, missing filters, incorrect aggregations, schema violations. They measured precision (how many flagged failures were real) and recall (how many real failures were caught).

Framework Precision Recall F1 Score False Alarms
Trace-based LLM 0.92 0.88 0.90 Low
Outcome-based LLM 0.78 0.88 0.83 5x higher
Deterministic heuristics 0.95 0.62 0.75 Very low

Trace-based evaluation achieved the best precision-recall balance. It caught 88% of known failures with only 8% false positives. The richer context (tool calls, intermediate state) let the LLM judge distinguish between "wrong answer for the right reason" and "right answer for the wrong reason."

Outcome-based evaluation matched on recall but generated five times more false alarms. Without visibility into the reasoning chain, the evaluator flagged many correct outputs as suspicious because it could not verify the logic.

Deterministic heuristics had excellent precision but missed 38% of failures. Schema checks catch syntax errors but not semantic mistakes like joining on the wrong foreign key.

Phase 3: Unknown Risk Discovery

The researchers ran both functional testing (trace-based evaluation) and security red-teaming (adversarial prompting with outcome-based evaluation) to discover novel failures not in the ground-truth set.

They found 24 new failure modes with zero overlap between the two approaches:

Functional testing uncovered:

  • Edge cases in date range handling
  • Incorrect null handling in aggregations
  • Subtle logic errors in multi-step workflows

Security red-teaming uncovered:

  • Prompt injection bypassing access controls
  • Data exfiltration through error messages
  • Privilege escalation via tool chaining

The zero overlap is the key finding. Functional correctness and security validation are orthogonal concerns. An agent can produce correct analytics while leaking sensitive data. An agent can enforce access controls while returning wrong numbers.

Architecture: Instrumenting Multi-Tool Workflows

Trace-based evaluation requires deep instrumentation. Here is what you need to capture:

@dataclass class AgentTrace: query: str plan: List[Step] tool_calls: List[ToolInvocation] intermediate_outputs: Dict[str, Any] final_result: Any metadata: Dict[str, Any] @dataclass class ToolInvocation: tool_name: str input_args: Dict[str, Any] output: Any timestamp: float error: Optional[str] # Critical for evaluation context_before: Dict[str, Any] context_after: Dict[str, Any]

The context_before and context_after fields capture state transitions. Did the agent update its belief about which table to query? Did it realize it needed an additional join? This state delta is what lets trace-based evaluators catch reasoning errors.

For outcome-based evaluation, you only need:

@dataclass class OutcomeTrace: query: str final_result: Any expected_schema: Optional[Dict]

Simpler to instrument, but you lose visibility into how the agent arrived at the answer.

Observability Boundaries

The study reveals a fundamental tension in agent observability. More context improves evaluation quality but increases coupling between the agent and the eval framework.

Trace-based evaluation requires:

  • Hooks into the orchestration layer to capture tool calls
  • Access to intermediate state (working memory, context updates)
  • Serialization of complex objects (DataFrames, query plans)

This creates tight coupling. If you change the agent's internal representation, you break the eval harness.

Outcome-based evaluation requires:

  • Only the final output and the original query
  • No access to internal state

This is loosely coupled but blind to process. You cannot distinguish between an agent that got lucky and one that reasoned correctly.

The researchers recommend a hybrid approach: use trace-based evaluation for functional testing (where you control both the agent and the eval framework) and outcome-based evaluation for security red-teaming (where adversarial independence matters).

Failure Modes and Detection Rates

Different evaluation paradigms catch different failure modes:

Trace-based LLM judging excels at:

  • Incorrect tool selection (agent chose query_sales instead of query_inventory)
  • Missing validation steps (agent skipped null check before aggregation)
  • Logic errors in multi-step plans (agent joined tables in wrong order)

Outcome-based LLM judging excels at:

  • Semantic correctness of final answers (numbers make sense given the query)
  • Adversarial robustness (agent resists prompt injection)
  • Output format violations (agent returned JSON when CSV was expected)

Deterministic heuristics excel at:

  • Schema violations (agent referenced non-existent column)
  • Type mismatches (agent passed string to numeric function)
  • Syntax errors (agent generated malformed SQL)

The study found that combining trace-based evaluation with deterministic heuristics catches 94% of functional failures. Adding security red-teaming brings total coverage to 98%, but only because security failures are orthogonal to functional failures.

When LLM Judges Beat Human Reviewers

The researchers had human experts manually review a subset of traces. LLM-based evaluators flagged 18 failures that human reviewers missed. These were subtle errors:

  • Using last month's exchange rate instead of the transaction date rate
  • Aggregating before filtering, which changed the result by 0.3%
  • Returning the correct number but from a cached result instead of fresh data

Human reviewers focused on "does this number look right?" LLM judges with access to traces could verify "did the agent compute this number correctly?"

This finding has implications for production monitoring. If you rely on human spot-checks, you will miss subtle correctness issues. If you rely on outcome-based evaluation, you will miss process failures that happen to produce correct outputs.

Deployment Shape

The study used frozen execution traces, which means they replayed recorded agent runs rather than executing live queries. This is the right choice for evaluation infrastructure:

Frozen traces enable:

  • Deterministic replay (same inputs always produce same outputs)
  • Fast iteration (no waiting for database queries)
  • Safe testing (no risk of corrupting production data)

Frozen traces require:

  • Trace capture infrastructure in production
  • Storage for large trace objects (DataFrames, query plans)
  • Versioning to handle schema evolution

The researchers stored traces in a structured format (likely Parquet or JSON Lines) with metadata for filtering and sampling. This lets you run evals on representative subsets without processing every trace.

For production deployment, you need two evaluation pipelines:

  1. CI/CD pipeline: Run trace-based evaluation on frozen traces from your test suite. Fast feedback, high coverage.
  2. Production monitoring: Run outcome-based evaluation on live traffic samples. Catch drift, adversarial inputs, and edge cases not in your test suite.

Cost and Latency Trade-offs

LLM-based evaluation is expensive. The study does not report costs, but we can estimate:

  • Trace-based evaluation: 2-5K tokens per evaluation (full trace context)
  • Outcome-based evaluation: 500-1K tokens per evaluation (query + result)
  • Deterministic heuristics: negligible cost

At $0.01 per 1K tokens (GPT-4 class model), evaluating 1,000 traces costs:

  • Trace-based: $20-50
  • Outcome-based: $5-10
  • Deterministic: $0

For CI/CD, this is acceptable. For production monitoring at scale, you need sampling strategies. The study suggests:

  • Run deterministic heuristics on 100% of traffic (cheap, fast)
  • Run outcome-based LLM evaluation on 10% of traffic (catch semantic errors)
  • Run trace-based LLM evaluation on 1% of traffic or on flagged failures (deep diagnosis)

Security Implications

The zero overlap between functional testing and security red-teaming is a warning. You cannot assume that an agent which passes functional tests is secure.

Security red-teaming found:

  • Prompt injection: Adversarial queries that trick the agent into ignoring access controls
  • Data exfiltration: Queries that leak sensitive data through error messages or intermediate outputs
  • Privilege escalation: Tool chaining that bypasses authorization checks

None of these failures were caught by trace-based functional evaluation because they involve adversarial inputs not in the test suite. Security evaluation requires:

  • Adversarial prompt generation (either manual or LLM-generated)
  • Outcome-based judging (to maintain adversarial independence)
  • Separate test cases focused on access control, data leakage, and privilege boundaries

The study recommends running security red-teaming as a separate pipeline with different success criteria. Functional tests ask "does this work correctly?" Security tests ask "can this be abused?"

Technical Verdict

Use trace-based LLM evaluation when:

  • You need high precision on functional correctness
  • You control both the agent and the eval framework (tight coupling is acceptable)
  • You are debugging specific failures and need visibility into reasoning chains
  • You have budget for 2-5K token evaluations

Use outcome-based LLM evaluation when:

  • You need adversarial independence (security testing)
  • You are evaluating third-party agents or black-box systems
  • You want simpler instrumentation and looser coupling
  • You can tolerate higher false alarm rates

Use deterministic heuristics when:

  • You need fast, cheap checks on 100% of traffic
  • You are catching schema violations, syntax errors, and type mismatches
  • You want zero false positives (high precision, lower recall)

Avoid relying on a single paradigm. The study shows that functional and security evaluation are complementary. A production-grade evaluation strategy needs both trace-based functional testing and outcome-based security red-teaming, plus deterministic heuristics for fast filtering.

The 97% stability threshold means LLM-based evals are ready for CI/CD, but you need careful prompt engineering and temperature tuning. The 5x false alarm rate for outcome-based evaluation means you need human review capacity or a second-stage filter.

If you are deploying analytics agents in enterprise settings, budget for two evaluation frameworks and accept that neither will catch everything.

Source Links