I could see how long the model call took in an OpenTelemetry span. My metrics export showed a counter I’d added during the same run. I looked for the inference duration histogram and came up empty. The counter told me the metrics exporter worked.

I filed AI SDK issue #24229 with the reproduction. The histogram shipped in @ai-sdk/otel 1.0.160 through PR #24354. That gave me a way to check what gen_ai.client.inference.duration actually measures.

I ran 25 calls on each version, five each for three generation delays, a stream and an error. A local mock model supplied the delays with timers, and I flushed the in-memory export after every call. The older version recorded zero histogram observations. The new version recorded 25, including all five errors.

The histogram in 1.0.160. It contains 25 observations.

A span attribute does not create a metric

I started with ai 7.0.157 and @ai-sdk/otel 1.0.157. The reproduction printed this:

metric names: ["repro.control"] duration attribute present: true 

An empty export could have sent me back to the exporter setup. Seeing repro.control gave me a reason to look at the integration first.

The duration I saw was gen_ai.client.operation.duration, an attribute on the span. To query the distribution I needed the integration to emit a histogram.

Follow both paths

The span and the histogram take separate paths. A TracerProvider sends spans to a span exporter. The metric needs a MeterProvider, a reader and a metric exporter.

The export paths for the span and the histogram.

In my reproduction I registered new OpenTelemetry() before setting the global meter provider. That order works with the shipped code because it looks up the meter when it records the duration.

Keep the experiment local

I pinned ai and @ai-sdk/otel at 7.0.157 and 1.0.157 for the first run, then 7.0.160 and 1.0.160 for the second configuration. Both used OpenTelemetry API 1.9.1, metrics SDK 2.8.0, trace SDK 2.7.1 and Node 24.22.2.

Both configurations ran on the same Linux server. Within each one I called MockLanguageModelV4 sequentially and flushed after each attempt. I disabled retries and left CPU and network settings alone.

Group

Work inside the mock model

Repetitions

Short generation

Wait 20 ms, return text

5

Medium generation

Wait 80 ms, return text

5

Longer generation

Wait 320 ms, return text

5

Streaming

Wait 40 ms, emit text, wait 160 ms, finish

5

Failure

Wait 80 ms, throw an error

5

An 80 ms timer can run a little late. I wanted to see the groups separate in the export, so I used delays far enough apart to see that.

Give the exporter a control measurement

The in-memory exporter let me read the measurements directly. Here’s how I set it up after registering tracing and the integration.

 import { metrics } from '@opentelemetry/api'; import { AggregationTemporality, InMemoryMetricExporter, MeterProvider, PeriodicExportingMetricReader, } from '@opentelemetry/sdk-metrics'; const exporter = new InMemoryMetricExporter( AggregationTemporality.CUMULATIVE, ); const provider = new MeterProvider({ readers: [new PeriodicExportingMetricReader({ exporter, exportIntervalMillis: 60_000, })], }); metrics.setGlobalMeterProvider(provider); const control = metrics.getMeter('control') .createCounter('repro.control'); 

I incremented repro.control and flushed after every attempt. The issue reproduction has the tracing setup and mock model. To repeat the comparison, pin both AI SDK packages together.

Both counters ended at 25. Version 1.0.157 exported no inference histogram. Version 1.0.160 exported a histogram with 25 observations.

Check the recorded durations

After each call, I subtracted the previous cumulative sum for the same attribute set, checking that its count had increased by one. That gave me a duration in seconds for that call. I converted the medians to milliseconds for the table.

My measurements by mock group. Labels show the medians.

Group

Median recorded inference duration

20 ms generation

21.8 ms

80 ms generation

81.2 ms

320 ms generation

320.8 ms

40 + 160 ms stream

202.3 ms

80 ms failure

81.2 ms

The three generation groups landed close to their timer delays. The stream and the errors contributed their own measurements.

Let the stream finish

I had the mock wait 40 ms, emit text, then wait another 160 ms before closing. The caller consumed all of textStream.

First text reached my caller at a median of 43.6 ms across five calls. The histogram recorded a median of 202.3 ms for the full invocation. Those clocks start in different places, so I’d track time to first text as its own measurement.

First text at the caller and full inference duration.

For future reproductions I’d save the GenAI semantic convention revision alongside the package versions.

Include the calls that fail

The failing mock waited 80 ms and threw Error('controlled failure'). All five attempts were recorded with retries disabled.

I want to see those failures when I’m checking latency. The PR and its tests also cover cancellation and failed stream attempts during retries. My test exercised a thrown error.

The count is model attempts. One user action can cause several model calls, and retries can add more attempts. Local tool execution time is outside this duration.

Read the attributes beside the value

The error point had the operation and requested model alongside error.type. The provider was normalized from openai.chat to openai. The point can also carry a response model when the integration has one.

I chose the model IDs as labels for the mock groups. The exported point had no prompt or response text.

The updated integration docs show that telemetry.isEnabled: false turns off metrics for the call. The payload controls recordInputs and recordOutputs leave duration recording on. This histogram comes from the new integration.

Read cumulative counts once

I flushed after each attempt and chose cumulative aggregation. By the fifth flush, the snapshot for that group already included the previous four calls.

Five calls in the medium group. Adding each count gives 15.

The export puts bucket counts into discrete intervals. I’d check how my backend represents them before writing a percentile query. The integration suggests boundaries in seconds, and the metrics SDK can override them.

I’d start the same way on a real service. Check the control counter and one successful call, then read the metric name, unit, attributes, count and sum in the export. Try one failed call and let one stream close. I would have missed that last 160 ms if I’d stopped reading at the first chunk.