Overview

Every framework primitive emits OpenTelemetry spans when tracing is enabled. You don't instrument anything by hand — the handler boundaries, the transactional scope, and the streaming deliveries are all traced automatically, and the spans nest into a single waterfall per request.

#Auto-emitted spans

Span	What it marks
client.mutation.<rpcTag> / client.subscription.<rpcTag>	The browser-side span that roots the trace and sets the shared traceId.
mutation.<rpcTag>	The rpc mutation-handler boundary.
action.<rpcTag>	The unary action-handler boundary.
subscription.<rpcTag>	The streaming-rpc handler setup (subscribe → first snapshot). NOT the open-duration.
subscription.<rpcTag>.snapshot / .delta	Each data delivery to the subscriber, with its real produce→push latency.
store.transactional	The postgres BEGIN/COMMIT (or retry) scope. Carries db.system=postgresql + db.operation=transaction.
webhook<path>	An inbound *.webhook.tsx route (an incoming HTTP handler, not a query). Continues the caller's trace when the request carries a traceparent header.

#Span attributes

Handler spans carry:

• rpc.tag — the rpc tag of the call
• subject.type — the authenticated subject type
• tenant.id — the active tenant

@effect/rpc's own RpcServer.<tag> transport span is suppressed from the trace view — the framework spans carry the real latencies. For streaming subscriptions the transport span would just be a misleading open-duration bar, so the per-delivery .snapshot / .delta spans surface the real latency instead.

#Nesting

Spans nest. A mutation handler span wraps the store.transactional span, which wraps any Effect.withSpan you add inside the executor:

client.mutation.createOrder
└─ mutation.createOrder           rpc.tag, subject.type, tenant.id
   └─ store.transactional         db.system=postgresql, db.operation=transaction
      └─ createOrder.charge-card  (your custom span)

#Enabling modes

Three modes, env-driven, no code changes:

# Default — no EXPORTER. `voltro dev` still runs a buffer-only tracer
# (powers the Traces dashboards + traceId-in-logs); nothing is shipped
# off-box. `VOLTRO_TRACING_BUFFER=off` disables even that.
unset FRAMEWORK_TRACING OTEL_EXPORTER_OTLP_ENDPOINT

# Console (local dev — spans printed to stdout)
FRAMEWORK_TRACING=console voltro dev .

# OTLP (production — sends to any OTLP/HTTP collector: Jaeger, Tempo, Honeycomb, etc.)
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 voltro dev .
# OTEL_SERVICE_NAME defaults to 'voltro-api'; override per app:
OTEL_SERVICE_NAME=my-api OTEL_EXPORTER_OTLP_ENDPOINT=... voltro dev .

The framework auto-detects: if any standard OpenTelemetry env-var (OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT) is set, the mode resolves to otlp automatically. Set FRAMEWORK_TRACING explicitly (console / otlp / off) to force a mode.

Even with no exporter, voltro dev installs a buffer-only tracer — that's what powers the DevTools / cloud Traces panel and the trace id stamped on every log line. Set VOLTRO_TRACING_BUFFER=off to disable it entirely.

#Metrics export

Separate from tracing, the framework records metrics into Effect's global MetricRegistry — one source of truth (voltro_rpc_*, voltro_http_*, voltro_plugin_hook_*, voltro_subscription_* + voltro_subscriptions_active, plus effect_fiber_* and any custom metric). Three ways to get them out:

# OTLP metrics — the SAME OTEL endpoint that enables trace export also enables
# metrics. Ships to any OTLP/HTTP collector (Prometheus OTLP, Grafana Agent,
# the Datadog Agent's OTLP port, etc.). @effect/opentelemetry auto-bridges the
# Effect MetricRegistry into the OTel MeterProvider — no per-metric wiring.
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 voltro start
# metrics-only endpoint (no traces):
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://collector:4318/v1/metrics voltro start
# console (local dev — periodic metric dump to stdout):
FRAMEWORK_METRICS=console voltro dev .
# export cadence (ms): OTEL_METRIC_EXPORT_INTERVAL (default 60000).

The framework auto-detects: OTEL_EXPORTER_OTLP_ENDPOINT (or the metrics-specific endpoint) → metrics export otlp; force with FRAMEWORK_METRICS (console / otlp / off).

Two pull/push consumers read the SAME registry, so they can't disagree:

• @voltro/plugin-prometheus — GET /metrics scrape endpoint (Prometheus / Grafana).
• @voltro/plugin-datadog — agentless push to Datadog's /api/v2/series (for setups without an Agent / OTLP collector). With an Agent, prefer pointing OTEL_EXPORTER_OTLP_ENDPOINT at it instead.
• The dashboard Metrics panel reads the same snapshot via GET /_voltro/inspect/metrics.

#Routing traces to a vendor

The env-driven OTLP path above ships traces to any OTLP/HTTP collector. For deep, opt-in vendor integration — install one plugin, get traces + errors + logs all correlated by the same traceId, zero OTEL_* env — a plugin can contribute to the framework's tracer directly via contributeObservability:

interface ObservabilityContribution {
  resourceAttributes?: Record<string, string>
  spanProcessors?: ReadonlyArray<unknown>   // OTel SpanProcessor — receives every framework span
  metricReaders?: ReadonlyArray<unknown>    // OTel MetricReader
  sampler?: unknown                          // OTel Sampler (use with care — see below)
}

The CLI gathers every plugin's contribution at boot and passes the merged span-processors / metric-readers / resource-attributes into the framework's NodeSdk tracer. The vendor SDK runs as an OTel consumer — it gets a SpanProcessor alongside the framework's own buffer sink; it never becomes the global provider. So the in-app Traces dashboard keeps working and the vendor receives the identical spans.

This is how @voltro/plugin-sentry (SentrySpanProcessor) and @voltro/plugin-datadog (OTLPTraceExporter → DD Agent) route traces with no env fiddling.

Don't contribute a sampler that drops spans. The framework's buffer sink (powering the in-app Traces dashboard) is a SpanProcessor — it only sees recorded spans. A vendor sampler set as the tracer's sampler gates recording for the WHOLE tracer, blinding the dashboard. The framework stays always-on; vendors sample at their own export layer (e.g. Sentry's tracesSampleRate is applied by the Sentry client, not a tracer sampler).

To attach an error to the exact active span, the interceptor context carries both ctx.traceId and ctx.spanId — plugins read them explicitly (the active OTel span lives in Effect's fiber context, not the AsyncLocalStorage vendor SDKs read implicitly).

#Adding your own spans

Inside a mutation / action / workflow executor, just use Effect:

import { Effect } from 'effect'

export const myAction = defineAction({ ... })

export default (input, ctx) =>
  Effect.gen(function* () {
    yield* someExpensiveWork.pipe(
      Effect.withSpan('myAction.expensive-work', {
        attributes: { 'input.id': input.id },
      }),
    )
    return { ok: true }
  })

The span nests under the auto-emitted action.myAction span. In synchronous (non-Effect) executors, ad-hoc spans require explicitly adopting the Effect runtime — usually not worth it; rely on the auto spans the framework emits at each handler boundary.

Continue to Distributed tracing for how one traceId flows frontend → api → api, and Traces & logs from the shell for the voltro traces / voltro logs workflow.