Prometheus

@voltro/plugin-prometheus exposes the framework's metrics to Prometheus / Grafana at a scrape endpoint. It only READS — the framework emits the metrics automatically; the plugin formats them.

#The Metrics-API (single source of truth)

The framework records metrics into Effect's global MetricRegistry. One snapshot sees everything:

• Core RPC — voltro_rpc_requests_total{tag,status}, voltro_rpc_errors_total{tag}, voltro_rpc_duration_seconds{tag} (histogram). Emitted at the mutation / action handler boundary.
• HTTP routes — voltro_http_requests_total{route,status}, voltro_http_duration_seconds{route}.
• Plugin interceptors — voltro_plugin_hook_duration_seconds{hook}, voltro_plugin_hook_errors_total{hook}.
• Subscriptions — voltro_subscriptions_active{tag} (gauge of currently-open subscriptions), voltro_subscription_deliveries_total{tag,kind} + voltro_subscription_delivery_seconds{tag,kind} (per-delivery produce→push latency; kind = snapshot | delta). Backpressure & resume: voltro_subscription_buffered_bytes{tag} (gauge of pending bytes per blocked subscription), voltro_subscription_coalesced_total{tag} (updates collapsed onto the newest state while a consumer was blocked), voltro_subscription_overrun_total{tag} (streams closed with SubscriptionOverrun), voltro_subscription_oversized_total{tag} (events over reactive.socket.oversizedEventBytes — telemetry, not a cap).
• Schedules (crons) — voltro_schedule_runs_total{schedule,status} (firings by name + outcome — status = succeeded | failed), voltro_schedule_duration_seconds{schedule} (histogram), and voltro_schedule_last_success_timestamp_seconds{schedule} (a gauge holding the UNIX time of the last SUCCESS). Emitted by the framework scheduler, so every cron gets them with no per-handler wiring. A cron fires unattended — the failure mode is silent — so this is the series to alert on: time() - voltro_schedule_last_success_timestamp_seconds{schedule="…"} > <interval × N> fires when a job stops succeeding (a failure counter alone can't catch a job that stopped firing at all, but the last-success gauge going stale does). A failure moves the counter but deliberately NOT the gauge.
• Workflows (durable execution) — voltro_workflow_runs_total{workflow,status} (terminal outcomes — status = succeeded | failed), voltro_workflow_duration_seconds{workflow} (histogram), and voltro_workflow_last_success_timestamp_seconds{workflow} (last-success gauge). Emitted by the workflow run-recording seam. Because the framework applies no retry of its own, a failed run is terminal — it is the dead-letter state — so voltro_workflow_runs_total{status="failed"} is the dead-letter rate, and the last-success gauge going stale is the "this workflow stopped completing" alert (same shape as the schedule alert). A failure moves the counter but not the gauge.
• Partial prerendering (web) — voltro_ppr_shell_serves_total{page}, voltro_ppr_hole_passes_total{page}, voltro_ppr_hole_settles_total{page}, voltro_ppr_hole_errors_total{page} and voltro_ppr_hole_pass_seconds{page} (histogram). page is the DECLARED route pattern (/blog/[slug]), never a resolved URL — a label that grows with visitors is how a scrape target falls over. voltro dev and voltro start emit the same set. voltro_ppr_hole_errors_total is the alert: the shell already went out with a 200, so a failed hole pass leaves every <Await> boundary on its fallback and nothing else says so.
• Queues — voltro_queue_consumed_total{topic,outcome} (outcome = ok | dead-lettered; the two together are every message the runner finished with, so the dead-letter RATE is a division with no join), voltro_queue_retries_total{topic}, voltro_queue_produced_total{topic}, and voltro_queue_lag_messages{topic,partition} — a gauge of the backlog behind the message just picked up, read out of the fetch response rather than an admin round trip. A message abandoned by a rebalance is deliberately in no outcome: its new owner redelivers and counts it there.
• @voltro/cache counters, Effect's own effect_fiber_* runtime metrics, and any custom metric you or another plugin defines.

Two consumers read the SAME snapshot, so they never disagree:

• GET /metrics (this plugin) → Prometheus text exposition format, consumed by Prometheus / Grafana.
• GET /_voltro/inspect/metrics → JSON MetricSample[], consumed by the dashboard's Metrics panel (quantiles derived from the histogram buckets, histogram_quantile-style).

#Custom metrics

Define your own counter / gauge / histogram and it shows up in both consumers automatically:

import { counter, gauge, histogramMetric } from '@voltro/runtime'
import { Effect, Metric } from 'effect'

const signups = counter('app_signups_total', 'User sign-ups.')

export default (input, _ctx) =>
  Effect.gen(function* () {
    // … create the user …
    yield* Metric.increment(signups)
    return { ok: true }
  })

#Wiring

// app.config.ts
import { prometheusPlugin } from '@voltro/plugin-prometheus'

export default {
  type: 'api' as const,
  name: 'api',
  plugins: [prometheusPlugin()],
}

Scrape config — one target per replica, never a single load-balanced hostname:

scrape_configs:
  - job_name: voltro
    metrics_path: /metrics
    static_configs:
      # one target per replica — NOT one LB-fronted hostname
      - targets: ['api-0.internal:4000', 'api-1.internal:4000']

#Multi-replica scraping

The registry is per-process: each replica serves its own counters, and Prometheus keeps the series apart via the per-target instance label — exactly the pull model. But that only works when every replica is scraped directly. Behind a load-balancer, successive scrapes hit different processes and the per-process counters interleave into one incoherent series. On Kubernetes, use pod-level discovery so every pod becomes its own target:

scrape_configs:
  - job_name: voltro
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: 'true'

The plugin's name option disambiguates multiple plugin instances within one process (it suffixes the plugin name) — it does not identify replicas; replica identity is the scrape target's instance label.

#Options

prometheusPlugin({
  path: '/metrics',          // scrape path (default /metrics)
  token: process.env.PROMETHEUS_TOKEN,  // require `Authorization: Bearer <token>`
  processMetrics: true,      // include node process_* metrics (default true)
  version: '1.4.2',          // sets voltro_build_info{version} = 1
})

• token — defaults to the PROMETHEUS_TOKEN env. Unset → the endpoint is open; gate it at the network layer (Prometheus uses bearer_token to send it).
• processMetrics — emits process_resident_memory_bytes, process_heap_bytes, process_cpu_seconds_total.
• The response is text/plain; version=0.0.4; charset=utf-8 with cache-control: no-store.

#Output shape

Counters / gauges render as name{labels} value. Histograms render as cumulative name_bucket{labels,le="…"} series plus name_sum + name_count + the +Inf bucket — exactly what histogram_quantile() expects in Grafana.

# HELP voltro_rpc_requests_total Total RPC handler invocations.
# TYPE voltro_rpc_requests_total counter
voltro_rpc_requests_total{status="ok",tag="mutation.todos.create"} 42
# TYPE voltro_rpc_duration_seconds histogram
voltro_rpc_duration_seconds_bucket{tag="mutation.todos.create",le="0.001"} 3
voltro_rpc_duration_seconds_bucket{tag="mutation.todos.create",le="+Inf"} 42
voltro_rpc_duration_seconds_sum{tag="mutation.todos.create"} 0.21
voltro_rpc_duration_seconds_count{tag="mutation.todos.create"} 42

#Scope

Mutation / action / HTTP / plugin metrics, subscription metrics (active gauge + per-delivery latency histogram, split snapshot vs delta), schedule/cron metrics AND workflow metrics (run counter + duration + last-success gauge each) are all covered. OTLP metrics export reads the same registry: the OTEL_EXPORTER_OTLP_ENDPOINT that enables trace export also exports metrics (@effect/opentelemetry bridges the registry into the OTel MeterProvider) — that is the path for the Datadog Agent / any OTLP collector; agentless Datadog is @voltro/plugin-datadog. This plugin is specifically the Prometheus pull (scrape) surface.