Where does the Cosmo Router expose Prometheus metrics?

On http://127.0.0.1:8088/metrics by default. Change the address with telemetry.metrics.prometheus.listen_addr (PROMETHEUS_LISTEN_ADDR) and the path with path (PROMETHEUS_HTTP_PATH). Point any Prometheus-compatible scraper at it.

How do I expose the metrics endpoint from a container or Kubernetes pod?

The default listen address, 127.0.0.1:8088, is not reachable from outside the container. Set telemetry.metrics.prometheus.listen_addr to "0.0.0.0:8088", or set PROMETHEUS_LISTEN_ADDR=0.0.0.0:8088. Then scrape port 8088 on the /metrics path.

What metrics are exposed?

R.E.D. (rate, errors, and duration) for both router and subgraph requests. Core series include router_http_requests_total, router_http_request_duration_milliseconds, router_http_requests_error_total, router_http_requests_in_flight, and router_graphql_operation_planning_time.

Which Prometheus metrics should I track for Cosmo Router in production?

The metric reference recommends router_http_requests_total for traffic, router_http_requests_in_flight for concurrency, router_http_request_duration_milliseconds for latency, router_graphql_operation_planning_time for query planning, and router_info for config changes and uptime. Use router_http_requests_error_total for errors. For the process, it lists go_memstats_sys_bytes, go_memstats_heap_alloc_bytes, go_gc_duration_seconds, and go_goroutines.

Why don't I see any router_* metrics?

The router creates request metrics only after it serves a request. Send a query through the router, then reload /metrics. Until then you see only the default process and Go metrics. Opt-in families, such as connection or circuit breaker metrics, appear only after you turn them on.

Which labels do the request metrics carry?

Request metrics carry wg_operation_type, wg_operation_protocol, wg_client_name, and wg_client_version. Requests the router sends to subgraphs add wg_subgraph_name and wg_subgraph_id. router_http_requests_total also carries http_status_code. router_http_requests_in_flight carries only static and subgraph dimensions. wg_operation_name is left out unless the router exports to Cosmo Cloud or you add it with the operation_name context field under telemetry.metrics.attributes. Optional metrics for cache, engine, connection pool, and circuit breaker add their own labels.

How is label cardinality controlled?

The router applies a default cardinality limit of 2000 (telemetry.metrics.experiment_cardinality_limit, marked experimental). exclude_metrics and exclude_metric_labels take Go regex patterns and drop whole metrics or single labels. exclude_scope_info: true removes OpenTelemetry scope info. High-cardinality attributes such as the operation name stay off metrics unless the router exports to Cosmo Cloud or you turn them on.

How do I query p99 latency for a specific operation?

Use histogram_quantile() over router_http_request_duration_milliseconds_bucket, summed by le and wg_operation_name. Without Cosmo Cloud export, turn on the operation_name metric attribute first. Use the result in a Prometheus alerting rule.

Does the router's Prometheus endpoint include runtime metrics?

The router's Prometheus endpoint serves the default process and Go metrics, such as go_memstats_heap_alloc_bytes, go_gc_duration_seconds, and go_goroutines. These cannot be excluded. The router's own asynchronous runtime instruments (runtime.uptime, process.cpu.usage, process.runtime.go.*) are not on that endpoint. They go out through OTLP only. router_runtime controls them and is on by default.

Do I have to choose between OTEL and Prometheus?

No. Prometheus is built on the same OpenTelemetry foundation. Core R.E.D. metrics are available via both OTEL export and the Prometheus scrape endpoint at the same time. Asynchronous instruments, such as the router runtime and uptime metrics, are available via OTLP only.

Prometheus Metrics for Federated GraphQL | Cosmo by WunderGraph

The problem

Generic metrics cannot answer GraphQL questions

Operations teams need per-operation, per-subgraph, per-client metrics. Generic HTTP series do not carry those dimensions.

Generic HTTP metrics miss GraphQL

Generic HTTP metrics report request counts and durations without the dimensions that matter: operation name, operation type, client, and subgraph.

Subgraph latency is invisible

When a federated query slows down, the only metric is the aggregate request duration. Without per-subgraph series, you cannot tell which service is causing the spike.

Cardinality grows until something breaks

Adding GraphQL operation labels without limits can blow up the time-series database, forcing teams to drop labels.

Our solution

GraphQL-aware Prometheus metrics, built in

The router exports R.E.D. metrics for router and subgraph traffic with the GraphQL labels operators actually need. Cardinality is bounded out of the box.

What happens on every request

The router collects R.E.D. metrics via the OpenTelemetry SDK on every request.
A Prometheus exporter publishes them on http://127.0.0.1:8088/metrics by default.
Labels carry GraphQL context: wg_operation_type, wg_operation_protocol, wg_client_name, wg_client_version, wg_subgraph_name, wg_subgraph_id, and http_status_code. wg_operation_name is added when the router exports to Cosmo Cloud or you turn on the operation_name metric attribute.
Prometheus scrapes the endpoint on its configured interval and stores the series.
Grafana reads the series for dashboards. Prometheus alerting rules evaluate them for alerts.
A default cardinality limit of 2000 (experimental) and regex exclusions keep cardinality bounded.

Plug Prometheus into the endpoint. Dashboards and alerts follow.

Prometheus metrics

Before & After

Before Cosmo	With Cosmo
Generic HTTP metrics without operation or subgraph dimensions	R.E.D. metrics with wg_subgraph_name, wg_client_name, and related GraphQL labels
Aggregate request duration hides which subgraph is slow	Per-subgraph latency and error series on the same endpoint
High-cardinality labels overwhelm the time-series database	Default cardinality limit of 2000 plus regex exclusions for metrics and labels
Custom instrumentation to expose federation metrics	/metrics on :8088 by default: scrape and query

Optional metrics

Beyond R.E.D.

Cache. Hit and miss ratios, costs, and key statistics.
Engine. Connections, subscriptions, and triggers.
Connection pool. Utilization and acquisition duration.
Circuit breaker. State and short-circuits for the breakers you set in traffic shaping.

Each family is off by default. Turn it on under telemetry.metrics.prometheus.

How Prometheus metrics work in Cosmo Router

01

R.E.D. for router and every subgraph.

Emit

Rate, errors, and duration for router and subgraph requests. Default endpoint http://127.0.0.1:8088/metrics. No custom instrumentation required.

02

GraphQL dimensions on every series.

Label

Series carry wg_operation_type, wg_operation_protocol, wg_client_name, wg_client_version, wg_subgraph_name, wg_subgraph_id, and http_status_code. wg_operation_name is added when you export to Cosmo Cloud or turn it on as a metric attribute.

03

Default limit of 2000, regex exclusions.

Bound

A default cardinality limit of 2000 (telemetry.metrics.experiment_cardinality_limit, experimental) caps metric cardinality. Regex exclusions remove labels or whole metrics by pattern.

04

Standard PromQL for dashboards and alerts.

Alert

Use the metrics in Grafana dashboards and Prometheus alerting rules.

Telemetry controls

PromQL, cardinality, and config

Query by operation and subgraph. Bound label cardinality. Turn on the metric families you need.

p99 latency by operation

histogram_quantile(
  0.99,
  sum by (le, wg_operation_name) (
    rate(router_http_request_duration_milliseconds_bucket{wg_subgraph_name=""}[5m])
  )
)

Grouping by wg_operation_name needs Cosmo Cloud export or the operation_name metric attribute.

Error rate by subgraph

sum by (wg_subgraph_name) (
  rate(router_http_requests_error_total{wg_subgraph_name!=""}[5m])
)

Cardinality cap

A default cardinality limit of 2000, marked experimental. Regex exclusions remove labels or whole metrics by pattern.

Runtime metrics

The endpoint serves the default process and Go metrics, such as go_memstats_heap_alloc_bytes, go_gc_duration_seconds, and go_goroutines. The router's own runtime and uptime instruments are asynchronous, so they go out through OTLP export only.

Metrics worth alerting on

router_http_requests_error_total: Failed requests. A GraphQL response with status 200 can still count. Client disconnects do not.
router_http_request_duration_milliseconds: Request latency histogram. The top bucket boundary is 10s.
router_graphql_operation_planning_time: Query planning time. High values delay execution.
router_http_requests_in_flight: Concurrent requests. A high count points to overload.
router_info: Always 1 while the router runs. A missing series means the router is down.
router_circuit_breaker_state: 1 means the circuit is open. Opt-in.
go_goroutines: Steady growth can signal a goroutine leak.

router_info has one series for the base config plus one per feature flag (wg_feature_flag).

Opt-in metrics and exemplars

Each metric family below is off by default. exemplar_filter: trace_based links metric values to the sampled traces that produced them. The default is always_off.

telemetry:
  metrics:
    prometheus:
      listen_addr: "0.0.0.0:8088" # default 127.0.0.1:8088
      path: "/metrics"
      graphql_cache: true
      response_cache: true
      connection_stats: true
      circuit_breaker: true
      streams: true
      network:
        enabled: true
      resolver:
        enabled: true
      engine_stats:
        subscriptions: true
      cost_stats: # also needs security.cost_control.enabled: true
        estimated_enabled: true
        actual_enabled: true
      exemplar_filter: trace_based # default always_off

Scrape Cosmo Router today

The Prometheus endpoint is on by default. Add the scrape job and start querying.

Start Free Read the Docs

R.E.D. metrics for every GraphQL request and subgraph